[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:55:27.169364 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:34439
I20260709 12:55:27.171348 12409 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:55:27.172608 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:27.190939 12416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:27.190981 12418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:27.190973 12415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:27.192166 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:28.430971 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:28.431231 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:28.431396 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601728431378 us; error 0 us; skew 500 ppm
I20260709 12:55:28.432294 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:28.439620 12409 webserver.cc:533] Webserver started at http://127.12.30.126:40951/ using document root <none> and password file <none>
I20260709 12:55:28.440742 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:28.440965 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:28.441404 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:28.447400 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "b2f0cdf666794e69b2e20ac775b719f0"
format_stamp: "Formatted at 2026-07-09 12:55:28 on dist-test-slave-mlt7"
I20260709 12:55:28.455904 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:55:28.461524 12424 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:28.462880 12409 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:55:28.463218 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "b2f0cdf666794e69b2e20ac775b719f0"
format_stamp: "Formatted at 2026-07-09 12:55:28 on dist-test-slave-mlt7"
I20260709 12:55:28.463557 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:28.532271 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:28.534075 12409 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:55:28.534562 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:28.615754 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:34439
I20260709 12:55:28.615829 12475 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:34439 every 8 connection(s)
I20260709 12:55:28.624934 12476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:28.654052 12476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0: Bootstrap starting.
I20260709 12:55:28.660629 12476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:28.663776 12476 log.cc:826] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0: Log is configured to *not* fsync() on all Append() calls
I20260709 12:55:28.669570 12476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0: No bootstrap required, opened a new log
I20260709 12:55:28.692798 12476 raft_consensus.cc:359] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f0cdf666794e69b2e20ac775b719f0" member_type: VOTER }
I20260709 12:55:28.693392 12476 raft_consensus.cc:385] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:28.693608 12476 raft_consensus.cc:740] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f0cdf666794e69b2e20ac775b719f0, State: Initialized, Role: FOLLOWER
I20260709 12:55:28.694486 12476 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f0cdf666794e69b2e20ac775b719f0" member_type: VOTER }
I20260709 12:55:28.694976 12476 raft_consensus.cc:399] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:55:28.695230 12476 raft_consensus.cc:493] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:55:28.695525 12476 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:28.700146 12476 raft_consensus.cc:515] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f0cdf666794e69b2e20ac775b719f0" member_type: VOTER }
I20260709 12:55:28.701090 12476 leader_election.cc:304] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2f0cdf666794e69b2e20ac775b719f0; no voters: 
I20260709 12:55:28.702986 12476 leader_election.cc:290] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:55:28.703336 12479 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:28.705940 12479 raft_consensus.cc:697] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 1 LEADER]: Becoming Leader. State: Replica: b2f0cdf666794e69b2e20ac775b719f0, State: Running, Role: LEADER
I20260709 12:55:28.706961 12479 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f0cdf666794e69b2e20ac775b719f0" member_type: VOTER }
I20260709 12:55:28.708055 12476 sys_catalog.cc:565] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:55:28.718606 12481 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2f0cdf666794e69b2e20ac775b719f0. Latest consensus state: current_term: 1 leader_uuid: "b2f0cdf666794e69b2e20ac775b719f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f0cdf666794e69b2e20ac775b719f0" member_type: VOTER } }
I20260709 12:55:28.718375 12480 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2f0cdf666794e69b2e20ac775b719f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f0cdf666794e69b2e20ac775b719f0" member_type: VOTER } }
I20260709 12:55:28.719194 12480 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:55:28.719156 12481 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:55:28.723011 12488 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:55:28.736691 12488 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:55:28.739068 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:55:28.755661 12488 catalog_manager.cc:1383] Generated new cluster ID: 7c39595d5ad545c09c2257841dae790d
I20260709 12:55:28.756026 12488 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:55:28.774506 12488 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:55:28.776603 12488 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:55:28.794461 12488 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0: Generated new TSK 0
I20260709 12:55:28.795480 12488 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:55:28.806016 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:28.817764 12498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:28.821194 12499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:28.821677 12501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:28.825569 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:28.826592 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:28.826869 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:28.827045 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601728827030 us; error 0 us; skew 500 ppm
I20260709 12:55:28.828058 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:28.832964 12409 webserver.cc:533] Webserver started at http://127.12.30.65:44705/ using document root <none> and password file <none>
I20260709 12:55:28.833554 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:28.833787 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:28.834095 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:28.835819 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "5f140077bd674645a89e0f96ed93c6a7"
format_stamp: "Formatted at 2026-07-09 12:55:28 on dist-test-slave-mlt7"
I20260709 12:55:28.843066 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:55:28.848769 12506 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:28.849831 12409 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:55:28.850159 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "5f140077bd674645a89e0f96ed93c6a7"
format_stamp: "Formatted at 2026-07-09 12:55:28 on dist-test-slave-mlt7"
I20260709 12:55:28.850505 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:28.875515 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:28.877172 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:28.880298 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:28.886186 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:28.886488 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:28.886801 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:28.887167 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:29.004827 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:42093
I20260709 12:55:29.004899 12568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:42093 every 8 connection(s)
I20260709 12:55:29.012523 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:29.023294 12573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:29.025763 12574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:29.031766 12576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:29.032403 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:29.033485 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:29.033753 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:29.033957 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601729033941 us; error 0 us; skew 500 ppm
I20260709 12:55:29.034745 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:29.039546 12409 webserver.cc:533] Webserver started at http://127.12.30.66:34837/ using document root <none> and password file <none>
I20260709 12:55:29.040302 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:29.040555 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:29.040902 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:29.042366 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "a9961e5157c1489f829d80a64a117246"
format_stamp: "Formatted at 2026-07-09 12:55:29 on dist-test-slave-mlt7"
I20260709 12:55:29.043731 12569 heartbeater.cc:344] Connected to a master server at 127.12.30.126:34439
I20260709 12:55:29.044238 12569 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:29.045271 12569 heartbeater.cc:507] Master 127.12.30.126:34439 requested a full tablet report, sending...
I20260709 12:55:29.048599 12441 ts_manager.cc:194] Registered new tserver with Master: 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093)
I20260709 12:55:29.049512 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.009s	sys 0.000s
I20260709 12:55:29.050984 12441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55138
I20260709 12:55:29.054808 12581 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:29.055757 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:55:29.056104 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "a9961e5157c1489f829d80a64a117246"
format_stamp: "Formatted at 2026-07-09 12:55:29 on dist-test-slave-mlt7"
I20260709 12:55:29.056468 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:29.086529 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:29.088299 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:29.090924 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:29.094800 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:29.095088 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:29.095381 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:29.095600 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:29.185596 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:35517
I20260709 12:55:29.185958 12643 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:35517 every 8 connection(s)
I20260709 12:55:29.192859 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:29.204756 12647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:29.207345 12644 heartbeater.cc:344] Connected to a master server at 127.12.30.126:34439
I20260709 12:55:29.207791 12644 heartbeater.cc:461] Registering TS with master...
W20260709 12:55:29.206156 12648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:29.209076 12644 heartbeater.cc:507] Master 127.12.30.126:34439 requested a full tablet report, sending...
I20260709 12:55:29.212756 12441 ts_manager.cc:194] Registered new tserver with Master: a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:29.214886 12441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55148
I20260709 12:55:29.218362 12409 server_base.cc:1061] running on GCE node
W20260709 12:55:29.218552 12650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:29.219833 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:29.220173 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:29.220394 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601729220377 us; error 0 us; skew 500 ppm
I20260709 12:55:29.221284 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:29.224421 12409 webserver.cc:533] Webserver started at http://127.12.30.67:43429/ using document root <none> and password file <none>
I20260709 12:55:29.225109 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:29.225387 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:29.225736 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:29.227437 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "3b7c89c0026a401b9f38d8439dec5d24"
format_stamp: "Formatted at 2026-07-09 12:55:29 on dist-test-slave-mlt7"
I20260709 12:55:29.234133 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.003s	sys 0.002s
I20260709 12:55:29.238647 12655 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:29.239562 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:55:29.239946 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "3b7c89c0026a401b9f38d8439dec5d24"
format_stamp: "Formatted at 2026-07-09 12:55:29 on dist-test-slave-mlt7"
I20260709 12:55:29.240340 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:29.258668 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:29.260360 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:29.262454 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:29.266378 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:29.266685 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:29.266978 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:29.267207 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:29.356715 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:33627
I20260709 12:55:29.356732 12717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:33627 every 8 connection(s)
I20260709 12:55:29.377254 12718 heartbeater.cc:344] Connected to a master server at 127.12.30.126:34439
I20260709 12:55:29.377902 12718 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:29.379444 12718 heartbeater.cc:507] Master 127.12.30.126:34439 requested a full tablet report, sending...
I20260709 12:55:29.382120 12441 ts_manager.cc:194] Registered new tserver with Master: 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:29.382253 12409 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019656126s
I20260709 12:55:29.383692 12441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55154
I20260709 12:55:29.445892 12441 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55158:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:55:29.450732 12441 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:55:29.552455 12533 tablet_service.cc:1511] Processing CreateTablet for tablet cb2d78d6920a4691bf741f673273af69 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:55:29.555122 12533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb2d78d6920a4691bf741f673273af69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.552455 12532 tablet_service.cc:1511] Processing CreateTablet for tablet 6f547f26a6ee4126852ef442f97a92d5 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:55:29.560508 12534 tablet_service.cc:1511] Processing CreateTablet for tablet dea60e9b396c4cd896d7b9a2ab2fe7e5 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:55:29.568476 12532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f547f26a6ee4126852ef442f97a92d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.570320 12534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea60e9b396c4cd896d7b9a2ab2fe7e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.574795 12531 tablet_service.cc:1511] Processing CreateTablet for tablet 4c1d5ab2b5c04370a27121532c65e743 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:55:29.576375 12531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c1d5ab2b5c04370a27121532c65e743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.579288 12530 tablet_service.cc:1511] Processing CreateTablet for tablet 71eaffb19a734e94847e7ec6efd155e6 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:55:29.580765 12530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71eaffb19a734e94847e7ec6efd155e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.584782 12529 tablet_service.cc:1511] Processing CreateTablet for tablet dac941ae651a414ea7db7fa8c6e86034 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:55:29.586331 12529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac941ae651a414ea7db7fa8c6e86034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.588140 12609 tablet_service.cc:1511] Processing CreateTablet for tablet dea60e9b396c4cd896d7b9a2ab2fe7e5 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:55:29.589628 12609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea60e9b396c4cd896d7b9a2ab2fe7e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.589495 12608 tablet_service.cc:1511] Processing CreateTablet for tablet cb2d78d6920a4691bf741f673273af69 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:55:29.591977 12607 tablet_service.cc:1511] Processing CreateTablet for tablet 6f547f26a6ee4126852ef442f97a92d5 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:55:29.593362 12607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f547f26a6ee4126852ef442f97a92d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.593223 12605 tablet_service.cc:1511] Processing CreateTablet for tablet 71eaffb19a734e94847e7ec6efd155e6 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:55:29.594645 12605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71eaffb19a734e94847e7ec6efd155e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.595726 12606 tablet_service.cc:1511] Processing CreateTablet for tablet 4c1d5ab2b5c04370a27121532c65e743 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:55:29.597106 12606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c1d5ab2b5c04370a27121532c65e743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.600512 12608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb2d78d6920a4691bf741f673273af69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.605880 12604 tablet_service.cc:1511] Processing CreateTablet for tablet dac941ae651a414ea7db7fa8c6e86034 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:55:29.607398 12604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac941ae651a414ea7db7fa8c6e86034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.610333 12603 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac008d3702446898ff4f4e463f78156 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:55:29.611819 12603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac008d3702446898ff4f4e463f78156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.608978 12528 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac008d3702446898ff4f4e463f78156 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:55:29.616057 12528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac008d3702446898ff4f4e463f78156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.627818 12527 tablet_service.cc:1511] Processing CreateTablet for tablet 82337432dcdf429882e1d51ee37f2445 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:55:29.637158 12602 tablet_service.cc:1511] Processing CreateTablet for tablet 82337432dcdf429882e1d51ee37f2445 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:55:29.640211 12527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82337432dcdf429882e1d51ee37f2445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.642423 12602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82337432dcdf429882e1d51ee37f2445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.669201 12601 tablet_service.cc:1511] Processing CreateTablet for tablet 18a6efafe35a4b0dad3b1a0b7e192024 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:55:29.679697 12682 tablet_service.cc:1511] Processing CreateTablet for tablet cb2d78d6920a4691bf741f673273af69 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:55:29.668663 12532 tablet_service.cc:1511] Processing CreateTablet for tablet 18a6efafe35a4b0dad3b1a0b7e192024 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:55:29.680656 12683 tablet_service.cc:1511] Processing CreateTablet for tablet dea60e9b396c4cd896d7b9a2ab2fe7e5 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:55:29.687341 12682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb2d78d6920a4691bf741f673273af69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.690177 12736 tablet_bootstrap.cc:492] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:29.683224 12681 tablet_service.cc:1511] Processing CreateTablet for tablet 6f547f26a6ee4126852ef442f97a92d5 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:55:29.699910 12532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a6efafe35a4b0dad3b1a0b7e192024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.700924 12601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a6efafe35a4b0dad3b1a0b7e192024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.700344 12683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea60e9b396c4cd896d7b9a2ab2fe7e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.715116 12680 tablet_service.cc:1511] Processing CreateTablet for tablet 4c1d5ab2b5c04370a27121532c65e743 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:55:29.716948 12681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f547f26a6ee4126852ef442f97a92d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.715571 12678 tablet_service.cc:1511] Processing CreateTablet for tablet dac941ae651a414ea7db7fa8c6e86034 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:55:29.729951 12680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c1d5ab2b5c04370a27121532c65e743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.733906 12677 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac008d3702446898ff4f4e463f78156 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:55:29.745981 12677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac008d3702446898ff4f4e463f78156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.703764 12679 tablet_service.cc:1511] Processing CreateTablet for tablet 71eaffb19a734e94847e7ec6efd155e6 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:55:29.749053 12678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dac941ae651a414ea7db7fa8c6e86034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.754101 12679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71eaffb19a734e94847e7ec6efd155e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.757915 12737 tablet_bootstrap.cc:492] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:29.772607 12676 tablet_service.cc:1511] Processing CreateTablet for tablet 82337432dcdf429882e1d51ee37f2445 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:55:29.777390 12676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82337432dcdf429882e1d51ee37f2445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:29.791533 12675 tablet_service.cc:1511] Processing CreateTablet for tablet 18a6efafe35a4b0dad3b1a0b7e192024 (DEFAULT_TABLE table=test-workload [id=5034d1ef924149d1a6d322604078b381]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:55:29.796162 12675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a6efafe35a4b0dad3b1a0b7e192024. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:55:29.823004 12489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:29.840400 12736 tablet_bootstrap.cc:654] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:29.853475 12737 tablet_bootstrap.cc:654] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:29.857924 12736 tablet_bootstrap.cc:492] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:29.869807 12736 ts_tablet_manager.cc:1403] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.200s	user 0.016s	sys 0.007s
I20260709 12:55:29.874337 12739 tablet_bootstrap.cc:492] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:29.885504 12736 raft_consensus.cc:359] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.887818 12736 raft_consensus.cc:385] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:29.890424 12737 tablet_bootstrap.cc:492] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:29.890821 12737 ts_tablet_manager.cc:1403] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.133s	user 0.014s	sys 0.007s
I20260709 12:55:29.889290 12736 raft_consensus.cc:740] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:29.892191 12736 consensus_queue.cc:260] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.893929 12737 raft_consensus.cc:359] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.894979 12737 raft_consensus.cc:385] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:29.895291 12739 tablet_bootstrap.cc:654] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:29.895452 12737 raft_consensus.cc:740] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:29.896836 12737 consensus_queue.cc:260] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.901113 12569 heartbeater.cc:499] Master 127.12.30.126:34439 was elected leader, sending a full tablet report...
I20260709 12:55:29.902583 12736 ts_tablet_manager.cc:1434] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.008s
I20260709 12:55:29.904124 12736 tablet_bootstrap.cc:492] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:29.912919 12739 tablet_bootstrap.cc:492] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:29.913522 12736 tablet_bootstrap.cc:654] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:29.913965 12739 ts_tablet_manager.cc:1403] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.008s
I20260709 12:55:29.917853 12644 heartbeater.cc:499] Master 127.12.30.126:34439 was elected leader, sending a full tablet report...
I20260709 12:55:29.917448 12739 raft_consensus.cc:359] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.918298 12739 raft_consensus.cc:385] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:29.918586 12739 raft_consensus.cc:740] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:29.919293 12737 ts_tablet_manager.cc:1434] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.008s
I20260709 12:55:29.919418 12739 consensus_queue.cc:260] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.924743 12737 tablet_bootstrap.cc:492] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:29.925423 12736 tablet_bootstrap.cc:492] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:29.925928 12736 ts_tablet_manager.cc:1403] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:55:29.931222 12736 raft_consensus.cc:359] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.932727 12736 raft_consensus.cc:385] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:29.933362 12737 tablet_bootstrap.cc:654] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:29.933343 12736 raft_consensus.cc:740] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:29.935524 12739 ts_tablet_manager.cc:1434] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:55:29.935506 12718 heartbeater.cc:499] Master 127.12.30.126:34439 was elected leader, sending a full tablet report...
I20260709 12:55:29.935849 12736 consensus_queue.cc:260] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.936719 12739 tablet_bootstrap.cc:492] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:29.938933 12736 ts_tablet_manager.cc:1434] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:55:29.940320 12736 tablet_bootstrap.cc:492] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:29.943646 12739 tablet_bootstrap.cc:654] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
W20260709 12:55:29.945422 12645 tablet.cc:2406] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:55:29.949791 12736 tablet_bootstrap.cc:654] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:29.950690 12741 raft_consensus.cc:493] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:29.951424 12741 raft_consensus.cc:515] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:29.954452 12741 leader_election.cc:290] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:29.975052 12737 tablet_bootstrap.cc:492] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:29.975742 12737 ts_tablet_manager.cc:1403] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.004s
I20260709 12:55:29.979740 12737 raft_consensus.cc:359] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.006649 12737 raft_consensus.cc:385] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.007165 12737 raft_consensus.cc:740] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.011144 12737 consensus_queue.cc:260] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.019970 12737 ts_tablet_manager.cc:1434] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.003s
I20260709 12:55:30.024973 12739 tablet_bootstrap.cc:492] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.025477 12739 ts_tablet_manager.cc:1403] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.089s	user 0.031s	sys 0.055s
I20260709 12:55:30.026453 12736 tablet_bootstrap.cc:492] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:30.027006 12736 ts_tablet_manager.cc:1403] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.087s	user 0.037s	sys 0.041s
I20260709 12:55:30.028836 12739 raft_consensus.cc:359] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.029789 12739 raft_consensus.cc:385] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.030324 12739 raft_consensus.cc:740] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.031235 12739 consensus_queue.cc:260] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.031395 12736 raft_consensus.cc:359] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.032366 12736 raft_consensus.cc:385] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.032737 12736 raft_consensus.cc:740] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.033720 12739 ts_tablet_manager.cc:1434] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:55:30.033490 12736 consensus_queue.cc:260] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.034890 12739 tablet_bootstrap.cc:492] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:30.040046 12736 ts_tablet_manager.cc:1434] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:55:30.041165 12736 tablet_bootstrap.cc:492] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:30.042081 12739 tablet_bootstrap.cc:654] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.052114 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.052618 12736 tablet_bootstrap.cc:654] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.053238 12693 raft_consensus.cc:2468] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 0.
I20260709 12:55:30.055828 12507 leader_election.cc:304] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:30.057117 12741 raft_consensus.cc:2804] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.057492 12741 raft_consensus.cc:493] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.057830 12741 raft_consensus.cc:3060] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.062811 12737 tablet_bootstrap.cc:492] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:30.065220 12741 raft_consensus.cc:515] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.068869 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.069451 12693 raft_consensus.cc:3060] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.070317 12739 tablet_bootstrap.cc:492] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.070783 12739 ts_tablet_manager.cc:1403] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.036s	user 0.022s	sys 0.009s
I20260709 12:55:30.077863 12737 tablet_bootstrap.cc:654] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.077803 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.080772 12619 raft_consensus.cc:2468] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 0.
I20260709 12:55:30.083052 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:30.081223 12741 leader_election.cc:290] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.083837 12752 raft_consensus.cc:493] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.084261 12618 raft_consensus.cc:3060] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.085337 12693 raft_consensus.cc:2468] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 1.
I20260709 12:55:30.087493 12507 leader_election.cc:304] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:30.088007 12739 raft_consensus.cc:359] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.088721 12741 raft_consensus.cc:2804] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.088768 12739 raft_consensus.cc:385] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.096223 12752 raft_consensus.cc:515] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.096899 12739 raft_consensus.cc:740] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.098875 12739 consensus_queue.cc:260] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.100301 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
W20260709 12:55:30.103281 12510 leader_election.cc:343] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.103545 12739 ts_tablet_manager.cc:1434] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.002s
I20260709 12:55:30.103601 12736 tablet_bootstrap.cc:492] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:30.104295 12736 ts_tablet_manager.cc:1403] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.063s	user 0.018s	sys 0.010s
I20260709 12:55:30.104688 12739 tablet_bootstrap.cc:492] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:30.107264 12736 raft_consensus.cc:359] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.107986 12736 raft_consensus.cc:385] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.108340 12736 raft_consensus.cc:740] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.109000 12736 consensus_queue.cc:260] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.121932 12618 raft_consensus.cc:2468] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 1.
I20260709 12:55:30.124697 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
W20260709 12:55:30.126758 12507 leader_election.cc:343] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.127333 12507 leader_election.cc:304] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7; no voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.129395 12757 raft_consensus.cc:2749] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.130162 12752 leader_election.cc:290] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.131426 12736 ts_tablet_manager.cc:1434] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.005s
I20260709 12:55:30.131309 12741 raft_consensus.cc:697] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:30.132691 12736 tablet_bootstrap.cc:492] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:30.132460 12741 consensus_queue.cc:237] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.134402 12737 tablet_bootstrap.cc:492] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:30.135071 12737 ts_tablet_manager.cc:1403] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.073s	user 0.031s	sys 0.024s
I20260709 12:55:30.138697 12737 raft_consensus.cc:359] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.139521 12737 raft_consensus.cc:385] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.139880 12737 raft_consensus.cc:740] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.141413 12736 tablet_bootstrap.cc:654] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.140862 12737 consensus_queue.cc:260] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.142971 12739 tablet_bootstrap.cc:654] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.145725 12737 ts_tablet_manager.cc:1434] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:55:30.146840 12737 tablet_bootstrap.cc:492] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:30.154420 12739 tablet_bootstrap.cc:492] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.154989 12739 ts_tablet_manager.cc:1403] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.007s
I20260709 12:55:30.158161 12739 raft_consensus.cc:359] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.158923 12739 raft_consensus.cc:385] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.159245 12739 raft_consensus.cc:740] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.159580 12737 tablet_bootstrap.cc:654] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.160158 12739 consensus_queue.cc:260] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.168704 12736 tablet_bootstrap.cc:492] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:30.169214 12736 ts_tablet_manager.cc:1403] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.007s
I20260709 12:55:30.171177 12441 catalog_manager.cc:5697] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:30.172259 12736 raft_consensus.cc:359] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.173786 12743 raft_consensus.cc:493] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.174341 12743 raft_consensus.cc:515] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.177191 12743 leader_election.cc:290] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.177848 12736 raft_consensus.cc:385] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.178232 12736 raft_consensus.cc:740] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.179766 12736 consensus_queue.cc:260] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.181551 12739 ts_tablet_manager.cc:1434] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.001s
I20260709 12:55:30.183254 12739 tablet_bootstrap.cc:492] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:30.192190 12739 tablet_bootstrap.cc:654] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.196601 12736 ts_tablet_manager.cc:1434] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 12:55:30.197722 12736 tablet_bootstrap.cc:492] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:30.206372 12736 tablet_bootstrap.cc:654] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.230563 12736 tablet_bootstrap.cc:492] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:30.231057 12736 ts_tablet_manager.cc:1403] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.000s
I20260709 12:55:30.237664 12544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.238746 12544 raft_consensus.cc:2468] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.239802 12739 tablet_bootstrap.cc:492] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.241559 12739 ts_tablet_manager.cc:1403] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.015s
I20260709 12:55:30.241603 12584 leader_election.cc:304] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:30.242892 12743 raft_consensus.cc:2804] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.243264 12743 raft_consensus.cc:493] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.243618 12743 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.238451 12736 raft_consensus.cc:359] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.245285 12736 raft_consensus.cc:385] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.245730 12736 raft_consensus.cc:740] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.248636 12745 raft_consensus.cc:493] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.246884 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.252842 12745 raft_consensus.cc:515] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.256353 12743 raft_consensus.cc:515] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.259173 12743 leader_election.cc:290] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.259945 12544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:30.260071 12745 leader_election.cc:290] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:30.260594 12544 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:30.254187 12582 leader_election.cc:343] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.252122 12739 raft_consensus.cc:359] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.253154 12736 consensus_queue.cc:260] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.264905 12739 raft_consensus.cc:385] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.267078 12739 raft_consensus.cc:740] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.267845 12739 consensus_queue.cc:260] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.273132 12757 raft_consensus.cc:493] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.273626 12757 raft_consensus.cc:515] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.275187 12737 tablet_bootstrap.cc:492] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:30.275655 12737 ts_tablet_manager.cc:1403] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.129s	user 0.027s	sys 0.028s
I20260709 12:55:30.277858 12757 leader_election.cc:290] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.274029 12736 ts_tablet_manager.cc:1434] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.003s
I20260709 12:55:30.274803 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.281435 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.282734 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.283403 12692 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 0.
W20260709 12:55:30.287012 12510 leader_election.cc:343] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.282860 12739 ts_tablet_manager.cc:1434] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.002s
I20260709 12:55:30.288107 12507 leader_election.cc:304] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.290879 12757 raft_consensus.cc:2804] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.291213 12757 raft_consensus.cc:493] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.291524 12757 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.293322 12737 raft_consensus.cc:359] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.294160 12743 raft_consensus.cc:493] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.295249 12743 raft_consensus.cc:515] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
W20260709 12:55:30.295900 12582 leader_election.cc:343] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.297675 12743 leader_election.cc:290] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.294668 12737 raft_consensus.cc:385] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.298607 12737 raft_consensus.cc:740] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.299571 12737 consensus_queue.cc:260] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.300438 12544 raft_consensus.cc:2468] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.301023 12739 tablet_bootstrap.cc:492] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:30.304263 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.305461 12543 raft_consensus.cc:2468] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.306550 12584 leader_election.cc:304] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 3b7c89c0026a401b9f38d8439dec5d24
I20260709 12:55:30.307004 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.307875 12584 leader_election.cc:304] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
W20260709 12:55:30.308518 12582 leader_election.cc:343] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.309777 12736 tablet_bootstrap.cc:492] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:30.312263 12743 raft_consensus.cc:2804] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.306556 12737 ts_tablet_manager.cc:1434] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:55:30.313933 12737 tablet_bootstrap.cc:492] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:30.324062 12772 raft_consensus.cc:2804] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.324618 12772 raft_consensus.cc:493] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.325366 12772 raft_consensus.cc:3060] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.328621 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.337050 12743 raft_consensus.cc:697] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:30.337050 12757 raft_consensus.cc:515] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.342859 12543 raft_consensus.cc:2393] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b7c89c0026a401b9f38d8439dec5d24 in current term 1: Already voted for candidate 5f140077bd674645a89e0f96ed93c6a7 in this term.
I20260709 12:55:30.345410 12739 tablet_bootstrap.cc:654] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.347522 12737 tablet_bootstrap.cc:654] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.343040 12743 consensus_queue.cc:237] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.351460 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.352229 12757 leader_election.cc:290] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.352716 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:30.360488 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.361158 12693 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:30.361657 12659 leader_election.cc:343] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.362119 12659 leader_election.cc:304] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24; no voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.365015 12736 tablet_bootstrap.cc:654] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.361374 12757 raft_consensus.cc:493] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.367959 12757 raft_consensus.cc:515] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.374171 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.372529 12757 leader_election.cc:290] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
W20260709 12:55:30.375944 12510 leader_election.cc:343] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.376068 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
W20260709 12:55:30.377429 12507 leader_election.cc:343] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.378073 12507 leader_election.cc:304] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7; no voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.379588 12693 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 1.
I20260709 12:55:30.380429 12745 raft_consensus.cc:2749] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.380985 12507 leader_election.cc:304] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.381824 12741 raft_consensus.cc:2749] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.382280 12757 raft_consensus.cc:2804] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.382944 12757 raft_consensus.cc:697] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:30.383790 12757 consensus_queue.cc:237] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.388656 12737 tablet_bootstrap.cc:492] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:30.389653 12737 ts_tablet_manager.cc:1403] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.076s	user 0.006s	sys 0.012s
I20260709 12:55:30.386476 12772 raft_consensus.cc:515] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.391134 12745 raft_consensus.cc:493] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.392395 12745 raft_consensus.cc:515] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.394879 12745 leader_election.cc:290] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:30.395459 12739 tablet_bootstrap.cc:492] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.396301 12739 ts_tablet_manager.cc:1403] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.010s
I20260709 12:55:30.397856 12737 raft_consensus.cc:359] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.398679 12737 raft_consensus.cc:385] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.399016 12737 raft_consensus.cc:740] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.399905 12737 consensus_queue.cc:260] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.401078 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.402284 12737 ts_tablet_manager.cc:1434] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:55:30.403781 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
W20260709 12:55:30.407150 12658 leader_election.cc:343] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.405264 12739 raft_consensus.cc:359] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.407449 12737 tablet_bootstrap.cc:492] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:30.407652 12739 raft_consensus.cc:385] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:55:30.407867 12659 leader_election.cc:343] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.407898 12739 raft_consensus.cc:740] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.408530 12659 leader_election.cc:304] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24; no voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.408995 12739 consensus_queue.cc:260] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.409566 12745 raft_consensus.cc:2749] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.412478 12736 tablet_bootstrap.cc:492] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:30.413017 12736 ts_tablet_manager.cc:1403] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.109s	user 0.010s	sys 0.014s
I20260709 12:55:30.416042 12736 raft_consensus.cc:359] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.415907 12441 catalog_manager.cc:5697] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:30.416853 12736 raft_consensus.cc:385] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.417326 12736 raft_consensus.cc:740] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.417995 12737 tablet_bootstrap.cc:654] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.418148 12736 consensus_queue.cc:260] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.421708 12736 ts_tablet_manager.cc:1434] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:55:30.422603 12772 leader_election.cc:290] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.441624 12739 ts_tablet_manager.cc:1434] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.045s	user 0.011s	sys 0.008s
I20260709 12:55:30.488476 12736 tablet_bootstrap.cc:492] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:30.488672 12739 tablet_bootstrap.cc:492] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:30.448487 12743 raft_consensus.cc:493] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.492936 12745 raft_consensus.cc:493] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.492771 12743 raft_consensus.cc:515] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.493454 12745 raft_consensus.cc:515] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.495450 12743 leader_election.cc:290] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.498515 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.499054 12440 catalog_manager.cc:5697] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
W20260709 12:55:30.499997 12659 leader_election.cc:343] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.500401 12737 tablet_bootstrap.cc:492] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:30.495565 12745 leader_election.cc:290] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:30.496193 12544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.423604 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:30.501607 12543 raft_consensus.cc:3060] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.501020 12737 ts_tablet_manager.cc:1403] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.094s	user 0.010s	sys 0.003s
I20260709 12:55:30.502447 12542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
W20260709 12:55:30.504055 12658 leader_election.cc:343] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.504504 12658 leader_election.cc:304] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24; no voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.505270 12737 raft_consensus.cc:359] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.500267 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.496169 12739 tablet_bootstrap.cc:654] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.501036 12544 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.509148 12737 raft_consensus.cc:385] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.509292 12692 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.487593 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.509610 12737 raft_consensus.cc:740] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.510660 12737 consensus_queue.cc:260] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.508726 12736 tablet_bootstrap.cc:654] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.513093 12584 leader_election.cc:304] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:30.514014 12543 raft_consensus.cc:2468] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.514205 12743 raft_consensus.cc:2804] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.514725 12743 raft_consensus.cc:493] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.515127 12743 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:30.517110 12582 leader_election.cc:343] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.505625 12745 raft_consensus.cc:2749] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.520511 12584 leader_election.cc:304] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 3b7c89c0026a401b9f38d8439dec5d24
I20260709 12:55:30.521184 12772 raft_consensus.cc:2804] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.521656 12772 raft_consensus.cc:697] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:30.522379 12737 ts_tablet_manager.cc:1434] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:55:30.522504 12772 consensus_queue.cc:237] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.524910 12737 tablet_bootstrap.cc:492] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:30.525601 12757 raft_consensus.cc:493] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.526235 12739 tablet_bootstrap.cc:492] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.526065 12757 raft_consensus.cc:515] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.523833 12743 raft_consensus.cc:515] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.526751 12739 ts_tablet_manager.cc:1403] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.011s
I20260709 12:55:30.528358 12757 leader_election.cc:290] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.529667 12745 raft_consensus.cc:493] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.529868 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.530153 12745 raft_consensus.cc:515] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.530539 12693 raft_consensus.cc:2468] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 0.
I20260709 12:55:30.528496 12743 leader_election.cc:290] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.531790 12507 leader_election.cc:304] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:30.532909 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.533301 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
W20260709 12:55:30.534163 12510 leader_election.cc:343] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:30.534610 12659 leader_election.cc:343] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.535435 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:30.536031 12543 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.536412 12544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.537878 12741 raft_consensus.cc:2804] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.529886 12739 raft_consensus.cc:359] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.538898 12741 raft_consensus.cc:493] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.539247 12739 raft_consensus.cc:385] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.539346 12741 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.539575 12739 raft_consensus.cc:740] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.540422 12739 consensus_queue.cc:260] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
W20260709 12:55:30.541009 12658 leader_election.cc:343] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.541595 12658 leader_election.cc:304] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24; no voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.542742 12739 ts_tablet_manager.cc:1434] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:55:30.543717 12739 tablet_bootstrap.cc:492] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
I20260709 12:55:30.544492 12779 raft_consensus.cc:2749] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.545006 12693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.545610 12693 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.547379 12741 raft_consensus.cc:515] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.549477 12741 leader_election.cc:290] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.550180 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246"
W20260709 12:55:30.551463 12510 leader_election.cc:343] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:30.552409 12741 consensus_queue.cc:1048] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:30.553133 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.553706 12692 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.554894 12737 tablet_bootstrap.cc:654] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.562340 12739 tablet_bootstrap.cc:654] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.557212 12745 leader_election.cc:290] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:30.555322 12693 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.567581 12582 leader_election.cc:304] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:30.568457 12743 raft_consensus.cc:2804] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.569260 12743 raft_consensus.cc:697] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:30.570538 12743 consensus_queue.cc:237] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.574656 12692 raft_consensus.cc:2468] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 1.
I20260709 12:55:30.576290 12507 leader_election.cc:304] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.577373 12543 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.577397 12741 raft_consensus.cc:2804] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.578148 12741 raft_consensus.cc:697] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:30.579480 12741 consensus_queue.cc:237] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.589334 12739 tablet_bootstrap.cc:492] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.589895 12739 ts_tablet_manager.cc:1403] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.006s
I20260709 12:55:30.592986 12739 raft_consensus.cc:359] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.593748 12739 raft_consensus.cc:385] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.594076 12739 raft_consensus.cc:740] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.594799 12739 consensus_queue.cc:260] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.597224 12739 ts_tablet_manager.cc:1434] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:55:30.597743 12737 tablet_bootstrap.cc:492] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:30.598773 12737 ts_tablet_manager.cc:1403] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.074s	user 0.006s	sys 0.008s
I20260709 12:55:30.599591 12438 catalog_manager.cc:5697] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:30.605154 12737 raft_consensus.cc:359] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.598268 12739 tablet_bootstrap.cc:492] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24: Bootstrap starting.
W20260709 12:55:30.610332 12658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:42093, user_credentials={real_user=slave}} blocked reactor thread for 69338.1us
I20260709 12:55:30.612782 12737 raft_consensus.cc:385] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.613893 12737 raft_consensus.cc:740] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.615667 12737 consensus_queue.cc:260] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.619387 12739 tablet_bootstrap.cc:654] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.622112 12737 ts_tablet_manager.cc:1434] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:55:30.623586 12737 tablet_bootstrap.cc:492] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:30.628533 12440 catalog_manager.cc:5697] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:30.634204 12737 tablet_bootstrap.cc:654] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.635706 12739 tablet_bootstrap.cc:492] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24: No bootstrap required, opened a new log
I20260709 12:55:30.636209 12739 ts_tablet_manager.cc:1403] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.006s
I20260709 12:55:30.639562 12739 raft_consensus.cc:359] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.640576 12739 raft_consensus.cc:385] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.640928 12739 raft_consensus.cc:740] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.641793 12739 consensus_queue.cc:260] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.645622 12737 tablet_bootstrap.cc:492] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:30.646128 12737 ts_tablet_manager.cc:1403] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.006s
I20260709 12:55:30.648650 12737 raft_consensus.cc:359] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.649451 12737 raft_consensus.cc:385] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.649768 12737 raft_consensus.cc:740] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.650460 12737 consensus_queue.cc:260] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.652671 12737 ts_tablet_manager.cc:1434] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:55:30.653952 12737 tablet_bootstrap.cc:492] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246: Bootstrap starting.
I20260709 12:55:30.644215 12739 ts_tablet_manager.cc:1434] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:55:30.658062 12736 tablet_bootstrap.cc:492] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:30.658504 12736 ts_tablet_manager.cc:1403] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.170s	user 0.011s	sys 0.022s
I20260709 12:55:30.661146 12736 raft_consensus.cc:359] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.661799 12736 raft_consensus.cc:385] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.662063 12736 raft_consensus.cc:740] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.662698 12736 consensus_queue.cc:260] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.658100 12755 consensus_queue.cc:1048] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:55:30.666383 12736 ts_tablet_manager.cc:1434] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:55:30.667384 12736 tablet_bootstrap.cc:492] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7: Bootstrap starting.
I20260709 12:55:30.670751 12441 catalog_manager.cc:5697] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:30.676921 12736 tablet_bootstrap.cc:654] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.671197 12737 tablet_bootstrap.cc:654] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:30.695119 12737 tablet_bootstrap.cc:492] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246: No bootstrap required, opened a new log
I20260709 12:55:30.695660 12737 ts_tablet_manager.cc:1403] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.000s
I20260709 12:55:30.698258 12737 raft_consensus.cc:359] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.698915 12737 raft_consensus.cc:385] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.699222 12737 raft_consensus.cc:740] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.699795 12737 consensus_queue.cc:260] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.701686 12737 ts_tablet_manager.cc:1434] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:55:30.702888 12736 tablet_bootstrap.cc:492] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7: No bootstrap required, opened a new log
I20260709 12:55:30.702939 12772 raft_consensus.cc:493] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.703572 12736 ts_tablet_manager.cc:1403] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.010s
I20260709 12:55:30.703573 12772 raft_consensus.cc:515] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.705796 12772 leader_election.cc:290] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.706573 12736 raft_consensus.cc:359] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.707293 12736 raft_consensus.cc:385] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:30.707330 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.707733 12736 raft_consensus.cc:740] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Initialized, Role: FOLLOWER
I20260709 12:55:30.708487 12736 consensus_queue.cc:260] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.711282 12736 ts_tablet_manager.cc:1434] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:55:30.715030 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.716338 12691 raft_consensus.cc:2393] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9961e5157c1489f829d80a64a117246 in current term 1: Already voted for candidate 5f140077bd674645a89e0f96ed93c6a7 in this term.
I20260709 12:55:30.717691 12582 leader_election.cc:304] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9961e5157c1489f829d80a64a117246; no voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7
I20260709 12:55:30.718571 12772 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.724985 12772 raft_consensus.cc:2749] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.729532 12784 raft_consensus.cc:493] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.730090 12784 raft_consensus.cc:515] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.732215 12784 leader_election.cc:290] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:30.733224 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:30.733182 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.734102 12543 raft_consensus.cc:2393] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b7c89c0026a401b9f38d8439dec5d24 in current term 1: Already voted for candidate a9961e5157c1489f829d80a64a117246 in this term.
I20260709 12:55:30.735065 12658 leader_election.cc:304] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24; no voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246
I20260709 12:55:30.735714 12784 raft_consensus.cc:3060] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.742422 12784 raft_consensus.cc:2749] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:30.770637 12772 consensus_queue.cc:1048] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:30.780320 12691 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.783209 12772 raft_consensus.cc:493] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.783746 12772 raft_consensus.cc:515] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.789912 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.790131 12772 leader_election.cc:290] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.790545 12543 raft_consensus.cc:2468] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.791749 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.792362 12692 raft_consensus.cc:2468] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.793617 12582 leader_election.cc:304] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:30.794866 12743 raft_consensus.cc:2804] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.794880 12772 consensus_queue.cc:1048] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:30.795364 12743 raft_consensus.cc:493] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.795739 12743 raft_consensus.cc:3060] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.802281 12743 raft_consensus.cc:515] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.804610 12743 leader_election.cc:290] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.805485 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:30.806078 12543 raft_consensus.cc:3060] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.812337 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.813130 12691 raft_consensus.cc:3060] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.814905 12543 raft_consensus.cc:2468] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.816861 12584 leader_election.cc:304] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:30.818173 12743 raft_consensus.cc:2804] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.818666 12743 raft_consensus.cc:697] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:30.820077 12743 consensus_queue.cc:237] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.820518 12691 raft_consensus.cc:2468] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.828985 12441 catalog_manager.cc:5697] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:30.829813 12619 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.836766 12755 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:55:30.838984 12489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:30.851650 12756 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:30.865685 12743 raft_consensus.cc:493] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.866218 12743 raft_consensus.cc:515] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.868814 12743 leader_election.cc:290] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.869098 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7" is_pre_election: true
I20260709 12:55:30.869777 12543 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.870894 12584 leader_election.cc:304] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:30.871687 12743 raft_consensus.cc:2804] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:30.871562 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:30.872134 12743 raft_consensus.cc:493] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:30.872363 12691 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 0.
I20260709 12:55:30.872566 12743 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.878049 12743 raft_consensus.cc:515] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.879863 12743 leader_election.cc:290] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:30.881129 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:30.881788 12691 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.882026 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:30.882802 12543 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:30.886714 12691 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.888051 12582 leader_election.cc:304] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:30.888825 12743 raft_consensus.cc:2804] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:30.889380 12743 raft_consensus.cc:697] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:30.889566 12543 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 1.
I20260709 12:55:30.890677 12743 consensus_queue.cc:237] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:30.899607 12440 catalog_manager.cc:5697] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:30.973223 12743 consensus_queue.cc:1048] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:30.985790 12772 consensus_queue.cc:1048] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:30.997182 12756 raft_consensus.cc:493] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:30.997326 12773 consensus_queue.cc:1048] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:30.997839 12756 raft_consensus.cc:515] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.000118 12756 leader_election.cc:290] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.001207 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24" is_pre_election: true
I20260709 12:55:31.001837 12692 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 0.
I20260709 12:55:31.004364 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246" is_pre_election: true
I20260709 12:55:31.005422 12619 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 0.
I20260709 12:55:31.006597 12507 leader_election.cc:304] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:31.007504 12756 raft_consensus.cc:2804] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:31.007977 12756 raft_consensus.cc:493] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:31.008380 12756 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:31.016292 12773 consensus_queue.cc:1048] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.021708 12756 raft_consensus.cc:515] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.023890 12756 leader_election.cc:290] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.024608 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:31.025132 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:31.025811 12619 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:31.026005 12692 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:31.035971 12692 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 1.
I20260709 12:55:31.036226 12619 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 1.
I20260709 12:55:31.037326 12507 leader_election.cc:304] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:31.038347 12756 raft_consensus.cc:2804] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:31.039429 12756 raft_consensus.cc:697] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:31.040284 12756 consensus_queue.cc:237] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.050500 12441 catalog_manager.cc:5697] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:31.081036 12756 consensus_queue.cc:1048] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.092490 12757 consensus_queue.cc:1048] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.215528 12773 consensus_queue.cc:1048] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.227088 12743 consensus_queue.cc:1048] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.342214 12743 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.359186 12773 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:31.394065 12619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:49978
I20260709 12:55:31.394677 12619 raft_consensus.cc:606] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Received request to transfer leadership to TS 5f140077bd674645a89e0f96ed93c6a7
I20260709 12:55:31.400947 12619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:49978
I20260709 12:55:31.401523 12619 raft_consensus.cc:606] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Received request to transfer leadership to TS 5f140077bd674645a89e0f96ed93c6a7
I20260709 12:55:31.407371 12619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:49978
I20260709 12:55:31.407922 12619 raft_consensus.cc:606] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Received request to transfer leadership to TS 5f140077bd674645a89e0f96ed93c6a7
I20260709 12:55:31.431862 12543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:53790
I20260709 12:55:31.432488 12543 raft_consensus.cc:606] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:31.438591 12543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:53790
I20260709 12:55:31.439232 12543 raft_consensus.cc:606] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:31.446949 12543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:53790
I20260709 12:55:31.447245 12773 raft_consensus.cc:993] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246: : Instructing follower 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:31.447611 12543 raft_consensus.cc:606] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:31.447804 12773 raft_consensus.cc:1081] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Signalling peer 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:31.452353 12543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:57602
I20260709 12:55:31.452973 12543 raft_consensus.cc:493] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:31.453373 12543 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.453784 12619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:49978
I20260709 12:55:31.454340 12619 raft_consensus.cc:606] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Received request to transfer leadership to TS 3b7c89c0026a401b9f38d8439dec5d24
I20260709 12:55:31.460323 12543 raft_consensus.cc:515] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.460844 12757 raft_consensus.cc:993] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:31.461354 12741 raft_consensus.cc:1081] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:31.462615 12543 leader_election.cc:290] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.463687 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:31.463852 12618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34830
I20260709 12:55:31.464445 12618 raft_consensus.cc:493] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:31.464474 12692 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.464927 12618 raft_consensus.cc:3060] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.469671 12692 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:31.470750 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:31.470966 12507 leader_election.cc:304] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:31.471367 12619 raft_consensus.cc:3055] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:31.471817 12741 raft_consensus.cc:2804] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:31.471501 12618 raft_consensus.cc:515] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.472307 12741 raft_consensus.cc:697] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:31.473161 12741 consensus_queue.cc:237] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.473773 12618 leader_election.cc:290] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 2 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.474912 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:31.475375 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dac941ae651a414ea7db7fa8c6e86034" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:31.475477 12691 raft_consensus.cc:3060] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.475986 12543 raft_consensus.cc:3055] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:31.476341 12543 raft_consensus.cc:740] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:31.477169 12543 consensus_queue.cc:260] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.478530 12543 raft_consensus.cc:3060] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.471715 12619 raft_consensus.cc:740] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:31.480744 12619 consensus_queue.cc:260] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.482415 12691 raft_consensus.cc:2468] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 2.
I20260709 12:55:31.484180 12619 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.485445 12582 leader_election.cc:304] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:31.486299 12772 raft_consensus.cc:2804] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:31.486769 12772 raft_consensus.cc:697] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:31.487573 12772 consensus_queue.cc:237] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.488971 12543 raft_consensus.cc:2468] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 2.
I20260709 12:55:31.493335 12619 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:31.493309 12441 catalog_manager.cc:5697] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 1 to 2, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:31.501472 12440 catalog_manager.cc:5697] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 1 to 2, leader changed from 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:31.545365 12741 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.564337 12756 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:31.575289 12757 raft_consensus.cc:993] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:31.575800 12756 raft_consensus.cc:1081] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:31.577435 12619 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34830
I20260709 12:55:31.578039 12619 raft_consensus.cc:493] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:31.578351 12619 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.582664 12619 raft_consensus.cc:515] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.585083 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:31.585331 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:31.585654 12543 raft_consensus.cc:3055] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:31.585834 12691 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.585994 12543 raft_consensus.cc:740] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:31.586972 12543 consensus_queue.cc:260] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.588305 12543 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.589424 12619 leader_election.cc:290] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 2 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.592315 12691 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 2.
I20260709 12:55:31.593384 12582 leader_election.cc:304] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:31.594230 12772 raft_consensus.cc:2804] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:31.594702 12772 raft_consensus.cc:697] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:31.595467 12772 consensus_queue.cc:237] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.596829 12543 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 2.
I20260709 12:55:31.602226 12440 catalog_manager.cc:5697] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 1 to 2, leader changed from 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:31.614326 12773 raft_consensus.cc:993] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246: : Instructing follower 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:31.618533 12772 raft_consensus.cc:1081] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Signalling peer 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:31.621592 12543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:57602
I20260709 12:55:31.623220 12543 raft_consensus.cc:493] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:31.624464 12543 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.631512 12543 raft_consensus.cc:515] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.633199 12543 leader_election.cc:290] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.634047 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:31.634263 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:31.634634 12619 raft_consensus.cc:3055] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:31.634904 12691 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.634977 12619 raft_consensus.cc:740] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:31.635874 12619 consensus_queue.cc:260] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.637076 12619 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.641031 12691 raft_consensus.cc:2468] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:31.642117 12507 leader_election.cc:304] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:31.643044 12756 raft_consensus.cc:2804] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:31.643508 12756 raft_consensus.cc:697] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:31.644011 12619 raft_consensus.cc:2468] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:31.644270 12756 consensus_queue.cc:237] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.650709 12441 catalog_manager.cc:5697] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 1 to 2, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:31.696861 12757 raft_consensus.cc:993] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:31.697315 12756 raft_consensus.cc:1081] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:31.699035 12619 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34830
I20260709 12:55:31.699589 12619 raft_consensus.cc:493] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:31.699965 12619 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.706525 12619 raft_consensus.cc:515] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.708776 12619 leader_election.cc:290] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 2 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.709904 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:31.710143 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:31.710505 12691 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.710740 12543 raft_consensus.cc:3055] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:31.711133 12543 raft_consensus.cc:740] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:31.711884 12543 consensus_queue.cc:260] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.712924 12543 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.717293 12691 raft_consensus.cc:2468] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 2.
I20260709 12:55:31.717810 12543 raft_consensus.cc:2468] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 2.
I20260709 12:55:31.718717 12582 leader_election.cc:304] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:31.719643 12773 raft_consensus.cc:2804] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:31.720161 12773 raft_consensus.cc:697] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:31.720983 12773 consensus_queue.cc:237] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.729244 12441 catalog_manager.cc:5697] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 1 to 2, leader changed from 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:31.788828 12821 raft_consensus.cc:993] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: : Instructing follower 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:31.789356 12821 raft_consensus.cc:1081] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Signalling peer 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:31.799171 12543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:57602
I20260709 12:55:31.799798 12543 raft_consensus.cc:493] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:31.800205 12543 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.804937 12543 raft_consensus.cc:515] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.807442 12543 leader_election.cc:290] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:31.807857 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:31.808060 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:31.808494 12619 raft_consensus.cc:3055] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:31.808624 12691 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.808840 12619 raft_consensus.cc:740] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:31.809722 12619 consensus_queue.cc:260] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.810915 12619 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.816885 12821 raft_consensus.cc:993] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246: : Instructing follower 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:31.817418 12773 raft_consensus.cc:1081] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Signalling peer 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:31.818241 12619 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:31.819346 12692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024"
dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:33348
I20260709 12:55:31.820035 12692 raft_consensus.cc:493] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:31.820339 12691 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:31.820703 12692 raft_consensus.cc:3060] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.821810 12510 leader_election.cc:304] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:31.823143 12756 raft_consensus.cc:2804] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:31.823645 12756 raft_consensus.cc:697] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:31.824465 12756 consensus_queue.cc:237] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.826921 12692 raft_consensus.cc:515] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.829257 12692 leader_election.cc:290] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 2 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:31.830514 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:31.831188 12543 raft_consensus.cc:3060] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.831110 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6efafe35a4b0dad3b1a0b7e192024" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:31.832355 12619 raft_consensus.cc:3055] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:31.832759 12619 raft_consensus.cc:740] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:31.833370 12441 catalog_manager.cc:5697] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 1 to 2, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:31.833629 12619 consensus_queue.cc:260] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.835353 12619 raft_consensus.cc:3060] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:31.838704 12543 raft_consensus.cc:2468] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 2.
I20260709 12:55:31.843416 12619 raft_consensus.cc:2468] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 2.
I20260709 12:55:31.844969 12659 leader_election.cc:304] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:31.846079 12784 raft_consensus.cc:2804] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:31.849009 12784 raft_consensus.cc:697] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 LEADER]: Becoming Leader. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Running, Role: LEADER
I20260709 12:55:31.849936 12784 consensus_queue.cc:237] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:31.859536 12441 catalog_manager.cc:5697] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 reported cstate change: term changed from 1 to 2, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:31.899222 12543 raft_consensus.cc:1275] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:31.900955 12773 consensus_queue.cc:1048] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:31.910060 12692 raft_consensus.cc:1275] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:31.911859 12821 consensus_queue.cc:1048] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:31.991822 12692 raft_consensus.cc:1275] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:31.993081 12756 consensus_queue.cc:1048] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.000643 12619 raft_consensus.cc:1275] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.002372 12741 consensus_queue.cc:1048] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.081221 12692 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.082726 12821 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.089071 12619 raft_consensus.cc:1275] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.090642 12543 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.090714 12757 consensus_queue.cc:1048] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:55:32.094045 12821 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.104133 12692 raft_consensus.cc:1275] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.105902 12757 consensus_queue.cc:1048] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:55:32.234820 12619 raft_consensus.cc:1275] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.236125 12757 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.243685 12692 raft_consensus.cc:1275] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.245343 12755 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.267500 12692 raft_consensus.cc:1275] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.269013 12821 consensus_queue.cc:1048] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.278024 12543 raft_consensus.cc:1275] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.279803 12821 consensus_queue.cc:1048] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.319713 12543 raft_consensus.cc:1275] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.321225 12784 consensus_queue.cc:1048] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:32.330667 12619 raft_consensus.cc:1275] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:32.332103 12784 consensus_queue.cc:1048] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:34.490057 12409 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 12:55:34.491309 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:34.517231 12543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
mode: GRACEFUL
new_leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:53798
I20260709 12:55:34.517905 12543 raft_consensus.cc:606] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Received request to transfer leadership to TS 3b7c89c0026a401b9f38d8439dec5d24
I20260709 12:55:34.531380 12619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:49994
I20260709 12:55:34.531919 12619 raft_consensus.cc:606] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Received request to transfer leadership to TS 3b7c89c0026a401b9f38d8439dec5d24
I20260709 12:55:34.533248 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:55:34.533918 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:55:34.581208 12756 raft_consensus.cc:993] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7: : Instructing follower 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:34.581815 12756 raft_consensus.cc:1081] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Signalling peer 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:34.583874 12691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743"
dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:33336
I20260709 12:55:34.584596 12691 raft_consensus.cc:493] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:34.585032 12691 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:34.590941 12691 raft_consensus.cc:515] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:34.593678 12691 leader_election.cc:290] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 2 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:34.594403 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:34.594779 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:34.595057 12543 raft_consensus.cc:3055] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:34.595263 12619 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:34.595362 12543 raft_consensus.cc:740] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:34.596185 12543 consensus_queue.cc:260] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:34.597273 12543 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:34.601800 12619 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 2.
I20260709 12:55:34.603353 12659 leader_election.cc:304] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:34.603538 12543 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 2.
I20260709 12:55:34.604399 12833 raft_consensus.cc:2804] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:34.605736 12833 raft_consensus.cc:697] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 LEADER]: Becoming Leader. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Running, Role: LEADER
I20260709 12:55:34.606633 12833 consensus_queue.cc:237] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:34.617445 12440 catalog_manager.cc:5697] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 reported cstate change: term changed from 1 to 2, leader changed from 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65) to 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:34.695516 12834 raft_consensus.cc:993] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: : Instructing follower 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:34.696074 12834 raft_consensus.cc:1081] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Signalling peer 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:34.697858 12691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:33348
I20260709 12:55:34.698431 12691 raft_consensus.cc:493] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:34.698781 12691 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:34.705411 12691 raft_consensus.cc:515] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:34.707406 12691 leader_election.cc:290] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 3 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:34.708431 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:34.708547 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:34.709080 12543 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:34.709133 12619 raft_consensus.cc:3055] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:34.709517 12619 raft_consensus.cc:740] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:34.710332 12619 consensus_queue.cc:260] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:34.711244 12619 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:34.713968 12543 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 3.
I20260709 12:55:34.715343 12658 leader_election.cc:304] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:34.716141 12833 raft_consensus.cc:2804] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:34.716555 12833 raft_consensus.cc:697] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 LEADER]: Becoming Leader. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Running, Role: LEADER
I20260709 12:55:34.716888 12619 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 3.
I20260709 12:55:34.717386 12833 consensus_queue.cc:237] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:34.725703 12440 catalog_manager.cc:5697] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 reported cstate change: term changed from 2 to 3, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:35.099290 12543 raft_consensus.cc:1275] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:35.101140 12833 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:55:35.111896 12619 raft_consensus.cc:1275] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:35.116211 12851 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:55:35.184053 12543 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:35.185400 12851 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:35.195245 12618 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:35.196944 12851 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:55:35.534965 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:35.547528 12543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:53790
I20260709 12:55:35.548422 12543 raft_consensus.cc:606] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:35.555039 12543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:53790
I20260709 12:55:35.555601 12543 raft_consensus.cc:606] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:35.562408 12543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:53790
I20260709 12:55:35.563094 12543 raft_consensus.cc:606] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:35.579249 12691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743"
dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:40404
I20260709 12:55:35.579898 12691 raft_consensus.cc:606] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:35.592674 12691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
mode: GRACEFUL
new_leader_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:40404
I20260709 12:55:35.593401 12691 raft_consensus.cc:606] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 LEADER]: Received request to transfer leadership to TS a9961e5157c1489f829d80a64a117246
I20260709 12:55:35.607184 12841 raft_consensus.cc:993] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.607723 12841 raft_consensus.cc:1081] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.609333 12618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34840
I20260709 12:55:35.609848 12618 raft_consensus.cc:493] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:35.610253 12618 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.615803 12618 raft_consensus.cc:515] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.618132 12618 leader_election.cc:290] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:35.618566 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:35.618839 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:35.619163 12543 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.619477 12691 raft_consensus.cc:3055] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:35.619841 12691 raft_consensus.cc:740] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Running, Role: LEADER
I20260709 12:55:35.620712 12691 consensus_queue.cc:260] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.622073 12691 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.624303 12543 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:35.625476 12584 leader_election.cc:304] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:35.626314 12834 raft_consensus.cc:2804] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:35.627426 12834 raft_consensus.cc:697] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:35.628216 12691 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:35.628414 12834 consensus_queue.cc:237] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.635085 12440 catalog_manager.cc:5697] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 2 to 3, leader changed from 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 3 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:35.669075 12851 raft_consensus.cc:993] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.669589 12841 raft_consensus.cc:1081] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.671816 12618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34840
I20260709 12:55:35.672485 12618 raft_consensus.cc:493] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:35.672878 12618 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:35.679035 12618 raft_consensus.cc:515] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.681537 12618 leader_election.cc:290] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 4 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:35.682551 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:35.683013 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:35.683151 12543 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:35.683646 12691 raft_consensus.cc:3055] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:55:35.683949 12691 raft_consensus.cc:740] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Running, Role: LEADER
I20260709 12:55:35.685014 12691 consensus_queue.cc:260] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.686271 12691 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:35.688179 12543 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 4.
I20260709 12:55:35.689144 12584 leader_election.cc:304] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:35.689926 12834 raft_consensus.cc:2804] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:55:35.690322 12834 raft_consensus.cc:697] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:35.691107 12834 consensus_queue.cc:237] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.692567 12691 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 4.
I20260709 12:55:35.698765 12440 catalog_manager.cc:5697] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 3 to 4, leader changed from 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 4 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:35.818257 12864 raft_consensus.cc:993] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.818851 12864 raft_consensus.cc:1081] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.819418 12865 raft_consensus.cc:993] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.819886 12864 raft_consensus.cc:1081] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:35.820537 12618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34830
I20260709 12:55:35.821215 12618 raft_consensus.cc:493] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:35.821524 12618 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.821523 12619 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34830
I20260709 12:55:35.822177 12619 raft_consensus.cc:493] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:35.822439 12619 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.826282 12619 raft_consensus.cc:515] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.826489 12618 raft_consensus.cc:515] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.827854 12619 leader_election.cc:290] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:35.828186 12618 leader_election.cc:290] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:35.828552 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:35.829080 12543 raft_consensus.cc:3055] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:35.829020 12544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:35.829663 12544 raft_consensus.cc:3055] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:35.829948 12544 raft_consensus.cc:740] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:35.830127 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f547f26a6ee4126852ef442f97a92d5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:35.830441 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:35.830821 12692 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.830891 12691 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.830901 12544 consensus_queue.cc:260] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.832341 12544 raft_consensus.cc:3060] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.835626 12692 raft_consensus.cc:2468] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:35.829519 12543 raft_consensus.cc:740] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:35.836899 12582 leader_election.cc:304] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:35.837055 12691 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:35.836707 12543 consensus_queue.cc:260] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.837796 12834 raft_consensus.cc:2804] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:35.838224 12582 leader_election.cc:304] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:35.838320 12834 raft_consensus.cc:697] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:35.838575 12544 raft_consensus.cc:2468] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:35.839093 12860 raft_consensus.cc:2804] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:35.839563 12860 raft_consensus.cc:697] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:35.839339 12834 consensus_queue.cc:237] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.838353 12543 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:35.840272 12860 consensus_queue.cc:237] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:35.846657 12543 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:35.847123 12440 catalog_manager.cc:5697] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 2 to 3, leader changed from 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 3 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:35.859071 12440 catalog_manager.cc:5697] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 2 to 3, leader changed from 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 3 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:36.097189 12865 raft_consensus.cc:993] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7: : Instructing follower a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:36.097697 12864 raft_consensus.cc:1081] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Signalling peer a9961e5157c1489f829d80a64a117246 to start an election
I20260709 12:55:36.097783 12543 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:36.098902 12860 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:55:36.099303 12618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
 from {username='slave'} at 127.0.0.1:34830
I20260709 12:55:36.099861 12618 raft_consensus.cc:493] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:36.100284 12618 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:36.109098 12618 raft_consensus.cc:515] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:36.112022 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:36.112022 12543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea60e9b396c4cd896d7b9a2ab2fe7e5" candidate_uuid: "a9961e5157c1489f829d80a64a117246" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:36.112727 12691 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:36.112813 12543 raft_consensus.cc:3055] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:36.113235 12543 raft_consensus.cc:740] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:36.114049 12543 consensus_queue.cc:260] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:36.115159 12543 raft_consensus.cc:3060] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:36.115176 12618 leader_election.cc:290] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:36.119841 12691 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:36.121234 12582 leader_election.cc:304] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:36.121567 12543 raft_consensus.cc:2468] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9961e5157c1489f829d80a64a117246 in term 3.
I20260709 12:55:36.122416 12859 raft_consensus.cc:2804] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:36.123037 12859 raft_consensus.cc:697] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Becoming Leader. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:36.123853 12859 consensus_queue.cc:237] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:36.124827 12691 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:36.125993 12834 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:55:36.132355 12440 catalog_manager.cc:5697] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 reported cstate change: term changed from 2 to 3, leader changed from 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65) to a9961e5157c1489f829d80a64a117246 (127.12.30.66). New cstate: current_term: 3 leader_uuid: "a9961e5157c1489f829d80a64a117246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:36.158262 12691 raft_consensus.cc:1275] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.159679 12834 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:36.168589 12543 raft_consensus.cc:1275] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.170037 12870 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:36.262533 12691 raft_consensus.cc:1275] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.262672 12543 raft_consensus.cc:1275] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.264091 12834 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:36.264642 12870 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:36.323094 12691 raft_consensus.cc:1275] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.324321 12870 consensus_queue.cc:1048] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:36.332357 12543 raft_consensus.cc:1275] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.334378 12859 consensus_queue.cc:1048] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:36.623675 12691 raft_consensus.cc:1275] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.624977 12859 consensus_queue.cc:1048] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:36.634433 12543 raft_consensus.cc:1275] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Refusing update from remote peer a9961e5157c1489f829d80a64a117246: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:36.636101 12834 consensus_queue.cc:1048] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:55:37.014693 12565 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5060 after lost signal to thread 12412
W20260709 12:55:37.015693 12565 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5740 after lost signal to thread 12475
W20260709 12:55:37.016641 12565 debug-util.cc:398] Leaking SignalData structure 0x7b08001f0160 after lost signal to thread 12568
W20260709 12:55:37.017537 12565 debug-util.cc:398] Leaking SignalData structure 0x7b080019cf40 after lost signal to thread 12643
W20260709 12:55:37.018301 12565 debug-util.cc:398] Leaking SignalData structure 0x7b08001eba40 after lost signal to thread 12717
I20260709 12:55:38.669627 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:38.677101 12618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:49994
I20260709 12:55:38.677888 12618 raft_consensus.cc:606] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Received request to transfer leadership to TS 3b7c89c0026a401b9f38d8439dec5d24
I20260709 12:55:38.681164 12618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:49994
I20260709 12:55:38.681715 12618 raft_consensus.cc:606] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Received request to transfer leadership to TS 5f140077bd674645a89e0f96ed93c6a7
I20260709 12:55:38.684465 12618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:49994
I20260709 12:55:38.684969 12618 raft_consensus.cc:606] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 LEADER]: Received request to transfer leadership to TS 5f140077bd674645a89e0f96ed93c6a7
I20260709 12:55:38.687665 12618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:49994
I20260709 12:55:38.688177 12618 raft_consensus.cc:606] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Received request to transfer leadership to TS 3b7c89c0026a401b9f38d8439dec5d24
I20260709 12:55:38.690513 12618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69"
dest_uuid: "a9961e5157c1489f829d80a64a117246"
mode: GRACEFUL
new_leader_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:49994
I20260709 12:55:38.691020 12618 raft_consensus.cc:606] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Received request to transfer leadership to TS 5f140077bd674645a89e0f96ed93c6a7
I20260709 12:55:38.692050 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:55:38.692478 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:55:38.842999 12885 raft_consensus.cc:993] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: : Instructing follower 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:38.843545 12860 raft_consensus.cc:1081] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 LEADER]: Signalling peer 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:38.845458 12542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:57602
I20260709 12:55:38.846129 12542 raft_consensus.cc:493] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:38.846601 12542 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:55:38.855403 12542 raft_consensus.cc:515] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.857499 12860 raft_consensus.cc:993] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246: : Instructing follower 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:38.858062 12860 raft_consensus.cc:1081] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Signalling peer 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:38.859014 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:38.859329 12692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82337432dcdf429882e1d51ee37f2445" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:38.859050 12542 leader_election.cc:290] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 5 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:38.859867 12618 raft_consensus.cc:3055] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:55:38.860004 12692 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:55:38.860369 12618 raft_consensus.cc:740] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:38.861423 12618 consensus_queue.cc:260] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.862994 12691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743"
dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:33348
I20260709 12:55:38.863368 12618 raft_consensus.cc:3060] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:55:38.863724 12691 raft_consensus.cc:493] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:38.864198 12691 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:38.868427 12692 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 5.
I20260709 12:55:38.869937 12507 leader_election.cc:304] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:38.874529 12691 raft_consensus.cc:515] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.877259 12691 leader_election.cc:290] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 4 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:38.877657 12618 raft_consensus.cc:2468] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 5.
I20260709 12:55:38.878381 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:38.878963 12619 raft_consensus.cc:3055] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:55:38.879251 12619 raft_consensus.cc:740] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:38.879999 12542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1d5ab2b5c04370a27121532c65e743" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:38.880203 12886 raft_consensus.cc:2804] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:55:38.880091 12619 consensus_queue.cc:260] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.880684 12542 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:38.881433 12619 raft_consensus.cc:3060] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:38.888304 12619 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 4.
I20260709 12:55:38.888738 12886 raft_consensus.cc:697] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 5 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:38.888304 12542 raft_consensus.cc:2468] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 4.
I20260709 12:55:38.889909 12659 leader_election.cc:304] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:38.891161 12851 raft_consensus.cc:2804] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:55:38.889955 12886 consensus_queue.cc:237] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.892489 12851 raft_consensus.cc:697] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 LEADER]: Becoming Leader. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Running, Role: LEADER
I20260709 12:55:38.894282 12851 consensus_queue.cc:237] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.897152 12885 raft_consensus.cc:993] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246: : Instructing follower 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:38.897723 12885 raft_consensus.cc:1081] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Signalling peer 3b7c89c0026a401b9f38d8439dec5d24 to start an election
I20260709 12:55:38.899703 12691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156"
dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
 from {username='slave'} at 127.0.0.1:33348
I20260709 12:55:38.900290 12691 raft_consensus.cc:493] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:38.900651 12691 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:38.908512 12691 raft_consensus.cc:515] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.911566 12542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
I20260709 12:55:38.912328 12542 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:38.924310 12542 raft_consensus.cc:2468] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 3.
I20260709 12:55:38.925644 12658 leader_election.cc:304] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
I20260709 12:55:38.926774 12888 raft_consensus.cc:2804] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:38.932512 12888 raft_consensus.cc:697] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 LEADER]: Becoming Leader. State: Replica: 3b7c89c0026a401b9f38d8439dec5d24, State: Running, Role: LEADER
I20260709 12:55:38.933678 12888 consensus_queue.cc:237] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.936939 12691 leader_election.cc:290] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [CANDIDATE]: Term 3 election: Requested vote from peers 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093), a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517)
I20260709 12:55:38.940455 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac008d3702446898ff4f4e463f78156" candidate_uuid: "3b7c89c0026a401b9f38d8439dec5d24" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:38.941251 12619 raft_consensus.cc:3055] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:38.941583 12619 raft_consensus.cc:740] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:38.942345 12619 consensus_queue.cc:260] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.943624 12619 raft_consensus.cc:3060] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:38.951330 12440 catalog_manager.cc:5697] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 4 to 5, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:38.957790 12619 raft_consensus.cc:2468] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7c89c0026a401b9f38d8439dec5d24 in term 3.
I20260709 12:55:38.963860 12860 raft_consensus.cc:993] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: : Instructing follower 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:38.959903 12438 catalog_manager.cc:5697] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 reported cstate change: term changed from 3 to 4, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:38.975572 12896 raft_consensus.cc:1081] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Signalling peer 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:38.979148 12544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:57602
I20260709 12:55:38.979732 12544 raft_consensus.cc:493] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:38.980221 12544 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:38.993991 12860 raft_consensus.cc:993] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246: : Instructing follower 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:38.994524 12860 raft_consensus.cc:1081] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Signalling peer 5f140077bd674645a89e0f96ed93c6a7 to start an election
I20260709 12:55:38.995301 12544 raft_consensus.cc:515] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:38.996593 12542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6"
dest_uuid: "5f140077bd674645a89e0f96ed93c6a7"
 from {username='slave'} at 127.0.0.1:57602
I20260709 12:55:38.997227 12542 raft_consensus.cc:493] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:38.997629 12542 raft_consensus.cc:3060] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:38.999083 12544 leader_election.cc:290] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 4 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:39.000931 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:39.001518 12691 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:39.001528 12619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb2d78d6920a4691bf741f673273af69" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:39.002347 12619 raft_consensus.cc:3055] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:55:39.002710 12619 raft_consensus.cc:740] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:39.003726 12619 consensus_queue.cc:260] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:39.005021 12619 raft_consensus.cc:3060] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:39.009441 12691 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 4.
I20260709 12:55:39.011574 12507 leader_election.cc:304] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7c89c0026a401b9f38d8439dec5d24, 5f140077bd674645a89e0f96ed93c6a7; no voters: 
W20260709 12:55:39.012843 12472 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 12412
I20260709 12:55:39.013322 12886 raft_consensus.cc:2804] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:55:39.013746 12472 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 12475
I20260709 12:55:39.013830 12886 raft_consensus.cc:697] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
W20260709 12:55:39.014891 12472 debug-util.cc:398] Leaking SignalData structure 0x7b0800147500 after lost signal to thread 12568
I20260709 12:55:39.014833 12886 consensus_queue.cc:237] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
W20260709 12:55:39.015864 12472 debug-util.cc:398] Leaking SignalData structure 0x7b08000be600 after lost signal to thread 12643
I20260709 12:55:39.016031 12542 raft_consensus.cc:515] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
W20260709 12:55:39.016855 12472 debug-util.cc:398] Leaking SignalData structure 0x7b08001199e0 after lost signal to thread 12717
I20260709 12:55:39.019361 12618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9961e5157c1489f829d80a64a117246"
I20260709 12:55:39.019928 12618 raft_consensus.cc:3055] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:39.020228 12618 raft_consensus.cc:740] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9961e5157c1489f829d80a64a117246, State: Running, Role: LEADER
I20260709 12:55:39.020988 12618 consensus_queue.cc:260] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:39.022118 12618 raft_consensus.cc:3060] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:39.024159 12619 raft_consensus.cc:2468] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 4.
I20260709 12:55:39.027065 12691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71eaffb19a734e94847e7ec6efd155e6" candidate_uuid: "5f140077bd674645a89e0f96ed93c6a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b7c89c0026a401b9f38d8439dec5d24"
I20260709 12:55:39.027601 12691 raft_consensus.cc:3060] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:39.029033 12618 raft_consensus.cc:2468] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:39.030206 12510 leader_election.cc:304] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f140077bd674645a89e0f96ed93c6a7, a9961e5157c1489f829d80a64a117246; no voters: 
I20260709 12:55:39.031286 12542 leader_election.cc:290] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [CANDIDATE]: Term 2 election: Requested vote from peers a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517), 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67:33627)
I20260709 12:55:39.031383 12887 raft_consensus.cc:2804] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:39.038827 12691 raft_consensus.cc:2468] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f140077bd674645a89e0f96ed93c6a7 in term 2.
I20260709 12:55:39.046705 12440 catalog_manager.cc:5697] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 3 to 4, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 4 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:39.040913 12438 catalog_manager.cc:5697] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 reported cstate change: term changed from 2 to 3, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 3b7c89c0026a401b9f38d8439dec5d24 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "3b7c89c0026a401b9f38d8439dec5d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:39.053714 12887 raft_consensus.cc:697] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Becoming Leader. State: Replica: 5f140077bd674645a89e0f96ed93c6a7, State: Running, Role: LEADER
I20260709 12:55:39.054741 12887 consensus_queue.cc:237] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } }
I20260709 12:55:39.072697 12438 catalog_manager.cc:5697] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 reported cstate change: term changed from 1 to 2, leader changed from a9961e5157c1489f829d80a64a117246 (127.12.30.66) to 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "5f140077bd674645a89e0f96ed93c6a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:39.335016 12542 raft_consensus.cc:1275] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:39.336740 12851 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:55:39.352237 12619 raft_consensus.cc:1275] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:39.354648 12888 consensus_queue.cc:1048] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:55:39.383040 12691 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 5 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:55:39.384706 12887 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:55:39.390856 12619 raft_consensus.cc:1275] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:39.393224 12891 consensus_queue.cc:1048] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:55:39.429350 12692 raft_consensus.cc:1275] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:39.429601 12542 raft_consensus.cc:1275] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Refusing update from remote peer 3b7c89c0026a401b9f38d8439dec5d24: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:39.431197 12851 consensus_queue.cc:1048] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f140077bd674645a89e0f96ed93c6a7" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:39.433971 12619 raft_consensus.cc:1275] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 5 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:55:39.435525 12901 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:55:39.436954 12886 consensus_queue.cc:1048] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:55:39.486605 12618 raft_consensus.cc:1275] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:39.489265 12901 consensus_queue.cc:1048] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:55:39.506350 12692 raft_consensus.cc:1275] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:39.508034 12901 consensus_queue.cc:1048] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7c89c0026a401b9f38d8439dec5d24" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 33627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:39.514710 12619 raft_consensus.cc:1275] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Refusing update from remote peer 5f140077bd674645a89e0f96ed93c6a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:39.518682 12901 consensus_queue.cc:1048] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9961e5157c1489f829d80a64a117246" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:55:39.693483 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:39.702806 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
W20260709 12:55:39.717849 12584 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:55:39.731886 12584 consensus_peers.cc:597] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 -> Peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Couldn't send request to peer 5f140077bd674645a89e0f96ed93c6a7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:39.762630 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:55:39.763607 12409 tablet_replica.cc:333] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.764724 12409 raft_consensus.cc:2243] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 5 LEADER]: Raft consensus shutting down.
W20260709 12:55:39.765059 12658 consensus_peers.cc:597] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 -> Peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Couldn't send request to peer 5f140077bd674645a89e0f96ed93c6a7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:39.765753 12409 raft_consensus.cc:2272] T 82337432dcdf429882e1d51ee37f2445 P 5f140077bd674645a89e0f96ed93c6a7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.769749 12409 tablet_replica.cc:333] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.770358 12409 raft_consensus.cc:2243] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:39.770836 12409 raft_consensus.cc:2272] T 9ac008d3702446898ff4f4e463f78156 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.774094 12409 tablet_replica.cc:333] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.774693 12409 raft_consensus.cc:2243] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:55:39.775417 12409 raft_consensus.cc:2272] T 71eaffb19a734e94847e7ec6efd155e6 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.777684 12409 tablet_replica.cc:333] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.778304 12409 raft_consensus.cc:2243] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:39.778882 12409 raft_consensus.cc:2272] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.781428 12409 tablet_replica.cc:333] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.782239 12409 raft_consensus.cc:2243] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:39.782827 12409 raft_consensus.cc:2272] T 4c1d5ab2b5c04370a27121532c65e743 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.785928 12409 tablet_replica.cc:333] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.786681 12409 raft_consensus.cc:2243] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:39.787212 12409 raft_consensus.cc:2272] T 6f547f26a6ee4126852ef442f97a92d5 P 5f140077bd674645a89e0f96ed93c6a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.789503 12409 tablet_replica.cc:333] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.790086 12409 raft_consensus.cc:2243] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:39.790557 12409 raft_consensus.cc:2272] T 18a6efafe35a4b0dad3b1a0b7e192024 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.792430 12409 tablet_replica.cc:333] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.792906 12409 raft_consensus.cc:2243] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:55:39.793639 12409 raft_consensus.cc:2272] T cb2d78d6920a4691bf741f673273af69 P 5f140077bd674645a89e0f96ed93c6a7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:39.795730 12409 tablet_replica.cc:333] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7: stopping tablet replica
I20260709 12:55:39.796237 12409 raft_consensus.cc:2243] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:39.796751 12409 raft_consensus.cc:2272] T dac941ae651a414ea7db7fa8c6e86034 P 5f140077bd674645a89e0f96ed93c6a7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:55:39.965018 12658 consensus_peers.cc:597] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 -> Peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Couldn't send request to peer 5f140077bd674645a89e0f96ed93c6a7. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:42093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:55:39.970108 12584 consensus_peers.cc:597] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 -> Peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Couldn't send request to peer 5f140077bd674645a89e0f96ed93c6a7. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:42093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:55:40.128863 12584 consensus_peers.cc:597] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 -> Peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Couldn't send request to peer 5f140077bd674645a89e0f96ed93c6a7. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:42093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:55:40.150753 12658 consensus_peers.cc:597] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 -> Peer 5f140077bd674645a89e0f96ed93c6a7 (127.12.30.65:42093): Couldn't send request to peer 5f140077bd674645a89e0f96ed93c6a7. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:42093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:40.288403 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
I20260709 12:55:40.319881 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
I20260709 12:55:40.353462 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:55:40.354377 12409 tablet_replica.cc:333] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.355533 12409 raft_consensus.cc:2243] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.356249 12409 raft_consensus.cc:2272] T 82337432dcdf429882e1d51ee37f2445 P a9961e5157c1489f829d80a64a117246 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.359179 12409 tablet_replica.cc:333] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.359687 12409 raft_consensus.cc:2243] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.360133 12409 raft_consensus.cc:2272] T 9ac008d3702446898ff4f4e463f78156 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.362109 12409 tablet_replica.cc:333] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.362645 12409 raft_consensus.cc:2243] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:55:40.363400 12409 raft_consensus.cc:2272] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.365295 12409 tablet_replica.cc:333] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.365782 12409 raft_consensus.cc:2243] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.366176 12409 raft_consensus.cc:2272] T 4c1d5ab2b5c04370a27121532c65e743 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.368356 12409 tablet_replica.cc:333] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.368854 12409 raft_consensus.cc:2243] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:55:40.369575 12409 raft_consensus.cc:2272] T 6f547f26a6ee4126852ef442f97a92d5 P a9961e5157c1489f829d80a64a117246 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.371232 12409 tablet_replica.cc:333] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.371696 12409 raft_consensus.cc:2243] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.372099 12409 raft_consensus.cc:2272] T 71eaffb19a734e94847e7ec6efd155e6 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.374235 12409 tablet_replica.cc:333] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.374732 12409 raft_consensus.cc:2243] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.375133 12409 raft_consensus.cc:2272] T 18a6efafe35a4b0dad3b1a0b7e192024 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.376818 12409 tablet_replica.cc:333] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.377256 12409 raft_consensus.cc:2243] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.377640 12409 raft_consensus.cc:2272] T cb2d78d6920a4691bf741f673273af69 P a9961e5157c1489f829d80a64a117246 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.379832 12409 tablet_replica.cc:333] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246: stopping tablet replica
I20260709 12:55:40.380340 12409 raft_consensus.cc:2243] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:55:40.381048 12409 raft_consensus.cc:2272] T dac941ae651a414ea7db7fa8c6e86034 P a9961e5157c1489f829d80a64a117246 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.411473 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
I20260709 12:55:40.430568 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
I20260709 12:55:40.457177 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:55:40.458637 12409 tablet_replica.cc:333] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.459707 12409 raft_consensus.cc:2243] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.460438 12409 raft_consensus.cc:2272] T 71eaffb19a734e94847e7ec6efd155e6 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.462975 12409 tablet_replica.cc:333] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.463587 12409 raft_consensus.cc:2243] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.464221 12409 raft_consensus.cc:2272] T dea60e9b396c4cd896d7b9a2ab2fe7e5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:55:40.464887 12659 consensus_peers.cc:597] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 -> Peer a9961e5157c1489f829d80a64a117246 (127.12.30.66:35517): Couldn't send request to peer a9961e5157c1489f829d80a64a117246. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:35517: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:40.466032 12409 tablet_replica.cc:333] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.466598 12409 raft_consensus.cc:2243] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.467113 12409 raft_consensus.cc:2272] T 82337432dcdf429882e1d51ee37f2445 P 3b7c89c0026a401b9f38d8439dec5d24 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.468811 12409 tablet_replica.cc:333] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.469333 12409 raft_consensus.cc:2243] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.469930 12409 raft_consensus.cc:2272] T cb2d78d6920a4691bf741f673273af69 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.472224 12409 tablet_replica.cc:333] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.473043 12409 raft_consensus.cc:2243] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:55:40.473871 12409 raft_consensus.cc:2272] T 18a6efafe35a4b0dad3b1a0b7e192024 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.475525 12409 tablet_replica.cc:333] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.476138 12409 raft_consensus.cc:2243] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.476558 12409 raft_consensus.cc:2272] T 6f547f26a6ee4126852ef442f97a92d5 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.478276 12409 tablet_replica.cc:333] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.478788 12409 raft_consensus.cc:2243] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:55:40.479506 12409 raft_consensus.cc:2272] T 4c1d5ab2b5c04370a27121532c65e743 P 3b7c89c0026a401b9f38d8439dec5d24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.481299 12409 tablet_replica.cc:333] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.481786 12409 raft_consensus.cc:2243] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:55:40.482486 12409 raft_consensus.cc:2272] T 9ac008d3702446898ff4f4e463f78156 P 3b7c89c0026a401b9f38d8439dec5d24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.484345 12409 tablet_replica.cc:333] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24: stopping tablet replica
I20260709 12:55:40.484805 12409 raft_consensus.cc:2243] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:40.485200 12409 raft_consensus.cc:2272] T dac941ae651a414ea7db7fa8c6e86034 P 3b7c89c0026a401b9f38d8439dec5d24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.515309 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
I20260709 12:55:40.534044 12409 master.cc:562] Master@127.12.30.126:34439 shutting down...
I20260709 12:55:40.691562 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:55:40.692342 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:40.692816 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P b2f0cdf666794e69b2e20ac775b719f0: stopping tablet replica
W20260709 12:55:41.710692 12409 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 12472)
I20260709 12:55:41.733677 12409 master.cc:584] Master@127.12.30.126:34439 shutdown complete.
3b7c89c0026a401b9f38d8439dec5d24  1
5f140077bd674645a89e0f96ed93c6a7  4
a9961e5157c1489f829d80a64a117246  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (14610 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:55:41.777034 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:37625
I20260709 12:55:41.778218 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:41.783775 12927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:41.785212 12926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:41.786528 12929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:41.787741 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:41.789214 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:41.789535 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:41.789805 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601741789755 us; error 0 us; skew 500 ppm
I20260709 12:55:41.790447 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:41.793710 12409 webserver.cc:533] Webserver started at http://127.12.30.126:35607/ using document root <none> and password file <none>
I20260709 12:55:41.794308 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:41.794505 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:41.794806 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:41.796241 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "b505f0bdb6434721959ef72d5d2c2746"
format_stamp: "Formatted at 2026-07-09 12:55:41 on dist-test-slave-mlt7"
I20260709 12:55:41.800798 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:55:41.804255 12934 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:41.805111 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:55:41.805441 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "b505f0bdb6434721959ef72d5d2c2746"
format_stamp: "Formatted at 2026-07-09 12:55:41 on dist-test-slave-mlt7"
I20260709 12:55:41.805747 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:41.817764 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:41.819044 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:41.857147 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:37625
I20260709 12:55:41.857288 12985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:37625 every 8 connection(s)
I20260709 12:55:41.861316 12986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:41.871873 12986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746: Bootstrap starting.
I20260709 12:55:41.877005 12986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:41.881742 12986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746: No bootstrap required, opened a new log
I20260709 12:55:41.883879 12986 raft_consensus.cc:359] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b505f0bdb6434721959ef72d5d2c2746" member_type: VOTER }
I20260709 12:55:41.884313 12986 raft_consensus.cc:385] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:41.884578 12986 raft_consensus.cc:740] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b505f0bdb6434721959ef72d5d2c2746, State: Initialized, Role: FOLLOWER
I20260709 12:55:41.885260 12986 consensus_queue.cc:260] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b505f0bdb6434721959ef72d5d2c2746" member_type: VOTER }
I20260709 12:55:41.885746 12986 raft_consensus.cc:399] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:55:41.885964 12986 raft_consensus.cc:493] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:55:41.886260 12986 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:41.890259 12986 raft_consensus.cc:515] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b505f0bdb6434721959ef72d5d2c2746" member_type: VOTER }
I20260709 12:55:41.890772 12986 leader_election.cc:304] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b505f0bdb6434721959ef72d5d2c2746; no voters: 
I20260709 12:55:41.891860 12986 leader_election.cc:290] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:55:41.892328 12989 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:41.893950 12989 raft_consensus.cc:697] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 1 LEADER]: Becoming Leader. State: Replica: b505f0bdb6434721959ef72d5d2c2746, State: Running, Role: LEADER
I20260709 12:55:41.894644 12989 consensus_queue.cc:237] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b505f0bdb6434721959ef72d5d2c2746" member_type: VOTER }
I20260709 12:55:41.894881 12986 sys_catalog.cc:565] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:55:41.898923 12990 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b505f0bdb6434721959ef72d5d2c2746" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b505f0bdb6434721959ef72d5d2c2746" member_type: VOTER } }
I20260709 12:55:41.899644 12990 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [sys.catalog]: This master's current role is: LEADER
I20260709 12:55:41.899488 12991 sys_catalog.cc:455] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b505f0bdb6434721959ef72d5d2c2746. Latest consensus state: current_term: 1 leader_uuid: "b505f0bdb6434721959ef72d5d2c2746" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b505f0bdb6434721959ef72d5d2c2746" member_type: VOTER } }
I20260709 12:55:41.900606 12991 sys_catalog.cc:458] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [sys.catalog]: This master's current role is: LEADER
I20260709 12:55:41.908367 12996 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:55:41.913697 12996 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:55:41.914736 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:55:41.923741 12996 catalog_manager.cc:1383] Generated new cluster ID: c2b21456123c4839b0dad2fa2ada7e00
I20260709 12:55:41.924105 12996 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:55:41.957531 12996 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:55:41.958719 12996 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:55:41.976632 12996 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746: Generated new TSK 0
I20260709 12:55:41.977260 12996 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:55:41.982208 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:41.988158 13007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:41.989292 13008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:41.990707 13010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:41.990878 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:41.991894 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:41.992149 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:41.992270 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601741992259 us; error 0 us; skew 500 ppm
I20260709 12:55:41.992727 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:41.994925 12409 webserver.cc:533] Webserver started at http://127.12.30.65:39757/ using document root <none> and password file <none>
I20260709 12:55:41.995368 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:41.995513 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:41.995733 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:41.996800 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "0da16b4100324a709e8662bc7053426c"
format_stamp: "Formatted at 2026-07-09 12:55:41 on dist-test-slave-mlt7"
I20260709 12:55:42.001205 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:55:42.004745 13015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.005645 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:55:42.006018 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "0da16b4100324a709e8662bc7053426c"
format_stamp: "Formatted at 2026-07-09 12:55:41 on dist-test-slave-mlt7"
I20260709 12:55:42.006306 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:42.023161 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:42.024600 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:42.026216 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:42.028645 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:42.028894 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.029151 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:42.029351 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.072713 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:42681
I20260709 12:55:42.072809 13077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:42681 every 8 connection(s)
I20260709 12:55:42.077709 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:42.085525 13082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:42.086606 13083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:42.090668 12409 server_base.cc:1061] running on GCE node
W20260709 12:55:42.091460 13085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:42.092377 13078 heartbeater.cc:344] Connected to a master server at 127.12.30.126:37625
I20260709 12:55:42.092443 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:42.092751 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:42.092831 13078 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:42.092937 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601742092917 us; error 0 us; skew 500 ppm
I20260709 12:55:42.093734 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:42.093775 13078 heartbeater.cc:507] Master 127.12.30.126:37625 requested a full tablet report, sending...
I20260709 12:55:42.096648 12951 ts_manager.cc:194] Registered new tserver with Master: 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681)
I20260709 12:55:42.096745 12409 webserver.cc:533] Webserver started at http://127.12.30.66:34163/ using document root <none> and password file <none>
I20260709 12:55:42.097334 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:42.097555 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:42.097906 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:42.098572 12951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42450
I20260709 12:55:42.099130 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "0e354d68b8c64e379bc71b28c3af85ec"
format_stamp: "Formatted at 2026-07-09 12:55:42 on dist-test-slave-mlt7"
I20260709 12:55:42.103765 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:55:42.107241 13090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.108150 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:55:42.108505 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "0e354d68b8c64e379bc71b28c3af85ec"
format_stamp: "Formatted at 2026-07-09 12:55:42 on dist-test-slave-mlt7"
I20260709 12:55:42.108923 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:42.132195 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:42.133412 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:42.134894 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:42.137055 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:42.137285 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.137538 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:42.137719 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.178877 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:36333
I20260709 12:55:42.178995 13152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:36333 every 8 connection(s)
I20260709 12:55:42.183585 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:42.191143 13156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:42.191612 13157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:42.198160 13160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:42.199165 13153 heartbeater.cc:344] Connected to a master server at 127.12.30.126:37625
I20260709 12:55:42.199551 13153 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:42.200186 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:42.200358 13153 heartbeater.cc:507] Master 127.12.30.126:37625 requested a full tablet report, sending...
I20260709 12:55:42.201162 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:42.201418 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:42.201607 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601742201590 us; error 0 us; skew 500 ppm
I20260709 12:55:42.202169 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:42.202528 12951 ts_manager.cc:194] Registered new tserver with Master: 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:42.204187 12951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42452
I20260709 12:55:42.204826 12409 webserver.cc:533] Webserver started at http://127.12.30.67:43469/ using document root <none> and password file <none>
I20260709 12:55:42.205336 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:42.205542 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:42.205808 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:42.207015 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "f9825019a5484ebe9114919d65c77120"
format_stamp: "Formatted at 2026-07-09 12:55:42 on dist-test-slave-mlt7"
I20260709 12:55:42.211681 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:55:42.215366 13164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.216231 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:55:42.216531 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "f9825019a5484ebe9114919d65c77120"
format_stamp: "Formatted at 2026-07-09 12:55:42 on dist-test-slave-mlt7"
I20260709 12:55:42.216823 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:42.229512 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:42.230729 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:42.232268 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:42.234450 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:42.234670 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.234911 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:42.235085 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:42.275985 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:37883
I20260709 12:55:42.276278 13226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:37883 every 8 connection(s)
I20260709 12:55:42.289602 13227 heartbeater.cc:344] Connected to a master server at 127.12.30.126:37625
I20260709 12:55:42.290019 13227 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:42.290802 13227 heartbeater.cc:507] Master 127.12.30.126:37625 requested a full tablet report, sending...
I20260709 12:55:42.292737 12951 ts_manager.cc:194] Registered new tserver with Master: f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:42.293601 12409 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014370319s
I20260709 12:55:42.294101 12951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42468
I20260709 12:55:42.319671 12951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42476:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:55:42.322499 12951 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:55:42.395160 13043 tablet_service.cc:1511] Processing CreateTablet for tablet 61bc33a644d94fc29c466d7d50d32b3f (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:55:42.395448 13042 tablet_service.cc:1511] Processing CreateTablet for tablet 6c167a5eddee447daab59df9e2233a24 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:55:42.396764 13043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61bc33a644d94fc29c466d7d50d32b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.398402 13042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c167a5eddee447daab59df9e2233a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.403113 13192 tablet_service.cc:1511] Processing CreateTablet for tablet 61bc33a644d94fc29c466d7d50d32b3f (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:55:42.405400 13118 tablet_service.cc:1511] Processing CreateTablet for tablet 61bc33a644d94fc29c466d7d50d32b3f (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:55:42.405745 13117 tablet_service.cc:1511] Processing CreateTablet for tablet 6c167a5eddee447daab59df9e2233a24 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:55:42.406638 13118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61bc33a644d94fc29c466d7d50d32b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.407234 13117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c167a5eddee447daab59df9e2233a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.408133 13041 tablet_service.cc:1511] Processing CreateTablet for tablet ee6637bd86a34221abc66eeaffb87405 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:55:42.408449 13191 tablet_service.cc:1511] Processing CreateTablet for tablet 6c167a5eddee447daab59df9e2233a24 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:55:42.409683 13041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6637bd86a34221abc66eeaffb87405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.410583 13192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61bc33a644d94fc29c466d7d50d32b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.416352 13191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c167a5eddee447daab59df9e2233a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.432868 13116 tablet_service.cc:1511] Processing CreateTablet for tablet ee6637bd86a34221abc66eeaffb87405 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:55:42.434254 13116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6637bd86a34221abc66eeaffb87405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.435359 13190 tablet_service.cc:1511] Processing CreateTablet for tablet ee6637bd86a34221abc66eeaffb87405 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:55:42.436841 13190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee6637bd86a34221abc66eeaffb87405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.447391 13040 tablet_service.cc:1511] Processing CreateTablet for tablet 2109da5ae593490ba3803074d274d6d3 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:55:42.455631 13115 tablet_service.cc:1511] Processing CreateTablet for tablet 2109da5ae593490ba3803074d274d6d3 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:55:42.457039 13115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2109da5ae593490ba3803074d274d6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.471470 13189 tablet_service.cc:1511] Processing CreateTablet for tablet 2109da5ae593490ba3803074d274d6d3 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:55:42.472929 13189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2109da5ae593490ba3803074d274d6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.486382 13039 tablet_service.cc:1511] Processing CreateTablet for tablet 4f37dd0690244d93b6acbadd20a93782 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:55:42.490897 13114 tablet_service.cc:1511] Processing CreateTablet for tablet 4f37dd0690244d93b6acbadd20a93782 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:55:42.492373 13114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f37dd0690244d93b6acbadd20a93782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.496558 13188 tablet_service.cc:1511] Processing CreateTablet for tablet 4f37dd0690244d93b6acbadd20a93782 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:55:42.497979 13188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f37dd0690244d93b6acbadd20a93782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.501085 13113 tablet_service.cc:1511] Processing CreateTablet for tablet e12d95d7e0bc4d26911222a16b8cdf40 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:55:42.502524 13113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12d95d7e0bc4d26911222a16b8cdf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.471500 13244 tablet_bootstrap.cc:492] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:42.510910 13039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f37dd0690244d93b6acbadd20a93782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.518344 13243 tablet_bootstrap.cc:492] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:42.519609 13041 tablet_service.cc:1511] Processing CreateTablet for tablet e12d95d7e0bc4d26911222a16b8cdf40 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:55:42.520202 13040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2109da5ae593490ba3803074d274d6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.520879 13041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12d95d7e0bc4d26911222a16b8cdf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.523623 13244 tablet_bootstrap.cc:654] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.548682 13243 tablet_bootstrap.cc:654] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.550478 13192 tablet_service.cc:1511] Processing CreateTablet for tablet e12d95d7e0bc4d26911222a16b8cdf40 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:55:42.551828 13192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12d95d7e0bc4d26911222a16b8cdf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.555459 13041 tablet_service.cc:1511] Processing CreateTablet for tablet 58abd247ac31444d9d71d6874fe90bb7 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:55:42.556749 13041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58abd247ac31444d9d71d6874fe90bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.568297 13242 tablet_bootstrap.cc:492] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:42.582579 13115 tablet_service.cc:1511] Processing CreateTablet for tablet 58abd247ac31444d9d71d6874fe90bb7 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:55:42.583937 13115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58abd247ac31444d9d71d6874fe90bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.585276 13189 tablet_service.cc:1511] Processing CreateTablet for tablet 58abd247ac31444d9d71d6874fe90bb7 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:55:42.585769 13041 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba09c8e124b4b6686f4d805bf766387 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:55:42.586248 13189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58abd247ac31444d9d71d6874fe90bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.587028 13041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba09c8e124b4b6686f4d805bf766387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.590144 13244 tablet_bootstrap.cc:492] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:42.590602 13244 ts_tablet_manager.cc:1403] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.120s	user 0.012s	sys 0.035s
I20260709 12:55:42.593675 13113 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba09c8e124b4b6686f4d805bf766387 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:55:42.595050 13113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba09c8e124b4b6686f4d805bf766387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.595733 13244 raft_consensus.cc:359] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.596618 13244 raft_consensus.cc:385] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.596964 13244 raft_consensus.cc:740] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.598673 13242 tablet_bootstrap.cc:654] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.598170 13244 consensus_queue.cc:260] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.600366 13227 heartbeater.cc:499] Master 127.12.30.126:37625 was elected leader, sending a full tablet report...
I20260709 12:55:42.608608 13243 tablet_bootstrap.cc:492] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:42.608928 13244 ts_tablet_manager.cc:1434] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.007s
I20260709 12:55:42.609113 13243 ts_tablet_manager.cc:1403] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.013s
I20260709 12:55:42.609925 13244 tablet_bootstrap.cc:492] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:42.610110 13188 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba09c8e124b4b6686f4d805bf766387 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:55:42.618631 13188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba09c8e124b4b6686f4d805bf766387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.618695 13244 tablet_bootstrap.cc:654] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.620481 13040 tablet_service.cc:1511] Processing CreateTablet for tablet abd5e447b5bd4309ac709b3732072473 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:55:42.621963 13040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd5e447b5bd4309ac709b3732072473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.621968 13243 raft_consensus.cc:359] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.622880 13243 raft_consensus.cc:385] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.623188 13243 raft_consensus.cc:740] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.623976 13243 consensus_queue.cc:260] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.637456 13115 tablet_service.cc:1511] Processing CreateTablet for tablet abd5e447b5bd4309ac709b3732072473 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:55:42.639351 13115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd5e447b5bd4309ac709b3732072473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.642637 13242 tablet_bootstrap.cc:492] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:42.643134 13242 ts_tablet_manager.cc:1403] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.075s	user 0.015s	sys 0.004s
I20260709 12:55:42.643769 13189 tablet_service.cc:1511] Processing CreateTablet for tablet abd5e447b5bd4309ac709b3732072473 (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:55:42.645068 13189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd5e447b5bd4309ac709b3732072473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.646260 13041 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4c36c3db804264b072c71ea9da961d (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:55:42.647528 13041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4c36c3db804264b072c71ea9da961d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.648671 13114 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4c36c3db804264b072c71ea9da961d (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:55:42.649931 13114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4c36c3db804264b072c71ea9da961d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.650964 13242 raft_consensus.cc:359] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.652261 13242 raft_consensus.cc:385] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.652565 13242 raft_consensus.cc:740] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.653443 13242 consensus_queue.cc:260] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.656769 13153 heartbeater.cc:499] Master 127.12.30.126:37625 was elected leader, sending a full tablet report...
I20260709 12:55:42.657087 13243 ts_tablet_manager.cc:1434] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.048s	user 0.014s	sys 0.010s
I20260709 12:55:42.658103 13243 tablet_bootstrap.cc:492] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:42.664570 13243 tablet_bootstrap.cc:654] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.669736 13244 tablet_bootstrap.cc:492] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:42.670222 13244 ts_tablet_manager.cc:1403] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.005s
I20260709 12:55:42.673843 13192 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4c36c3db804264b072c71ea9da961d (DEFAULT_TABLE table=test-workload [id=4e7f87ebc5bc4aa6902afd9754b52c0f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:55:42.690557 13244 raft_consensus.cc:359] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.691350 13244 raft_consensus.cc:385] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.691641 13244 raft_consensus.cc:740] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.692211 13078 heartbeater.cc:499] Master 127.12.30.126:37625 was elected leader, sending a full tablet report...
I20260709 12:55:42.692337 13244 consensus_queue.cc:260] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.693387 13192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4c36c3db804264b072c71ea9da961d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:42.694509 13244 ts_tablet_manager.cc:1434] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:55:42.710570 13244 tablet_bootstrap.cc:492] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:42.721344 13243 tablet_bootstrap.cc:492] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:42.721894 13243 ts_tablet_manager.cc:1403] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.064s	user 0.005s	sys 0.009s
I20260709 12:55:42.724817 13243 raft_consensus.cc:359] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.725558 13243 raft_consensus.cc:385] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.725862 13243 raft_consensus.cc:740] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.726733 13243 consensus_queue.cc:260] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.729143 13243 ts_tablet_manager.cc:1434] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:55:42.730126 13243 tablet_bootstrap.cc:492] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:42.732445 13242 ts_tablet_manager.cc:1434] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.089s	user 0.018s	sys 0.006s
I20260709 12:55:42.733505 13242 tablet_bootstrap.cc:492] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:42.736574 13243 tablet_bootstrap.cc:654] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.745466 13244 tablet_bootstrap.cc:654] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.747800 13243 tablet_bootstrap.cc:492] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:42.749826 13243 ts_tablet_manager.cc:1403] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.006s
I20260709 12:55:42.753942 13242 tablet_bootstrap.cc:654] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.753748 13243 raft_consensus.cc:359] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.754546 13243 raft_consensus.cc:385] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.754899 13243 raft_consensus.cc:740] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.755725 13243 consensus_queue.cc:260] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.759035 13243 ts_tablet_manager.cc:1434] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:55:42.760105 13243 tablet_bootstrap.cc:492] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:42.768592 13243 tablet_bootstrap.cc:654] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.769562 13244 tablet_bootstrap.cc:492] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:42.770054 13244 ts_tablet_manager.cc:1403] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.011s
I20260709 12:55:42.773238 13244 raft_consensus.cc:359] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.774019 13244 raft_consensus.cc:385] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.774333 13244 raft_consensus.cc:740] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.774631 13242 tablet_bootstrap.cc:492] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:42.775249 13242 ts_tablet_manager.cc:1403] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.009s
I20260709 12:55:42.775241 13244 consensus_queue.cc:260] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.779328 13242 raft_consensus.cc:359] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.780088 13242 raft_consensus.cc:385] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.780385 13242 raft_consensus.cc:740] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.780987 13243 tablet_bootstrap.cc:492] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:42.781448 13243 ts_tablet_manager.cc:1403] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.003s
I20260709 12:55:42.781288 13242 consensus_queue.cc:260] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.783450 13242 ts_tablet_manager.cc:1434] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:55:42.784384 13242 tablet_bootstrap.cc:492] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:42.784385 13243 raft_consensus.cc:359] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.785213 13243 raft_consensus.cc:385] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.785512 13243 raft_consensus.cc:740] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.786404 13243 consensus_queue.cc:260] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.790503 13244 ts_tablet_manager.cc:1434] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.002s
I20260709 12:55:42.791435 13243 ts_tablet_manager.cc:1434] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:55:42.792404 13243 tablet_bootstrap.cc:492] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:42.799036 13243 tablet_bootstrap.cc:654] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.791720 13244 tablet_bootstrap.cc:492] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:42.800383 13242 tablet_bootstrap.cc:654] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.806720 13244 tablet_bootstrap.cc:654] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.810840 13243 tablet_bootstrap.cc:492] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:42.811342 13242 tablet_bootstrap.cc:492] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:42.811411 13243 ts_tablet_manager.cc:1403] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.010s
I20260709 12:55:42.813519 13242 ts_tablet_manager.cc:1403] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.003s
I20260709 12:55:42.814484 13243 raft_consensus.cc:359] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.815207 13243 raft_consensus.cc:385] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.815514 13243 raft_consensus.cc:740] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.816339 13243 consensus_queue.cc:260] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.816475 13242 raft_consensus.cc:359] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.817139 13242 raft_consensus.cc:385] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.817456 13242 raft_consensus.cc:740] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.818287 13242 consensus_queue.cc:260] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.820370 13242 ts_tablet_manager.cc:1434] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:55:42.821218 13242 tablet_bootstrap.cc:492] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:42.827672 13242 tablet_bootstrap.cc:654] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.829469 13244 tablet_bootstrap.cc:492] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:42.829741 13243 ts_tablet_manager.cc:1434] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 12:55:42.829983 13244 ts_tablet_manager.cc:1403] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.007s
I20260709 12:55:42.832734 13243 tablet_bootstrap.cc:492] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:42.833356 13244 raft_consensus.cc:359] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.834185 13244 raft_consensus.cc:385] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.834519 13244 raft_consensus.cc:740] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.835777 13244 consensus_queue.cc:260] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.839668 13243 tablet_bootstrap.cc:654] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.841197 13244 ts_tablet_manager.cc:1434] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 12:55:42.842270 13244 tablet_bootstrap.cc:492] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:42.854470 13244 tablet_bootstrap.cc:654] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.857843 13243 tablet_bootstrap.cc:492] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:42.858355 13243 ts_tablet_manager.cc:1403] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.014s
I20260709 12:55:42.858675 13247 raft_consensus.cc:493] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:42.859170 13247 raft_consensus.cc:515] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.860011 13242 tablet_bootstrap.cc:492] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:42.860625 13242 ts_tablet_manager.cc:1403] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.010s
I20260709 12:55:42.861132 13244 tablet_bootstrap.cc:492] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:42.861629 13244 ts_tablet_manager.cc:1403] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 12:55:42.861822 13243 raft_consensus.cc:359] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.862697 13243 raft_consensus.cc:385] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.863049 13243 raft_consensus.cc:740] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.864108 13243 consensus_queue.cc:260] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.864676 13242 raft_consensus.cc:359] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.866528 13242 raft_consensus.cc:385] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.866935 13242 raft_consensus.cc:740] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.867849 13242 consensus_queue.cc:260] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.869729 13247 leader_election.cc:290] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:42.870266 13247 raft_consensus.cc:493] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:42.870419 13242 ts_tablet_manager.cc:1434] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:55:42.870736 13247 raft_consensus.cc:515] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.871701 13242 tablet_bootstrap.cc:492] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:42.875451 13244 raft_consensus.cc:359] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.876238 13244 raft_consensus.cc:385] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.876539 13244 raft_consensus.cc:740] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.879009 13244 consensus_queue.cc:260] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.880376 13247 leader_election.cc:290] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:42.881212 13242 tablet_bootstrap.cc:654] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.886106 13251 raft_consensus.cc:493] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:42.886647 13251 raft_consensus.cc:515] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.888082 13243 ts_tablet_manager.cc:1434] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 12:55:42.881263 13244 ts_tablet_manager.cc:1434] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:55:42.889430 13243 tablet_bootstrap.cc:492] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:42.899638 13244 tablet_bootstrap.cc:492] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:42.900641 13251 leader_election.cc:290] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
W20260709 12:55:42.907758 12999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:42.909323 13244 tablet_bootstrap.cc:654] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.911602 13250 raft_consensus.cc:493] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:42.912189 13250 raft_consensus.cc:515] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.914482 13242 tablet_bootstrap.cc:492] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:42.915094 13242 ts_tablet_manager.cc:1403] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.004s
I20260709 12:55:42.928540 13242 raft_consensus.cc:359] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.929384 13242 raft_consensus.cc:385] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:42.930086 13242 raft_consensus.cc:740] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:42.931272 13250 leader_election.cc:290] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:42.934734 13251 raft_consensus.cc:493] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:42.935420 13251 raft_consensus.cc:515] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.938197 13251 leader_election.cc:290] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:42.952612 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:42.953631 13128 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 0.
I20260709 12:55:42.955061 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:42.955458 13017 leader_election.cc:304] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:42.955690 13128 raft_consensus.cc:2468] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 0.
I20260709 12:55:42.956324 13251 raft_consensus.cc:2804] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:42.956708 13251 raft_consensus.cc:493] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:42.957064 13251 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.958451 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:42.958959 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:42.959853 13052 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 0.
I20260709 12:55:42.960470 13017 leader_election.cc:304] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:42.961885 13168 leader_election.cc:304] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:42.962230 13250 raft_consensus.cc:493] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:42.962787 13250 raft_consensus.cc:515] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.963124 13265 raft_consensus.cc:2804] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:42.963531 13265 raft_consensus.cc:493] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:42.963850 13265 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.964252 13243 tablet_bootstrap.cc:654] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:42.964937 13250 leader_election.cc:290] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:42.965552 13251 raft_consensus.cc:515] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.966760 13053 raft_consensus.cc:2393] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9825019a5484ebe9114919d65c77120 in current term 1: Already voted for candidate 0da16b4100324a709e8662bc7053426c in this term.
I20260709 12:55:42.967264 13247 raft_consensus.cc:493] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:42.967762 13247 raft_consensus.cc:515] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.968931 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:42.969566 13128 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 0.
I20260709 12:55:42.970003 13247 leader_election.cc:290] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:42.970904 13127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:42.967581 13242 consensus_queue.cc:260] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.971594 13127 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 0.
I20260709 12:55:42.971391 13251 leader_election.cc:290] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:42.972661 13276 raft_consensus.cc:2804] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:42.973227 13276 raft_consensus.cc:493] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:42.973361 13127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:42.973582 13276 raft_consensus.cc:3060] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.973930 13127 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.976732 13166 leader_election.cc:304] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:42.977748 13247 raft_consensus.cc:2804] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:42.978134 13247 raft_consensus.cc:493] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:42.978477 13247 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.981254 13127 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 1.
I20260709 12:55:42.981282 13276 raft_consensus.cc:515] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.982836 13017 leader_election.cc:304] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:42.983652 13251 raft_consensus.cc:2804] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:42.984885 13265 raft_consensus.cc:515] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.986392 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:42.987287 13053 raft_consensus.cc:2393] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9825019a5484ebe9114919d65c77120 in current term 1: Already voted for candidate 0da16b4100324a709e8662bc7053426c in this term.
I20260709 12:55:42.987136 13127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:42.987856 13127 raft_consensus.cc:2468] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 0.
I20260709 12:55:42.988600 13247 raft_consensus.cc:515] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:42.989219 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:42.989533 13166 leader_election.cc:304] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 0da16b4100324a709e8662bc7053426c
I20260709 12:55:42.989758 13051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:42.989866 13052 raft_consensus.cc:2468] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 0.
I20260709 12:55:42.990417 13051 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.990770 13247 leader_election.cc:290] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:42.991003 13127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:42.991329 13247 raft_consensus.cc:2804] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:42.991658 13127 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.992347 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:42.995206 13244 tablet_bootstrap.cc:492] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:42.995713 13244 ts_tablet_manager.cc:1403] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.096s	user 0.036s	sys 0.023s
I20260709 12:55:42.996454 13094 leader_election.cc:304] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:42.998420 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.000042 13244 raft_consensus.cc:359] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.000787 13244 raft_consensus.cc:385] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.001082 13244 raft_consensus.cc:740] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.001641 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:43.002202 13128 raft_consensus.cc:3060] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.999236 13265 leader_election.cc:290] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:43.003358 13051 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 1.
I20260709 12:55:43.003947 13250 raft_consensus.cc:2804] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:43.004323 13250 raft_consensus.cc:493] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.004645 13168 leader_election.cc:304] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:43.005085 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:43.005592 13265 raft_consensus.cc:2804] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.005988 13126 raft_consensus.cc:2393] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f9825019a5484ebe9114919d65c77120 in current term 1: Already voted for candidate 0da16b4100324a709e8662bc7053426c in this term.
I20260709 12:55:43.006112 13265 raft_consensus.cc:697] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Becoming Leader. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:43.004645 13250 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:42.991699 13247 raft_consensus.cc:493] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.007470 13247 raft_consensus.cc:3060] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.007081 13265 consensus_queue.cc:237] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.012547 13127 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 1.
I20260709 12:55:43.013609 13128 raft_consensus.cc:2468] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 1.
I20260709 12:55:43.001813 13244 consensus_queue.cc:260] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.014887 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.015988 13202 raft_consensus.cc:2393] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0da16b4100324a709e8662bc7053426c in current term 1: Already voted for candidate f9825019a5484ebe9114919d65c77120 in this term.
I20260709 12:55:43.014918 13017 leader_election.cc:304] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.017578 13244 ts_tablet_manager.cc:1434] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 12:55:43.019299 13201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.024622 13201 raft_consensus.cc:2393] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0da16b4100324a709e8662bc7053426c in current term 1: Already voted for candidate f9825019a5484ebe9114919d65c77120 in this term.
I20260709 12:55:43.027551 13250 raft_consensus.cc:515] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.028005 13251 raft_consensus.cc:697] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:43.029115 13251 consensus_queue.cc:237] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.030059 13250 leader_election.cc:290] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.030006 12951 catalog_manager.cc:5697] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9825019a5484ebe9114919d65c77120 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "f9825019a5484ebe9114919d65c77120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:43.031831 13051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.032436 13051 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.020465 13276 leader_election.cc:290] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.021023 13243 tablet_bootstrap.cc:492] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:43.034582 13201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.035089 13243 ts_tablet_manager.cc:1403] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.146s	user 0.012s	sys 0.002s
I20260709 12:55:43.035717 13244 tablet_bootstrap.cc:492] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:43.036470 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.037285 13198 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.038650 13243 raft_consensus.cc:359] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.039386 13243 raft_consensus.cc:385] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.039677 13243 raft_consensus.cc:740] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.040074 13279 raft_consensus.cc:2804] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.040333 13199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.040658 13243 consensus_queue.cc:260] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.021436 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.021566 13247 raft_consensus.cc:515] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.022979 13200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.043347 13093 leader_election.cc:304] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec; no voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120
I20260709 12:55:43.040688 13279 raft_consensus.cc:697] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:43.054611 13244 tablet_bootstrap.cc:654] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.054667 13250 raft_consensus.cc:2749] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:43.060734 13198 raft_consensus.cc:2468] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 1.
I20260709 12:55:43.061899 13243 ts_tablet_manager.cc:1434] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.004s
I20260709 12:55:43.062237 13093 leader_election.cc:304] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:43.062846 13279 consensus_queue.cc:237] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.063410 13250 raft_consensus.cc:2804] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.065538 13202 raft_consensus.cc:2393] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0da16b4100324a709e8662bc7053426c in current term 1: Already voted for candidate f9825019a5484ebe9114919d65c77120 in this term.
I20260709 12:55:43.063705 13051 raft_consensus.cc:2468] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 1.
I20260709 12:55:43.069530 13243 tablet_bootstrap.cc:492] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:43.071089 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.074815 13168 leader_election.cc:304] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9825019a5484ebe9114919d65c77120; no voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec
I20260709 12:55:43.076122 13265 raft_consensus.cc:2749] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:55:43.079434 13242 ts_tablet_manager.cc:1434] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.164s	user 0.017s	sys 0.005s
I20260709 12:55:43.081550 13242 tablet_bootstrap.cc:492] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:43.081614 13247 leader_election.cc:290] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
W20260709 12:55:43.066635 13017 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:36333, user_credentials={real_user=slave}} blocked reactor thread for 51821.1us
I20260709 12:55:43.083678 12950 catalog_manager.cc:5697] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 1 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.091538 13250 raft_consensus.cc:697] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:43.091636 13242 tablet_bootstrap.cc:654] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.093405 13250 consensus_queue.cc:237] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.103185 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c167a5eddee447daab59df9e2233a24" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:43.103535 13285 raft_consensus.cc:493] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:43.104176 13128 raft_consensus.cc:2393] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f9825019a5484ebe9114919d65c77120 in current term 1: Already voted for candidate 0da16b4100324a709e8662bc7053426c in this term.
I20260709 12:55:43.104198 13285 raft_consensus.cc:515] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.115954 13166 leader_election.cc:304] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9825019a5484ebe9114919d65c77120; no voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec
I20260709 12:55:43.117081 13247 raft_consensus.cc:2749] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:55:43.118434 13285 leader_election.cc:290] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.119076 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:43.121117 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.121690 13243 tablet_bootstrap.cc:654] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.121757 13202 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 0.
I20260709 12:55:43.124125 13093 leader_election.cc:304] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:43.125433 13244 tablet_bootstrap.cc:492] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
W20260709 12:55:43.125718 13094 leader_election.cc:343] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:43.127524 13287 raft_consensus.cc:2804] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:43.128101 13287 raft_consensus.cc:493] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.128571 13287 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.136807 13243 tablet_bootstrap.cc:492] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:43.137332 13243 ts_tablet_manager.cc:1403] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.002s
I20260709 12:55:43.137888 13287 raft_consensus.cc:515] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.137647 12950 catalog_manager.cc:5697] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 1 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.141228 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.142937 13242 tablet_bootstrap.cc:492] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
W20260709 12:55:43.142810 13094 leader_election.cc:343] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:43.143610 13242 ts_tablet_manager.cc:1403] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.009s
I20260709 12:55:43.145134 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.145728 13202 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.126283 13244 ts_tablet_manager.cc:1403] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.091s	user 0.019s	sys 0.001s
I20260709 12:55:43.146953 13242 raft_consensus.cc:359] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.147681 13242 raft_consensus.cc:385] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.148003 13242 raft_consensus.cc:740] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.148774 13242 consensus_queue.cc:260] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.151093 13242 ts_tablet_manager.cc:1434] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:55:43.151487 13285 raft_consensus.cc:493] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:43.151978 13285 raft_consensus.cc:515] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.152400 13242 tablet_bootstrap.cc:492] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:43.152639 13243 raft_consensus.cc:359] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.153867 13243 raft_consensus.cc:385] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.154239 13243 raft_consensus.cc:740] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.155095 13243 consensus_queue.cc:260] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.156483 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.157142 13198 raft_consensus.cc:2468] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 0.
I20260709 12:55:43.158228 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:43.158836 13053 raft_consensus.cc:2468] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 0.
I20260709 12:55:43.159159 13242 tablet_bootstrap.cc:654] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.162240 13094 leader_election.cc:304] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.163149 13244 raft_consensus.cc:359] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.154093 13285 leader_election.cc:290] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.163878 13244 raft_consensus.cc:385] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.164244 13244 raft_consensus.cc:740] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.164388 13285 raft_consensus.cc:2804] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:43.164918 13285 raft_consensus.cc:493] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.165364 13285 raft_consensus.cc:3060] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.164981 13244 consensus_queue.cc:260] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.167402 13244 ts_tablet_manager.cc:1434] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 12:55:43.168475 13244 tablet_bootstrap.cc:492] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:43.170289 12949 catalog_manager.cc:5697] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 1 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.173221 13243 ts_tablet_manager.cc:1434] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.004s
I20260709 12:55:43.173866 13202 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 1.
I20260709 12:55:43.174162 13243 tablet_bootstrap.cc:492] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:43.175098 13093 leader_election.cc:304] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 0da16b4100324a709e8662bc7053426c
I20260709 12:55:43.176164 13295 raft_consensus.cc:2804] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.176678 13295 raft_consensus.cc:697] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:43.177860 13295 consensus_queue.cc:237] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.180343 13243 tablet_bootstrap.cc:654] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.180524 13287 leader_election.cc:290] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.184551 13285 raft_consensus.cc:515] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.186722 13285 leader_election.cc:290] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.187541 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.188167 13053 raft_consensus.cc:3060] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.189949 13244 tablet_bootstrap.cc:654] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.188163 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.196067 13279 raft_consensus.cc:493] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:43.195955 12949 catalog_manager.cc:5697] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 1 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.196807 13279 raft_consensus.cc:515] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.199843 13053 raft_consensus.cc:2468] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 1.
I20260709 12:55:43.201016 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:43.201650 13128 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 0.
I20260709 12:55:43.202324 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.202705 13242 tablet_bootstrap.cc:492] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:43.202754 13198 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 0.
I20260709 12:55:43.203267 13242 ts_tablet_manager.cc:1403] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.006s
I20260709 12:55:43.204198 13279 leader_election.cc:290] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.204077 13018 leader_election.cc:304] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:43.205421 13094 leader_election.cc:304] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.206256 13295 raft_consensus.cc:2804] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.206733 13295 raft_consensus.cc:697] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:43.207803 13279 raft_consensus.cc:2804] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:43.207605 13242 raft_consensus.cc:359] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.208271 13279 raft_consensus.cc:493] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.208591 13242 raft_consensus.cc:385] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.207743 13295 consensus_queue.cc:237] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.208880 13279 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.208954 13242 raft_consensus.cc:740] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.204860 13202 raft_consensus.cc:3060] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.213652 13244 tablet_bootstrap.cc:492] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:43.214262 13244 ts_tablet_manager.cc:1403] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.046s	user 0.006s	sys 0.005s
I20260709 12:55:43.217029 13244 raft_consensus.cc:359] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.217754 13244 raft_consensus.cc:385] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.218061 13244 raft_consensus.cc:740] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.219053 13244 consensus_queue.cc:260] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.222581 13202 raft_consensus.cc:2468] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 1.
I20260709 12:55:43.223075 13279 raft_consensus.cc:515] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.219772 13242 consensus_queue.cc:260] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.225566 13242 ts_tablet_manager.cc:1434] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.006s
I20260709 12:55:43.225960 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:43.226778 13244 ts_tablet_manager.cc:1434] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:55:43.227082 13128 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.227716 13244 tablet_bootstrap.cc:492] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:43.229360 12950 catalog_manager.cc:5697] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 1 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.233201 13243 tablet_bootstrap.cc:492] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:43.233738 13244 tablet_bootstrap.cc:654] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.233745 13243 ts_tablet_manager.cc:1403] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.009s
I20260709 12:55:43.235129 13128 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 1.
I20260709 12:55:43.238166 13242 tablet_bootstrap.cc:492] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:43.236835 13243 raft_consensus.cc:359] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.238581 13243 raft_consensus.cc:385] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.240023 13243 raft_consensus.cc:740] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.240325 13017 leader_election.cc:304] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.240870 13243 consensus_queue.cc:260] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.241511 13251 raft_consensus.cc:2804] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.243100 13251 raft_consensus.cc:697] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:43.244026 13243 ts_tablet_manager.cc:1434] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:55:43.245317 13242 tablet_bootstrap.cc:654] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.244549 13251 consensus_queue.cc:237] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.252274 13279 leader_election.cc:290] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.252527 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.253108 13202 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.259133 13243 tablet_bootstrap.cc:492] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec: Bootstrap starting.
I20260709 12:55:43.260182 13244 tablet_bootstrap.cc:492] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:43.260617 13244 ts_tablet_manager.cc:1403] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.011s
I20260709 12:55:43.263183 13244 raft_consensus.cc:359] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.263818 13244 raft_consensus.cc:385] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.264114 13244 raft_consensus.cc:740] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.264659 13242 tablet_bootstrap.cc:492] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:43.264724 13202 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 1.
I20260709 12:55:43.265183 13242 ts_tablet_manager.cc:1403] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.000s
I20260709 12:55:43.264873 13244 consensus_queue.cc:260] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.267197 13244 ts_tablet_manager.cc:1434] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:55:43.268150 13244 tablet_bootstrap.cc:492] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120: Bootstrap starting.
I20260709 12:55:43.268436 12950 catalog_manager.cc:5697] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 1 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.270653 13242 raft_consensus.cc:359] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.272092 13242 raft_consensus.cc:385] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.272300 13243 tablet_bootstrap.cc:654] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.272406 13242 raft_consensus.cc:740] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.273393 13242 consensus_queue.cc:260] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.275265 13244 tablet_bootstrap.cc:654] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.275637 13242 ts_tablet_manager.cc:1434] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:55:43.276816 13242 tablet_bootstrap.cc:492] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:43.283773 13243 tablet_bootstrap.cc:492] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec: No bootstrap required, opened a new log
I20260709 12:55:43.284345 13243 ts_tablet_manager.cc:1403] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260709 12:55:43.287014 13242 tablet_bootstrap.cc:654] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.287081 13243 raft_consensus.cc:359] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.288093 13243 raft_consensus.cc:385] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.288451 13243 raft_consensus.cc:740] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.289347 13243 consensus_queue.cc:260] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.292303 13243 ts_tablet_manager.cc:1434] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:55:43.300940 13242 tablet_bootstrap.cc:492] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:43.301404 13242 ts_tablet_manager.cc:1403] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.010s
I20260709 12:55:43.301842 13244 tablet_bootstrap.cc:492] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120: No bootstrap required, opened a new log
I20260709 12:55:43.302284 13244 ts_tablet_manager.cc:1403] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.013s
I20260709 12:55:43.303371 13242 raft_consensus.cc:359] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.303892 13242 raft_consensus.cc:385] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.304193 13242 raft_consensus.cc:740] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.304792 13242 consensus_queue.cc:260] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.304751 13244 raft_consensus.cc:359] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.305519 13244 raft_consensus.cc:385] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.305809 13244 raft_consensus.cc:740] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.306734 13242 ts_tablet_manager.cc:1434] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 12:55:43.306414 13244 consensus_queue.cc:260] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.307755 13242 tablet_bootstrap.cc:492] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c: Bootstrap starting.
I20260709 12:55:43.308588 13244 ts_tablet_manager.cc:1434] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:55:43.313247 13242 tablet_bootstrap.cc:654] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:43.318210 13242 tablet_bootstrap.cc:492] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c: No bootstrap required, opened a new log
I20260709 12:55:43.318730 13242 ts_tablet_manager.cc:1403] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:55:43.320572 13242 raft_consensus.cc:359] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.321048 13242 raft_consensus.cc:385] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:43.321277 13242 raft_consensus.cc:740] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Initialized, Role: FOLLOWER
I20260709 12:55:43.321759 13242 consensus_queue.cc:260] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.323516 13242 ts_tablet_manager.cc:1434] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.000s
I20260709 12:55:43.340886 13251 raft_consensus.cc:493] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:43.341332 13251 raft_consensus.cc:515] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.342934 13251 leader_election.cc:290] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.343725 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:43.344002 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.344672 13202 raft_consensus.cc:2393] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0da16b4100324a709e8662bc7053426c in current term 1: Already voted for candidate 0e354d68b8c64e379bc71b28c3af85ec in this term.
I20260709 12:55:43.345507 13018 leader_election.cc:304] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0da16b4100324a709e8662bc7053426c; no voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120
I20260709 12:55:43.346196 13251 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.350715 13251 raft_consensus.cc:2749] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:43.448357 13251 raft_consensus.cc:493] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:43.448871 13251 raft_consensus.cc:515] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.450634 13251 leader_election.cc:290] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.451473 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:43.451735 13247 raft_consensus.cc:493] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:43.451756 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.452123 13128 raft_consensus.cc:2468] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 0.
I20260709 12:55:43.452518 13202 raft_consensus.cc:2468] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 0.
I20260709 12:55:43.452250 13247 raft_consensus.cc:515] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.453145 13017 leader_election.cc:304] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.453923 13251 raft_consensus.cc:2804] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:43.454254 13251 raft_consensus.cc:493] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.454537 13251 raft_consensus.cc:3060] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.454927 13247 leader_election.cc:290] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:43.455545 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:43.455893 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec" is_pre_election: true
I20260709 12:55:43.456192 13053 raft_consensus.cc:2468] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 0.
I20260709 12:55:43.456466 13128 raft_consensus.cc:2468] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 0.
I20260709 12:55:43.457465 13168 leader_election.cc:304] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:43.458273 13247 raft_consensus.cc:2804] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:43.458617 13247 raft_consensus.cc:493] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.458930 13247 raft_consensus.cc:3060] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.460441 13251 raft_consensus.cc:515] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.461992 13251 leader_election.cc:290] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.462911 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:43.463097 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.463398 13128 raft_consensus.cc:3060] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.463712 13202 raft_consensus.cc:3060] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.464140 13247 raft_consensus.cc:515] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.466344 13247 leader_election.cc:290] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:43.467034 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.467607 13053 raft_consensus.cc:3060] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.469319 13128 raft_consensus.cc:2468] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 1.
I20260709 12:55:43.470347 13295 raft_consensus.cc:493] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:43.470806 13295 raft_consensus.cc:515] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.472723 13295 leader_election.cc:290] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.473203 13202 raft_consensus.cc:2468] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 1.
I20260709 12:55:43.474066 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c" is_pre_election: true
I20260709 12:55:43.474594 13017 leader_election.cc:304] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.474761 13052 raft_consensus.cc:2468] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 0.
I20260709 12:55:43.474978 13128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:43.474195 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120" is_pre_election: true
I20260709 12:55:43.475541 13128 raft_consensus.cc:3060] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.475668 13198 raft_consensus.cc:2468] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 0.
I20260709 12:55:43.476001 13279 raft_consensus.cc:2804] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.476029 13251 consensus_queue.cc:1048] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:43.476570 13279 raft_consensus.cc:697] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:43.477641 13094 leader_election.cc:304] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.477397 13279 consensus_queue.cc:237] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.478492 13295 raft_consensus.cc:2804] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:43.478863 13295 raft_consensus.cc:493] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:43.479212 13295 raft_consensus.cc:3060] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.482622 13128 raft_consensus.cc:2468] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 1.
I20260709 12:55:43.484850 13166 leader_election.cc:304] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:43.474061 13053 raft_consensus.cc:2468] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 1.
I20260709 12:55:43.486446 13295 raft_consensus.cc:515] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.489439 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.490031 13053 raft_consensus.cc:3060] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.490288 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.490834 13198 raft_consensus.cc:3060] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:43.487277 13247 raft_consensus.cc:2804] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.495227 13247 raft_consensus.cc:697] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Becoming Leader. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:43.496273 12948 catalog_manager.cc:5697] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 1 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.496387 13247 consensus_queue.cc:237] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.503058 13053 raft_consensus.cc:2468] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 1.
I20260709 12:55:43.504369 13094 leader_election.cc:304] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.504912 13279 consensus_queue.cc:1048] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.506651 13285 raft_consensus.cc:2804] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:43.507303 13285 raft_consensus.cc:697] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:43.508160 13285 consensus_queue.cc:237] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.508911 13198 raft_consensus.cc:2468] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 1.
I20260709 12:55:43.511780 13295 leader_election.cc:290] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 1 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.523891 12948 catalog_manager.cc:5697] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9825019a5484ebe9114919d65c77120 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "f9825019a5484ebe9114919d65c77120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:43.527669 12951 catalog_manager.cc:5697] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 1 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:43.560973 13285 consensus_queue.cc:1048] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.572603 13295 consensus_queue.cc:1048] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.591149 13278 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.603402 13278 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.628453 13247 consensus_queue.cc:1048] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.632151 13247 consensus_queue.cc:1048] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:43.708310 13295 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.718796 13250 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.735330 13278 consensus_queue.cc:1048] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.745711 13282 consensus_queue.cc:1048] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:43.776646 13250 consensus_queue.cc:1048] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.787421 13285 consensus_queue.cc:1048] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.814883 13202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:43052
I20260709 12:55:43.815419 13202 raft_consensus.cc:606] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:43.834290 13127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:41500
I20260709 12:55:43.834828 13127 raft_consensus.cc:606] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:43.839855 13127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:41500
I20260709 12:55:43.840423 13127 raft_consensus.cc:606] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:43.855997 13053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:52018
I20260709 12:55:43.856570 13053 raft_consensus.cc:606] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Received request to transfer leadership to TS 0e354d68b8c64e379bc71b28c3af85ec
I20260709 12:55:43.864305 13202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
mode: GRACEFUL
new_leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:43052
I20260709 12:55:43.864845 13202 raft_consensus.cc:606] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Received request to transfer leadership to TS 0e354d68b8c64e379bc71b28c3af85ec
I20260709 12:55:43.869874 13053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:52018
I20260709 12:55:43.870486 13053 raft_consensus.cc:606] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Received request to transfer leadership to TS f9825019a5484ebe9114919d65c77120
I20260709 12:55:43.870342 13282 consensus_queue.cc:1048] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.876933 13279 raft_consensus.cc:993] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c: : Instructing follower f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:43.877458 13251 raft_consensus.cc:1081] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Signalling peer f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:43.881570 13198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:55:43.882166 13198 raft_consensus.cc:493] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:43.882542 13198 raft_consensus.cc:3060] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:43.886569 13251 consensus_queue.cc:1048] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.889403 13265 consensus_queue.cc:1048] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:43.889431 13198 raft_consensus.cc:515] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.890782 13202 raft_consensus.cc:1240] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Rejecting Update request from peer 0da16b4100324a709e8662bc7053426c for earlier term 1. Current term is 2. Ops: []
I20260709 12:55:43.892294 13198 leader_election.cc:290] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 2 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:43.892664 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.893256 13053 raft_consensus.cc:3055] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:43.893572 13053 raft_consensus.cc:740] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:43.894438 13053 consensus_queue.cc:260] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.895430 13053 raft_consensus.cc:3060] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:43.896138 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d4c36c3db804264b072c71ea9da961d" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:43.896728 13126 raft_consensus.cc:3060] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:43.902707 13053 raft_consensus.cc:2468] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 2.
I20260709 12:55:43.903851 13168 leader_election.cc:304] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:43.904742 13265 raft_consensus.cc:2804] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:43.905478 13247 raft_consensus.cc:993] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120: : Instructing follower 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:43.905838 13126 raft_consensus.cc:2468] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 2.
I20260709 12:55:43.906250 13322 raft_consensus.cc:1081] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Signalling peer 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:43.907195 13265 raft_consensus.cc:697] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 2 LEADER]: Becoming Leader. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:43.909608 13265 consensus_queue.cc:237] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.911501 13247 consensus_queue.cc:1048] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:43.916229 13127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:41492
I20260709 12:55:43.916775 13127 raft_consensus.cc:493] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:43.917124 13127 raft_consensus.cc:3060] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:43.928671 13127 raft_consensus.cc:515] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.931145 12948 catalog_manager.cc:5697] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 reported cstate change: term changed from 1 to 2, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to f9825019a5484ebe9114919d65c77120 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "f9825019a5484ebe9114919d65c77120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:43.934150 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:43.934759 13053 raft_consensus.cc:3060] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:43.936992 13127 leader_election.cc:290] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 2 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:43.937597 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e447b5bd4309ac709b3732072473" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:43.938205 13202 raft_consensus.cc:3055] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:43.938510 13202 raft_consensus.cc:740] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:43.939281 13202 consensus_queue.cc:260] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.940354 13202 raft_consensus.cc:3060] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:43.944016 13053 raft_consensus.cc:2468] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 2.
I20260709 12:55:43.945086 13094 leader_election.cc:304] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:43.945869 13285 raft_consensus.cc:2804] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:43.946288 13285 raft_consensus.cc:697] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:43.946676 13202 raft_consensus.cc:2468] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 2.
I20260709 12:55:43.947036 13285 consensus_queue.cc:237] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:43.954494 12949 catalog_manager.cc:5697] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 1 to 2, leader changed from f9825019a5484ebe9114919d65c77120 (127.12.30.67) to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 2 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:44.091078 13285 consensus_queue.cc:1048] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:44.100636 13285 consensus_queue.cc:1048] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:44.127170 13251 raft_consensus.cc:993] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c: : Instructing follower 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:44.127588 13282 raft_consensus.cc:1081] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Signalling peer 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:44.128891 13127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:41496
I20260709 12:55:44.129346 13127 raft_consensus.cc:493] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:44.129650 13127 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.133895 13127 raft_consensus.cc:515] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.135432 13127 leader_election.cc:290] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 2 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:44.136260 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:44.136448 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:44.136857 13053 raft_consensus.cc:3055] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:44.136909 13202 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.137301 13053 raft_consensus.cc:740] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:44.138180 13053 consensus_queue.cc:260] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.139366 13053 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.144145 13202 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 2.
I20260709 12:55:44.145299 13093 leader_election.cc:304] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:44.145933 13053 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 2.
I20260709 12:55:44.146075 13285 raft_consensus.cc:2804] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:44.146672 13285 raft_consensus.cc:697] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:44.147477 13285 consensus_queue.cc:237] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.153270 12949 catalog_manager.cc:5697] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 1 to 2, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 2 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:44.202484 13250 raft_consensus.cc:993] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:44.202952 13285 raft_consensus.cc:1081] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:44.204319 13336 raft_consensus.cc:993] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:44.204579 13053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:55:44.204947 13336 raft_consensus.cc:1081] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:44.205379 13053 raft_consensus.cc:493] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:44.205775 13053 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.206627 13052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52002
I20260709 12:55:44.207159 13052 raft_consensus.cc:493] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:44.207459 13052 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.211726 13053 raft_consensus.cc:515] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.213426 13053 leader_election.cc:290] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:44.213418 13052 raft_consensus.cc:515] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.214582 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:44.214687 13127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:44.215178 13202 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.215389 13127 raft_consensus.cc:3055] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:44.216243 13127 raft_consensus.cc:740] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:44.216846 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:44.217183 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:44.216173 13052 leader_election.cc:290] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:44.217145 13127 consensus_queue.cc:260] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.217402 13126 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.218845 13127 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.217907 13198 raft_consensus.cc:3055] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:44.220170 13198 raft_consensus.cc:740] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:44.220964 13198 consensus_queue.cc:260] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.222002 13198 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.223634 13126 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:44.224175 13128 raft_consensus.cc:1240] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Rejecting Update request from peer f9825019a5484ebe9114919d65c77120 for earlier term 1. Current term is 2. Ops: []
I20260709 12:55:44.225925 13127 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:44.226224 13017 leader_election.cc:304] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:44.227142 13017 leader_election.cc:304] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:44.227793 13251 raft_consensus.cc:2804] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:44.227852 13282 raft_consensus.cc:2804] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:44.228412 13251 raft_consensus.cc:697] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:44.228534 13198 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:44.228446 13282 raft_consensus.cc:697] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:44.229387 13251 consensus_queue.cc:237] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.230432 13282 consensus_queue.cc:237] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.233026 13202 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:44.238534 12949 catalog_manager.cc:5697] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 1 to 2, leader changed from f9825019a5484ebe9114919d65c77120 (127.12.30.67) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 2 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:44.239826 12949 catalog_manager.cc:5697] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 1 to 2, leader changed from 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 2 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:44.298115 13052 raft_consensus.cc:1275] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.299530 13336 consensus_queue.cc:1048] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.309041 13126 raft_consensus.cc:1275] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.312783 13247 consensus_queue.cc:1048] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.317410 13250 raft_consensus.cc:993] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:44.317931 13295 raft_consensus.cc:1081] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:44.319466 13052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:55:44.320094 13052 raft_consensus.cc:493] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:44.320518 13052 raft_consensus.cc:3060] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.326786 13052 raft_consensus.cc:515] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.328595 13052 leader_election.cc:290] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:44.329329 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:44.329818 13126 raft_consensus.cc:3055] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:44.329825 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f37dd0690244d93b6acbadd20a93782" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:44.330170 13126 raft_consensus.cc:740] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:44.330425 13202 raft_consensus.cc:3060] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.331029 13126 consensus_queue.cc:260] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.332180 13126 raft_consensus.cc:3060] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:44.335397 13202 raft_consensus.cc:2468] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:44.336390 13018 leader_election.cc:304] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:44.337114 13282 raft_consensus.cc:2804] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:44.337121 13126 raft_consensus.cc:2468] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:44.337711 13282 raft_consensus.cc:697] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:44.338907 13282 consensus_queue.cc:237] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:44.344498 12951 catalog_manager.cc:5697] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 1 to 2, leader changed from 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 2 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:44.487237 13052 raft_consensus.cc:1275] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.488471 13295 consensus_queue.cc:1048] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.496730 13202 raft_consensus.cc:1275] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.498394 13250 consensus_queue.cc:1048] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.604256 13202 raft_consensus.cc:1275] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.605603 13250 consensus_queue.cc:1048] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.615646 13052 raft_consensus.cc:1275] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.617324 13250 consensus_queue.cc:1048] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.696588 13202 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.697985 13282 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.705197 13126 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.706655 13251 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.786299 13126 raft_consensus.cc:1275] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.787549 13251 consensus_queue.cc:1048] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.795740 13202 raft_consensus.cc:1275] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.797293 13251 consensus_queue.cc:1048] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.895582 13202 raft_consensus.cc:1275] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.897009 13251 consensus_queue.cc:1048] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:44.906080 13126 raft_consensus.cc:1275] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:44.908068 13279 consensus_queue.cc:1048] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:46.904071 12409 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 12:55:46.904805 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:46.927199 13053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:52028
I20260709 12:55:46.927814 13053 raft_consensus.cc:606] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Received request to transfer leadership to TS f9825019a5484ebe9114919d65c77120
I20260709 12:55:46.930371 13053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:52028
I20260709 12:55:46.930852 13053 raft_consensus.cc:606] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Received request to transfer leadership to TS f9825019a5484ebe9114919d65c77120
I20260709 12:55:46.931998 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:55:46.932463 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:55:47.038542 13282 raft_consensus.cc:993] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: : Instructing follower f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:47.039162 13282 raft_consensus.cc:1081] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Signalling peer f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:47.040864 13198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:55:47.041426 13198 raft_consensus.cc:493] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:47.041751 13198 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:47.047485 13198 raft_consensus.cc:515] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:47.049539 13198 leader_election.cc:290] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 2 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:47.050657 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:47.050657 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:47.051537 13126 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:47.051627 13053 raft_consensus.cc:3055] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:47.052011 13053 raft_consensus.cc:740] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:47.052873 13053 consensus_queue.cc:260] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:47.054171 13053 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:47.059283 13126 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 2.
I20260709 12:55:47.060784 13166 leader_election.cc:304] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:47.061641 13053 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 2.
I20260709 12:55:47.061681 13247 raft_consensus.cc:2804] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:47.062981 13247 raft_consensus.cc:697] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 LEADER]: Becoming Leader. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:47.063807 13247 consensus_queue.cc:237] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:47.072201 12948 catalog_manager.cc:5697] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 reported cstate change: term changed from 1 to 2, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to f9825019a5484ebe9114919d65c77120 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "f9825019a5484ebe9114919d65c77120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:47.227198 13349 raft_consensus.cc:993] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: : Instructing follower f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:47.228590 13282 raft_consensus.cc:1081] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Signalling peer f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:47.230253 13198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:55:47.230877 13198 raft_consensus.cc:493] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:47.231228 13198 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:47.236479 13198 raft_consensus.cc:515] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:47.238638 13198 leader_election.cc:290] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 3 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:47.239512 13053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:47.240159 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:47.241010 13126 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:47.240195 13053 raft_consensus.cc:3055] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:47.241711 13053 raft_consensus.cc:740] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:47.242594 13053 consensus_queue.cc:260] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:47.244191 13053 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:47.282035 13126 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 3.
I20260709 12:55:47.286085 13166 leader_election.cc:304] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:47.289522 13247 raft_consensus.cc:2804] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:47.290208 13247 raft_consensus.cc:697] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Becoming Leader. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:47.291774 13247 consensus_queue.cc:237] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:47.294665 13053 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 3.
I20260709 12:55:47.313446 12948 catalog_manager.cc:5697] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 reported cstate change: term changed from 2 to 3, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to f9825019a5484ebe9114919d65c77120 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "f9825019a5484ebe9114919d65c77120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:47.630720 13053 raft_consensus.cc:1275] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:47.632119 13247 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:47.637686 13126 raft_consensus.cc:1275] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:47.639200 13247 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:47.695477 13126 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:47.697284 13361 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:47.708846 13053 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:47.711714 13361 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:55:47.933557 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:47.951508 13202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:43052
I20260709 12:55:47.952241 13202 raft_consensus.cc:606] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:47.957758 13202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:43052
I20260709 12:55:47.958285 13202 raft_consensus.cc:606] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:47.965948 13127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:41500
I20260709 12:55:47.966547 13127 raft_consensus.cc:606] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:47.972165 13127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:41500
I20260709 12:55:47.972960 13127 raft_consensus.cc:606] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:47.978704 13127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
mode: GRACEFUL
new_leader_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:41500
I20260709 12:55:47.979405 13127 raft_consensus.cc:606] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Received request to transfer leadership to TS 0da16b4100324a709e8662bc7053426c
I20260709 12:55:48.015115 13358 raft_consensus.cc:993] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.016348 13368 raft_consensus.cc:1081] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.017949 13053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:55:48.018810 13053 raft_consensus.cc:493] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:48.019274 13053 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:48.024013 13053 raft_consensus.cc:515] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.025691 13053 leader_election.cc:290] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:48.026837 13127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:48.026881 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:48.027520 13202 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:48.027515 13127 raft_consensus.cc:3055] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:48.027997 13127 raft_consensus.cc:740] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:48.028695 13127 consensus_queue.cc:260] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.029708 13127 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:48.032384 13202 raft_consensus.cc:2468] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:48.033756 13018 leader_election.cc:304] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:48.034749 13282 raft_consensus.cc:2804] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:48.035161 13282 raft_consensus.cc:697] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:48.035450 13127 raft_consensus.cc:2468] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:48.036134 13282 consensus_queue.cc:237] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.043340 12951 catalog_manager.cc:5697] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 1 to 2, leader changed from 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 2 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:48.131336 13350 raft_consensus.cc:993] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.131853 13361 raft_consensus.cc:1081] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.133205 13369 raft_consensus.cc:993] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.133713 13368 raft_consensus.cc:1081] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.133616 13053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52002
I20260709 12:55:48.134269 13053 raft_consensus.cc:493] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:48.134708 13053 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:48.135466 13052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:55:48.136026 13052 raft_consensus.cc:493] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:48.136334 13052 raft_consensus.cc:3060] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:48.139763 13053 raft_consensus.cc:515] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.142073 13053 leader_election.cc:290] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 3 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:48.141863 13052 raft_consensus.cc:515] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.142474 13127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:48.143077 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:48.143663 13202 raft_consensus.cc:3055] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:48.143105 13127 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:48.144033 13202 raft_consensus.cc:740] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:48.144795 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:48.144939 13202 consensus_queue.cc:260] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.145375 13126 raft_consensus.cc:3055] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:55:48.145684 13126 raft_consensus.cc:740] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:48.145869 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba09c8e124b4b6686f4d805bf766387" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:48.146100 13202 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:48.146524 13198 raft_consensus.cc:3060] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:48.146488 13126 consensus_queue.cc:260] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.147743 13126 raft_consensus.cc:3060] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:55:48.151589 13127 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 3.
I20260709 12:55:48.153035 13202 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 3.
I20260709 12:55:48.153285 13017 leader_election.cc:304] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:48.144035 13052 leader_election.cc:290] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:48.154816 13282 raft_consensus.cc:2804] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:48.154733 13198 raft_consensus.cc:2468] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:48.155467 13282 raft_consensus.cc:697] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:48.154742 13126 raft_consensus.cc:2468] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 2.
I20260709 12:55:48.156059 13018 leader_election.cc:304] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:48.156479 13282 consensus_queue.cc:237] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.157121 13356 raft_consensus.cc:2804] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:55:48.158360 13356 raft_consensus.cc:697] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:48.159250 13356 consensus_queue.cc:237] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.164047 12951 catalog_manager.cc:5697] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 2 to 3, leader changed from f9825019a5484ebe9114919d65c77120 (127.12.30.67) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 3 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:48.174988 12951 catalog_manager.cc:5697] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 1 to 2, leader changed from 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 2 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:48.183429 13247 raft_consensus.cc:993] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.184006 13361 raft_consensus.cc:1081] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.185600 13052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52002
I20260709 12:55:48.186149 13052 raft_consensus.cc:493] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:48.186467 13052 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:48.191325 13052 raft_consensus.cc:515] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.192914 13052 leader_election.cc:290] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 4 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:48.193925 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:48.194089 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:48.194499 13126 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:48.194646 13198 raft_consensus.cc:3055] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:55:48.195035 13198 raft_consensus.cc:740] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:48.195895 13198 consensus_queue.cc:260] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.197234 13198 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:48.200839 13126 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 4.
I20260709 12:55:48.202395 13017 leader_election.cc:304] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, 0e354d68b8c64e379bc71b28c3af85ec; no voters: 
I20260709 12:55:48.203111 13356 raft_consensus.cc:2804] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:55:48.203536 13356 raft_consensus.cc:697] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:48.203502 13198 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 4.
I20260709 12:55:48.204595 13356 consensus_queue.cc:237] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.210496 12951 catalog_manager.cc:5697] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 3 to 4, leader changed from f9825019a5484ebe9114919d65c77120 (127.12.30.67) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 4 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:48.255316 13369 raft_consensus.cc:993] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec: : Instructing follower 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.255812 13369 raft_consensus.cc:1081] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 LEADER]: Signalling peer 0da16b4100324a709e8662bc7053426c to start an election
I20260709 12:55:48.257366 13052 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:55:48.257891 13052 raft_consensus.cc:493] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:48.258241 13052 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:48.263192 13052 raft_consensus.cc:515] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.265173 13052 leader_election.cc:290] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 3 election: Requested vote from peers 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:48.265959 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:48.266263 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58abd247ac31444d9d71d6874fe90bb7" candidate_uuid: "0da16b4100324a709e8662bc7053426c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:48.266541 13126 raft_consensus.cc:3055] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:48.266865 13198 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:48.267004 13126 raft_consensus.cc:740] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:48.267854 13126 consensus_queue.cc:260] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.268859 13126 raft_consensus.cc:3060] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:48.272421 13198 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 3.
I20260709 12:55:48.273577 13018 leader_election.cc:304] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da16b4100324a709e8662bc7053426c, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:48.273694 13126 raft_consensus.cc:2468] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0da16b4100324a709e8662bc7053426c in term 3.
I20260709 12:55:48.274513 13356 raft_consensus.cc:2804] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:48.275007 13356 raft_consensus.cc:697] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 3 LEADER]: Becoming Leader. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:48.275844 13356 consensus_queue.cc:237] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:48.282042 12951 catalog_manager.cc:5697] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c reported cstate change: term changed from 2 to 3, leader changed from 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66) to 0da16b4100324a709e8662bc7053426c (127.12.30.65). New cstate: current_term: 3 leader_uuid: "0da16b4100324a709e8662bc7053426c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:48.525000 13198 raft_consensus.cc:1275] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:48.526487 13356 consensus_queue.cc:1048] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:48.538489 13126 raft_consensus.cc:1275] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:48.540659 13379 consensus_queue.cc:1048] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:55:48.593371 13126 raft_consensus.cc:1275] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:48.594686 13379 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:55:48.605722 13198 raft_consensus.cc:1275] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:48.607460 13379 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:48.638090 13126 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:48.639360 13379 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:55:48.646849 13202 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 4 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:48.648525 13374 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:55:48.655370 13202 raft_consensus.cc:1275] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:48.656517 13282 consensus_queue.cc:1048] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:48.665302 13126 raft_consensus.cc:1275] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:48.666863 13356 consensus_queue.cc:1048] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:48.671378 13126 raft_consensus.cc:1275] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:48.674199 13356 consensus_queue.cc:1048] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:48.685930 13202 raft_consensus.cc:1275] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Refusing update from remote peer 0da16b4100324a709e8662bc7053426c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:55:48.688395 13282 consensus_queue.cc:1048] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:55:50.987835 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:50.995805 13052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:52028
I20260709 12:55:50.996620 13052 raft_consensus.cc:606] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 LEADER]: Received request to transfer leadership to TS 0e354d68b8c64e379bc71b28c3af85ec
I20260709 12:55:50.999432 13052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:52028
I20260709 12:55:50.999984 13052 raft_consensus.cc:606] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Received request to transfer leadership to TS f9825019a5484ebe9114919d65c77120
I20260709 12:55:51.002669 13052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:52028
I20260709 12:55:51.003171 13052 raft_consensus.cc:606] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Received request to transfer leadership to TS f9825019a5484ebe9114919d65c77120
I20260709 12:55:51.005538 13052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405"
dest_uuid: "0da16b4100324a709e8662bc7053426c"
mode: GRACEFUL
new_leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:52028
I20260709 12:55:51.005947 13052 raft_consensus.cc:606] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 LEADER]: Received request to transfer leadership to TS 0e354d68b8c64e379bc71b28c3af85ec
I20260709 12:55:51.006919 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:55:51.007692 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:55:51.016993 13282 raft_consensus.cc:993] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c: : Instructing follower f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:51.017519 13387 raft_consensus.cc:1081] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Signalling peer f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:51.019307 13202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:55:51.019855 13202 raft_consensus.cc:493] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:51.020215 13202 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:51.026140 13202 raft_consensus.cc:515] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.027585 13202 leader_election.cc:290] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 3 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:51.028762 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:51.028762 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12d95d7e0bc4d26911222a16b8cdf40" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:51.029598 13126 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:51.029592 13052 raft_consensus.cc:3055] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:51.030233 13052 raft_consensus.cc:740] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:51.031203 13052 consensus_queue.cc:260] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.032720 13052 raft_consensus.cc:3060] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:51.037957 13126 raft_consensus.cc:2468] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 3.
I20260709 12:55:51.039480 13166 leader_election.cc:304] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:51.040386 13361 raft_consensus.cc:2804] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:51.040566 13052 raft_consensus.cc:2468] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 3.
I20260709 12:55:51.041556 13361 raft_consensus.cc:697] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Becoming Leader. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:51.042696 13361 consensus_queue.cc:237] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.050364 12948 catalog_manager.cc:5697] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 reported cstate change: term changed from 2 to 3, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to f9825019a5484ebe9114919d65c77120 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "f9825019a5484ebe9114919d65c77120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:51.106652 13282 raft_consensus.cc:993] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: : Instructing follower 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:51.107134 13387 raft_consensus.cc:1081] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 LEADER]: Signalling peer 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:51.108685 13126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:41496
I20260709 12:55:51.109130 13126 raft_consensus.cc:493] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:51.109386 13126 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:51.113875 13126 raft_consensus.cc:515] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.115293 13126 leader_election.cc:290] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 4 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:51.116444 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:51.116529 13202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee6637bd86a34221abc66eeaffb87405" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:51.117024 13052 raft_consensus.cc:3055] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:55:51.117264 13202 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:51.117373 13052 raft_consensus.cc:740] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:51.118199 13052 consensus_queue.cc:260] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.119228 13052 raft_consensus.cc:3060] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:55:51.123700 13202 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 4.
I20260709 12:55:51.125047 13093 leader_election.cc:304] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:51.125414 13052 raft_consensus.cc:2468] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 4.
I20260709 12:55:51.125823 13282 raft_consensus.cc:993] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c: : Instructing follower f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:51.126003 13369 raft_consensus.cc:2804] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:55:51.126282 13282 raft_consensus.cc:1081] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Signalling peer f9825019a5484ebe9114919d65c77120 to start an election
I20260709 12:55:51.127599 13198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f"
dest_uuid: "f9825019a5484ebe9114919d65c77120"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:55:51.128073 13198 raft_consensus.cc:493] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:51.128386 13198 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:51.133272 13198 raft_consensus.cc:515] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.134586 13369 raft_consensus.cc:697] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:51.135831 13198 leader_election.cc:290] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 3 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66:36333)
I20260709 12:55:51.136188 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:51.136154 13369 consensus_queue.cc:237] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.136889 13126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61bc33a644d94fc29c466d7d50d32b3f" candidate_uuid: "f9825019a5484ebe9114919d65c77120" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
I20260709 12:55:51.137006 13052 raft_consensus.cc:3055] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:55:51.137451 13126 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:51.137434 13052 raft_consensus.cc:740] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:51.138477 13052 consensus_queue.cc:260] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.139822 13052 raft_consensus.cc:3060] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:55:51.143560 13126 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 3.
I20260709 12:55:51.144685 13166 leader_election.cc:304] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:51.146145 12948 catalog_manager.cc:5697] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 3 to 4, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 4 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:51.147549 13361 raft_consensus.cc:2804] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:55:51.147464 13052 raft_consensus.cc:2468] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9825019a5484ebe9114919d65c77120 in term 3.
I20260709 12:55:51.148185 13361 raft_consensus.cc:697] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Becoming Leader. State: Replica: f9825019a5484ebe9114919d65c77120, State: Running, Role: LEADER
I20260709 12:55:51.149053 13361 consensus_queue.cc:237] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.156680 12951 catalog_manager.cc:5697] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 reported cstate change: term changed from 2 to 3, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to f9825019a5484ebe9114919d65c77120 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "f9825019a5484ebe9114919d65c77120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:51.390455 13282 raft_consensus.cc:993] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: : Instructing follower 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:51.390936 13387 raft_consensus.cc:1081] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 LEADER]: Signalling peer 0e354d68b8c64e379bc71b28c3af85ec to start an election
I20260709 12:55:51.392696 13126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3"
dest_uuid: "0e354d68b8c64e379bc71b28c3af85ec"
 from {username='slave'} at 127.0.0.1:41496
I20260709 12:55:51.393287 13126 raft_consensus.cc:493] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:55:51.393572 13126 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:55:51.398044 13126 raft_consensus.cc:515] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.399489 13126 leader_election.cc:290] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 5 election: Requested vote from peers 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681), f9825019a5484ebe9114919d65c77120 (127.12.30.67:37883)
I20260709 12:55:51.400489 13052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "0da16b4100324a709e8662bc7053426c"
I20260709 12:55:51.400738 13198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2109da5ae593490ba3803074d274d6d3" candidate_uuid: "0e354d68b8c64e379bc71b28c3af85ec" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f9825019a5484ebe9114919d65c77120"
I20260709 12:55:51.400969 13052 raft_consensus.cc:3055] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:55:51.401285 13052 raft_consensus.cc:740] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0da16b4100324a709e8662bc7053426c, State: Running, Role: LEADER
I20260709 12:55:51.401320 13198 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:55:51.401986 13052 consensus_queue.cc:260] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.402874 13052 raft_consensus.cc:3060] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:55:51.406831 13198 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 5.
I20260709 12:55:51.407500 13052 raft_consensus.cc:2468] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e354d68b8c64e379bc71b28c3af85ec in term 5.
I20260709 12:55:51.408211 13093 leader_election.cc:304] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e354d68b8c64e379bc71b28c3af85ec, f9825019a5484ebe9114919d65c77120; no voters: 
I20260709 12:55:51.409041 13369 raft_consensus.cc:2804] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:55:51.409549 13369 raft_consensus.cc:697] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 5 LEADER]: Becoming Leader. State: Replica: 0e354d68b8c64e379bc71b28c3af85ec, State: Running, Role: LEADER
I20260709 12:55:51.410315 13369 consensus_queue.cc:237] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } }
I20260709 12:55:51.416601 12951 catalog_manager.cc:5697] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec reported cstate change: term changed from 4 to 5, leader changed from 0da16b4100324a709e8662bc7053426c (127.12.30.65) to 0e354d68b8c64e379bc71b28c3af85ec (127.12.30.66). New cstate: current_term: 5 leader_uuid: "0e354d68b8c64e379bc71b28c3af85ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:51.528383 13126 raft_consensus.cc:1275] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:51.529531 13361 consensus_queue.cc:1048] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:51.540320 13052 raft_consensus.cc:1275] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:51.542263 13388 consensus_queue.cc:1048] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:51.553548 13126 raft_consensus.cc:1275] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:51.554719 13388 consensus_queue.cc:1048] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e354d68b8c64e379bc71b28c3af85ec" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:51.568125 13052 raft_consensus.cc:1275] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 4 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:51.569399 13394 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:55:51.571746 13198 raft_consensus.cc:1275] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 4 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:55:51.574283 13052 raft_consensus.cc:1275] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Refusing update from remote peer f9825019a5484ebe9114919d65c77120: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:55:51.574183 13369 consensus_queue.cc:1048] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:55:51.575673 13404 consensus_queue.cc:1048] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:55:51.841259 13053 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 5 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:55:51.842554 13369 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da16b4100324a709e8662bc7053426c" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:55:51.851866 13198 raft_consensus.cc:1275] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 5 FOLLOWER]: Refusing update from remote peer 0e354d68b8c64e379bc71b28c3af85ec: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:55:51.853165 13408 consensus_queue.cc:1048] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9825019a5484ebe9114919d65c77120" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:55:52.008713 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:55:52.015350 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
I20260709 12:55:52.044798 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:55:52.045497 12409 tablet_replica.cc:333] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.046121 12409 raft_consensus.cc:2243] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.046963 12409 raft_consensus.cc:2272] T 3ba09c8e124b4b6686f4d805bf766387 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.049463 12409 tablet_replica.cc:333] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.050309 12409 raft_consensus.cc:2243] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.050778 12409 raft_consensus.cc:2272] T 5d4c36c3db804264b072c71ea9da961d P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.052644 12409 tablet_replica.cc:333] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.053083 12409 raft_consensus.cc:2243] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.053418 12409 raft_consensus.cc:2272] T 2109da5ae593490ba3803074d274d6d3 P 0da16b4100324a709e8662bc7053426c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.056056 12409 tablet_replica.cc:333] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.056685 12409 raft_consensus.cc:2243] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.057063 12409 raft_consensus.cc:2272] T abd5e447b5bd4309ac709b3732072473 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.058691 12409 tablet_replica.cc:333] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.059139 12409 raft_consensus.cc:2243] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.059468 12409 raft_consensus.cc:2272] T ee6637bd86a34221abc66eeaffb87405 P 0da16b4100324a709e8662bc7053426c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.061697 12409 tablet_replica.cc:333] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.062183 12409 raft_consensus.cc:2243] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.062855 12409 raft_consensus.cc:2272] T 4f37dd0690244d93b6acbadd20a93782 P 0da16b4100324a709e8662bc7053426c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.065388 12409 tablet_replica.cc:333] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.066062 12409 raft_consensus.cc:2243] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.066562 12409 raft_consensus.cc:2272] T e12d95d7e0bc4d26911222a16b8cdf40 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.068660 12409 tablet_replica.cc:333] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.069162 12409 raft_consensus.cc:2243] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.069875 12409 raft_consensus.cc:2272] T 6c167a5eddee447daab59df9e2233a24 P 0da16b4100324a709e8662bc7053426c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.071732 12409 tablet_replica.cc:333] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.072240 12409 raft_consensus.cc:2243] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.072935 12409 raft_consensus.cc:2272] T 58abd247ac31444d9d71d6874fe90bb7 P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.074471 12409 tablet_replica.cc:333] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c: stopping tablet replica
I20260709 12:55:52.074932 12409 raft_consensus.cc:2243] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.075315 12409 raft_consensus.cc:2272] T 61bc33a644d94fc29c466d7d50d32b3f P 0da16b4100324a709e8662bc7053426c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:55:52.079080 13168 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
W20260709 12:55:52.082281 13168 consensus_peers.cc:597] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 -> Peer 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681): Couldn't send request to peer 0da16b4100324a709e8662bc7053426c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:52.099285 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
I20260709 12:55:52.121086 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
W20260709 12:55:52.146600 13094 consensus_peers.cc:597] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec -> Peer 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681): Couldn't send request to peer 0da16b4100324a709e8662bc7053426c. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:42681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:55:52.150456 13168 consensus_peers.cc:597] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 -> Peer 0da16b4100324a709e8662bc7053426c (127.12.30.65:42681): Couldn't send request to peer 0da16b4100324a709e8662bc7053426c. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:42681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:52.158572 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:55:52.159427 12409 tablet_replica.cc:333] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.160116 12409 raft_consensus.cc:2243] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.160593 12409 raft_consensus.cc:2272] T 3ba09c8e124b4b6686f4d805bf766387 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.162319 12409 tablet_replica.cc:333] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.162868 12409 raft_consensus.cc:2243] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.163287 12409 raft_consensus.cc:2272] T e12d95d7e0bc4d26911222a16b8cdf40 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.165737 12409 tablet_replica.cc:333] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.166334 12409 raft_consensus.cc:2243] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.166841 12409 raft_consensus.cc:2272] T 5d4c36c3db804264b072c71ea9da961d P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.168570 12409 tablet_replica.cc:333] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.169221 12409 raft_consensus.cc:2243] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.170063 12409 raft_consensus.cc:2272] T 2109da5ae593490ba3803074d274d6d3 P 0e354d68b8c64e379bc71b28c3af85ec [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.172748 12409 tablet_replica.cc:333] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.173215 12409 raft_consensus.cc:2243] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.173897 12409 raft_consensus.cc:2272] T abd5e447b5bd4309ac709b3732072473 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.175626 12409 tablet_replica.cc:333] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.176147 12409 raft_consensus.cc:2243] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.176506 12409 raft_consensus.cc:2272] T 4f37dd0690244d93b6acbadd20a93782 P 0e354d68b8c64e379bc71b28c3af85ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.178200 12409 tablet_replica.cc:333] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.178718 12409 raft_consensus.cc:2243] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.179352 12409 raft_consensus.cc:2272] T ee6637bd86a34221abc66eeaffb87405 P 0e354d68b8c64e379bc71b28c3af85ec [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.181411 12409 tablet_replica.cc:333] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.181874 12409 raft_consensus.cc:2243] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.182227 12409 raft_consensus.cc:2272] T 6c167a5eddee447daab59df9e2233a24 P 0e354d68b8c64e379bc71b28c3af85ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.184305 12409 tablet_replica.cc:333] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.184751 12409 raft_consensus.cc:2243] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.185104 12409 raft_consensus.cc:2272] T 58abd247ac31444d9d71d6874fe90bb7 P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.186744 12409 tablet_replica.cc:333] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec: stopping tablet replica
I20260709 12:55:52.187136 12409 raft_consensus.cc:2243] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.187490 12409 raft_consensus.cc:2272] T 61bc33a644d94fc29c466d7d50d32b3f P 0e354d68b8c64e379bc71b28c3af85ec [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.216077 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
I20260709 12:55:52.234823 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
I20260709 12:55:52.262347 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:55:52.263154 12409 tablet_replica.cc:333] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.263772 12409 raft_consensus.cc:2243] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.264299 12409 raft_consensus.cc:2272] T 3ba09c8e124b4b6686f4d805bf766387 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.266062 12409 tablet_replica.cc:333] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.266594 12409 raft_consensus.cc:2243] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.267345 12409 raft_consensus.cc:2272] T 5d4c36c3db804264b072c71ea9da961d P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.269155 12409 tablet_replica.cc:333] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.269716 12409 raft_consensus.cc:2243] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.270135 12409 raft_consensus.cc:2272] T 2109da5ae593490ba3803074d274d6d3 P f9825019a5484ebe9114919d65c77120 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.272439 12409 tablet_replica.cc:333] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.273020 12409 raft_consensus.cc:2243] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.273748 12409 raft_consensus.cc:2272] T e12d95d7e0bc4d26911222a16b8cdf40 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.275720 12409 tablet_replica.cc:333] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.276310 12409 raft_consensus.cc:2243] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.276753 12409 raft_consensus.cc:2272] T abd5e447b5bd4309ac709b3732072473 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.278373 12409 tablet_replica.cc:333] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.278904 12409 raft_consensus.cc:2243] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.279304 12409 raft_consensus.cc:2272] T 4f37dd0690244d93b6acbadd20a93782 P f9825019a5484ebe9114919d65c77120 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.280992 12409 tablet_replica.cc:333] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.281457 12409 raft_consensus.cc:2243] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.281873 12409 raft_consensus.cc:2272] T ee6637bd86a34221abc66eeaffb87405 P f9825019a5484ebe9114919d65c77120 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.284029 12409 tablet_replica.cc:333] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.284521 12409 raft_consensus.cc:2243] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.284909 12409 raft_consensus.cc:2272] T 6c167a5eddee447daab59df9e2233a24 P f9825019a5484ebe9114919d65c77120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.286979 12409 tablet_replica.cc:333] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.287420 12409 raft_consensus.cc:2243] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:55:52.287819 12409 raft_consensus.cc:2272] T 58abd247ac31444d9d71d6874fe90bb7 P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.289377 12409 tablet_replica.cc:333] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120: stopping tablet replica
I20260709 12:55:52.289821 12409 raft_consensus.cc:2243] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.290542 12409 raft_consensus.cc:2272] T 61bc33a644d94fc29c466d7d50d32b3f P f9825019a5484ebe9114919d65c77120 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.320461 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
I20260709 12:55:52.338745 12409 master.cc:562] Master@127.12.30.126:37625 shutting down...
I20260709 12:55:52.358747 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:55:52.359277 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:55:52.359616 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P b505f0bdb6434721959ef72d5d2c2746: stopping tablet replica
I20260709 12:55:52.379685 12409 master.cc:584] Master@127.12.30.126:37625 shutdown complete.
0da16b4100324a709e8662bc7053426c  5
0e354d68b8c64e379bc71b28c3af85ec  4
f9825019a5484ebe9114919d65c77120  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10644 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:55:52.421360 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:40047
I20260709 12:55:52.422499 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:52.429160 13417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.429224 13416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.429188 13419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:52.432684 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:52.433899 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:52.434113 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:52.434255 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601752434244 us; error 0 us; skew 500 ppm
I20260709 12:55:52.434737 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:52.437369 12409 webserver.cc:533] Webserver started at http://127.12.30.126:37591/ using document root <none> and password file <none>
I20260709 12:55:52.437860 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:52.438019 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:52.438242 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:52.439440 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "7c7c6d1a2ec946878b513e7eb3a47a90"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.445487 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:55:52.449020 13424 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.449857 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:55:52.450189 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "7c7c6d1a2ec946878b513e7eb3a47a90"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.450516 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:52.480232 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:52.481478 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:52.520100 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:40047
I20260709 12:55:52.520164 13475 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:40047 every 8 connection(s)
I20260709 12:55:52.523968 13476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:52.533515 13476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Bootstrap starting.
I20260709 12:55:52.538058 13476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:52.542074 13476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: No bootstrap required, opened a new log
I20260709 12:55:52.544211 13476 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" member_type: VOTER }
I20260709 12:55:52.544605 13476 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:52.544814 13476 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7c6d1a2ec946878b513e7eb3a47a90, State: Initialized, Role: FOLLOWER
I20260709 12:55:52.545436 13476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" member_type: VOTER }
I20260709 12:55:52.545879 13476 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:55:52.546103 13476 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:55:52.546342 13476 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:52.550477 13476 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" member_type: VOTER }
I20260709 12:55:52.550987 13476 leader_election.cc:304] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c7c6d1a2ec946878b513e7eb3a47a90; no voters: 
I20260709 12:55:52.552232 13476 leader_election.cc:290] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:55:52.552608 13479 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:52.553923 13479 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 1 LEADER]: Becoming Leader. State: Replica: 7c7c6d1a2ec946878b513e7eb3a47a90, State: Running, Role: LEADER
I20260709 12:55:52.554615 13479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" member_type: VOTER }
I20260709 12:55:52.555266 13476 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:55:52.559186 13480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" member_type: VOTER } }
I20260709 12:55:52.559360 13481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c7c6d1a2ec946878b513e7eb3a47a90. Latest consensus state: current_term: 1 leader_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c7c6d1a2ec946878b513e7eb3a47a90" member_type: VOTER } }
I20260709 12:55:52.559908 13480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [sys.catalog]: This master's current role is: LEADER
I20260709 12:55:52.560031 13481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [sys.catalog]: This master's current role is: LEADER
I20260709 12:55:52.562837 13485 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:55:52.567652 13485 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:55:52.572643 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:55:52.576735 13485 catalog_manager.cc:1383] Generated new cluster ID: 6c8341a2be864d92a2c4132149a75673
I20260709 12:55:52.577100 13485 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:55:52.602367 13485 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:55:52.603736 13485 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:55:52.622233 13485 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Generated new TSK 0
I20260709 12:55:52.622829 13485 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:55:52.640096 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:52.645906 13497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.647222 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.648788 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:52.649497 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:52.650456 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:52.650707 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:52.650893 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601752650858 us; error 0 us; skew 500 ppm
I20260709 12:55:52.651448 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:52.653993 12409 webserver.cc:533] Webserver started at http://127.12.30.65:36381/ using document root <none> and password file <none>
I20260709 12:55:52.654512 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:52.654711 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:52.655001 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:52.656164 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "ad2fdb1517c542439654bd3f7c72dad0"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.660696 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:55:52.664397 13505 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.665287 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:55:52.665530 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "ad2fdb1517c542439654bd3f7c72dad0"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.665822 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:52.676856 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:52.678047 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:52.679431 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:52.681721 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:52.681900 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.682193 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:52.682340 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.723443 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:39209
I20260709 12:55:52.723563 13568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:39209 every 8 connection(s)
I20260709 12:55:52.728374 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:52.735580 13573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.737310 13574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.738497 13576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:52.740077 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:52.741219 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:52.741407 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:52.741528 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601752741518 us; error 0 us; skew 500 ppm
I20260709 12:55:52.741966 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:52.744694 12409 webserver.cc:533] Webserver started at http://127.12.30.66:40767/ using document root <none> and password file <none>
I20260709 12:55:52.745329 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:52.745566 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:52.745882 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:52.747124 13569 heartbeater.cc:344] Connected to a master server at 127.12.30.126:40047
I20260709 12:55:52.747244 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.747608 13569 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:52.748451 13569 heartbeater.cc:507] Master 127.12.30.126:40047 requested a full tablet report, sending...
I20260709 12:55:52.750624 13441 ts_manager.cc:194] Registered new tserver with Master: ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:55:52.752575 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 12:55:52.753055 13441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49880
I20260709 12:55:52.756536 13581 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.757526 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 12:55:52.757789 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.758030 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:52.769411 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:52.770567 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:52.772034 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:52.774290 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:52.774472 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.774778 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:52.774936 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.814539 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:39367
I20260709 12:55:52.814648 13643 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:39367 every 8 connection(s)
I20260709 12:55:52.819372 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:55:52.826695 13647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.828238 13648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:55:52.830760 13650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:55:52.831673 12409 server_base.cc:1061] running on GCE node
I20260709 12:55:52.832531 13644 heartbeater.cc:344] Connected to a master server at 127.12.30.126:40047
I20260709 12:55:52.832911 13644 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:52.833108 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:55:52.833369 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:55:52.833542 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601752833524 us; error 0 us; skew 500 ppm
I20260709 12:55:52.833787 13644 heartbeater.cc:507] Master 127.12.30.126:40047 requested a full tablet report, sending...
I20260709 12:55:52.834172 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:55:52.836045 13441 ts_manager.cc:194] Registered new tserver with Master: d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:52.837101 12409 webserver.cc:533] Webserver started at http://127.12.30.67:45625/ using document root <none> and password file <none>
I20260709 12:55:52.837735 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:55:52.837869 13441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49892
I20260709 12:55:52.837976 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:55:52.838376 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:55:52.839820 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "058c47afa94546dba814cf345f414f4a"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.844426 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:55:52.847710 13655 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.848589 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:55:52.848860 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "058c47afa94546dba814cf345f414f4a"
format_stamp: "Formatted at 2026-07-09 12:55:52 on dist-test-slave-mlt7"
I20260709 12:55:52.849115 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:55:52.862627 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:55:52.863799 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:55:52.865295 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:55:52.867725 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:55:52.868057 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:55:52.868314 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:55:52.868484 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:55:52.911989 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:41755
I20260709 12:55:52.912145 13717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:41755 every 8 connection(s)
I20260709 12:55:52.927999 13718 heartbeater.cc:344] Connected to a master server at 127.12.30.126:40047
I20260709 12:55:52.928515 13718 heartbeater.cc:461] Registering TS with master...
I20260709 12:55:52.929687 13718 heartbeater.cc:507] Master 127.12.30.126:40047 requested a full tablet report, sending...
I20260709 12:55:52.931718 13441 ts_manager.cc:194] Registered new tserver with Master: 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:52.931860 12409 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016423212s
I20260709 12:55:52.933202 13441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49896
I20260709 12:55:52.962914 13441 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49904:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:55:52.969341 13441 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:55:53.188578 13683 tablet_service.cc:1511] Processing CreateTablet for tablet 35f2068be68d418fb0b123e59c3db757 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:55:53.189103 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:55:53.190743 13683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f2068be68d418fb0b123e59c3db757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.191406 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.191744 13679 tablet_service.cc:1511] Processing CreateTablet for tablet 5db10903ab7d403ba303d72b7f26dfed (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:55:53.193199 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db10903ab7d403ba303d72b7f26dfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.197643 13680 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee08c4df55f41c7a18951168c44fb5a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:55:53.198132 13678 tablet_service.cc:1511] Processing CreateTablet for tablet bc9a7a0b2fe343e48ea3ff1bae778069 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:55:53.199282 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee08c4df55f41c7a18951168c44fb5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.202766 13531 tablet_service.cc:1511] Processing CreateTablet for tablet 12822fb99cf24dbcac8dbb9d77e56af8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:55:53.203047 13529 tablet_service.cc:1511] Processing CreateTablet for tablet 5db10903ab7d403ba303d72b7f26dfed (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:55:53.205179 13528 tablet_service.cc:1511] Processing CreateTablet for tablet bc9a7a0b2fe343e48ea3ff1bae778069 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:55:53.205288 13677 tablet_service.cc:1511] Processing CreateTablet for tablet a315f47e47184081bd93e74c72dba511 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:55:53.207094 13677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a315f47e47184081bd93e74c72dba511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.207314 13530 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee08c4df55f41c7a18951168c44fb5a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:55:53.209640 13676 tablet_service.cc:1511] Processing CreateTablet for tablet c88adb75300c48e98d20928bce53551d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:55:53.211169 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c88adb75300c48e98d20928bce53551d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.201349 13681 tablet_service.cc:1511] Processing CreateTablet for tablet 12822fb99cf24dbcac8dbb9d77e56af8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:55:53.212776 13533 tablet_service.cc:1511] Processing CreateTablet for tablet 35f2068be68d418fb0b123e59c3db757 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:55:53.214365 13681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12822fb99cf24dbcac8dbb9d77e56af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.201385 13532 tablet_service.cc:1511] Processing CreateTablet for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:55:53.215130 13675 tablet_service.cc:1511] Processing CreateTablet for tablet 2c5bef52669b468c8765408698f5884d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:55:53.216670 13678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9a7a0b2fe343e48ea3ff1bae778069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.217463 13675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c5bef52669b468c8765408698f5884d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.222080 13527 tablet_service.cc:1511] Processing CreateTablet for tablet a315f47e47184081bd93e74c72dba511 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:55:53.223097 13674 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd04e83486049ae859032792f652869 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:55:53.223949 13531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12822fb99cf24dbcac8dbb9d77e56af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.225028 13674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd04e83486049ae859032792f652869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.229076 13527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a315f47e47184081bd93e74c72dba511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.234724 13526 tablet_service.cc:1511] Processing CreateTablet for tablet c88adb75300c48e98d20928bce53551d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:55:53.239209 13525 tablet_service.cc:1511] Processing CreateTablet for tablet 2c5bef52669b468c8765408698f5884d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:55:53.242260 13524 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd04e83486049ae859032792f652869 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:55:53.248696 13609 tablet_service.cc:1511] Processing CreateTablet for tablet 35f2068be68d418fb0b123e59c3db757 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:55:53.250557 13609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f2068be68d418fb0b123e59c3db757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.252938 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c88adb75300c48e98d20928bce53551d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.253276 13608 tablet_service.cc:1511] Processing CreateTablet for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:55:53.255683 13607 tablet_service.cc:1511] Processing CreateTablet for tablet 12822fb99cf24dbcac8dbb9d77e56af8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:55:53.258045 13604 tablet_service.cc:1511] Processing CreateTablet for tablet bc9a7a0b2fe343e48ea3ff1bae778069 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:55:53.264400 13525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c5bef52669b468c8765408698f5884d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.265175 13532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.265730 13608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.266636 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9a7a0b2fe343e48ea3ff1bae778069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.266415 13606 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee08c4df55f41c7a18951168c44fb5a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:55:53.268427 13533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f2068be68d418fb0b123e59c3db757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.269277 13605 tablet_service.cc:1511] Processing CreateTablet for tablet 5db10903ab7d403ba303d72b7f26dfed (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:55:53.269981 13530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee08c4df55f41c7a18951168c44fb5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.272455 13606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee08c4df55f41c7a18951168c44fb5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.296375 13605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db10903ab7d403ba303d72b7f26dfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.272984 13524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd04e83486049ae859032792f652869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.282351 13603 tablet_service.cc:1511] Processing CreateTablet for tablet a315f47e47184081bd93e74c72dba511 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:55:53.300817 13733 tablet_bootstrap.cc:492] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:53.301034 13603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a315f47e47184081bd93e74c72dba511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.285410 13602 tablet_service.cc:1511] Processing CreateTablet for tablet c88adb75300c48e98d20928bce53551d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:55:53.288378 13601 tablet_service.cc:1511] Processing CreateTablet for tablet 2c5bef52669b468c8765408698f5884d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:55:53.291292 13600 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd04e83486049ae859032792f652869 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:55:53.302888 13607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12822fb99cf24dbcac8dbb9d77e56af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.306375 13600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd04e83486049ae859032792f652869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.307251 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c88adb75300c48e98d20928bce53551d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.307554 13529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db10903ab7d403ba303d72b7f26dfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.308305 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c5bef52669b468c8765408698f5884d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.309849 13604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9a7a0b2fe343e48ea3ff1bae778069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.274241 13683 tablet_service.cc:1511] Processing CreateTablet for tablet 12c636bac0584e2e92ff66a089d880ee (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:55:53.339496 13675 tablet_service.cc:1511] Processing CreateTablet for tablet 87a8c869a6b840e591423340bc32af00 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:55:53.341038 13675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87a8c869a6b840e591423340bc32af00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.346550 13674 tablet_service.cc:1511] Processing CreateTablet for tablet 72d95a7092b0445eb00a28a1fdb8d239 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:55:53.364087 13678 tablet_service.cc:1511] Processing CreateTablet for tablet 5406eb61fd534169b3c1645f80144779 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:55:53.378450 13683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c636bac0584e2e92ff66a089d880ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.392092 13674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d95a7092b0445eb00a28a1fdb8d239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.392603 13735 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:53.404768 13609 tablet_service.cc:1511] Processing CreateTablet for tablet 12c636bac0584e2e92ff66a089d880ee (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:55:53.406673 13609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c636bac0584e2e92ff66a089d880ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.408879 13734 tablet_bootstrap.cc:492] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:53.418874 13735 tablet_bootstrap.cc:654] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:53.420680 13680 tablet_service.cc:1511] Processing CreateTablet for tablet 2181ddb48f3d44d583ad7ecf793222fa (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:55:53.422233 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2181ddb48f3d44d583ad7ecf793222fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.425294 13676 tablet_service.cc:1511] Processing CreateTablet for tablet 22f14b8d35f741a28ad886b19614c4c8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:55:53.427522 13733 tablet_bootstrap.cc:654] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:53.431416 13531 tablet_service.cc:1511] Processing CreateTablet for tablet 12c636bac0584e2e92ff66a089d880ee (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:55:53.447692 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 68566bdf08224971a44a28240a8bf9e4 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:55:53.439002 13527 tablet_service.cc:1511] Processing CreateTablet for tablet 87a8c869a6b840e591423340bc32af00 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:55:53.468209 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68566bdf08224971a44a28240a8bf9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.468920 13531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c636bac0584e2e92ff66a089d880ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.469887 13527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87a8c869a6b840e591423340bc32af00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.471215 13678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5406eb61fd534169b3c1645f80144779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.474104 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f14b8d35f741a28ad886b19614c4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.497702 13681 tablet_service.cc:1511] Processing CreateTablet for tablet 91a3f49b0e164d27994cfe440af6f514 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:55:53.501260 13525 tablet_service.cc:1511] Processing CreateTablet for tablet 72d95a7092b0445eb00a28a1fdb8d239 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:55:53.503381 13532 tablet_service.cc:1511] Processing CreateTablet for tablet 5406eb61fd534169b3c1645f80144779 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
W20260709 12:55:53.482087 13427 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 43955.6us
I20260709 12:55:53.506702 13681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a3f49b0e164d27994cfe440af6f514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.509821 13734 tablet_bootstrap.cc:654] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:53.518606 13733 tablet_bootstrap.cc:492] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:53.519166 13733 ts_tablet_manager.cc:1403] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.219s	user 0.013s	sys 0.002s
I20260709 12:55:53.522311 13733 raft_consensus.cc:359] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:53.529341 13532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5406eb61fd534169b3c1645f80144779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.529531 13733 raft_consensus.cc:385] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:53.530169 13733 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:53.531081 13733 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:53.531824 13735 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:53.532410 13735 ts_tablet_manager.cc:1403] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.140s	user 0.007s	sys 0.019s
I20260709 12:55:53.534236 13524 tablet_service.cc:1511] Processing CreateTablet for tablet 68566bdf08224971a44a28240a8bf9e4 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:55:53.547006 13735 raft_consensus.cc:359] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:53.547873 13735 raft_consensus.cc:385] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:53.548213 13735 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:53.557360 13674 tablet_service.cc:1511] Processing CreateTablet for tablet 86a0c010e8cc4b828fec8e8547e37902 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:55:53.559052 13674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86a0c010e8cc4b828fec8e8547e37902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.566505 13679 tablet_service.cc:1511] Processing CreateTablet for tablet be11851ed11044eea9f0fc3dbdbe0599 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:55:53.566756 13735 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:53.568132 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:53.570523 13601 tablet_service.cc:1511] Processing CreateTablet for tablet 2181ddb48f3d44d583ad7ecf793222fa (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:55:53.574883 13604 tablet_service.cc:1511] Processing CreateTablet for tablet 87a8c869a6b840e591423340bc32af00 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:55:53.578745 13680 tablet_service.cc:1511] Processing CreateTablet for tablet 82077fd60c1346d7b057a65a778ee2f5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:55:53.579905 13525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d95a7092b0445eb00a28a1fdb8d239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.592113 13605 tablet_service.cc:1511] Processing CreateTablet for tablet 72d95a7092b0445eb00a28a1fdb8d239 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:55:53.594031 13524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68566bdf08224971a44a28240a8bf9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.601547 13528 tablet_service.cc:1511] Processing CreateTablet for tablet 2181ddb48f3d44d583ad7ecf793222fa (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:55:53.602795 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2181ddb48f3d44d583ad7ecf793222fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.603837 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2181ddb48f3d44d583ad7ecf793222fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.584059 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be11851ed11044eea9f0fc3dbdbe0599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.587545 13675 tablet_service.cc:1511] Processing CreateTablet for tablet f125a6a76f974409807014b9ddaa1b8c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:55:53.608377 13526 tablet_service.cc:1511] Processing CreateTablet for tablet 91a3f49b0e164d27994cfe440af6f514 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:55:53.609467 13604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87a8c869a6b840e591423340bc32af00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.591365 13533 tablet_service.cc:1511] Processing CreateTablet for tablet 22f14b8d35f741a28ad886b19614c4c8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:55:53.620158 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a3f49b0e164d27994cfe440af6f514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.620622 13733 ts_tablet_manager.cc:1434] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.101s	user 0.044s	sys 0.039s
I20260709 12:55:53.621455 13718 heartbeater.cc:499] Master 127.12.30.126:40047 was elected leader, sending a full tablet report...
I20260709 12:55:53.627902 13733 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:53.640343 13605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d95a7092b0445eb00a28a1fdb8d239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.650051 13569 heartbeater.cc:499] Master 127.12.30.126:40047 was elected leader, sending a full tablet report...
I20260709 12:55:53.648015 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82077fd60c1346d7b057a65a778ee2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.652539 13602 tablet_service.cc:1511] Processing CreateTablet for tablet 22f14b8d35f741a28ad886b19614c4c8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:55:53.658671 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f14b8d35f741a28ad886b19614c4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.664626 13735 ts_tablet_manager.cc:1434] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.132s	user 0.015s	sys 0.002s
I20260709 12:55:53.670801 13734 tablet_bootstrap.cc:492] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:53.678690 13734 ts_tablet_manager.cc:1403] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.270s	user 0.018s	sys 0.031s
I20260709 12:55:53.678752 13608 tablet_service.cc:1511] Processing CreateTablet for tablet 91a3f49b0e164d27994cfe440af6f514 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:55:53.680210 13606 tablet_service.cc:1511] Processing CreateTablet for tablet 5406eb61fd534169b3c1645f80144779 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:55:53.682008 13734 raft_consensus.cc:359] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:53.682760 13734 raft_consensus.cc:385] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:53.683054 13734 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:53.672241 13533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f14b8d35f741a28ad886b19614c4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.666497 13603 tablet_service.cc:1511] Processing CreateTablet for tablet 68566bdf08224971a44a28240a8bf9e4 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:55:53.698493 13675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f125a6a76f974409807014b9ddaa1b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.699656 13608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a3f49b0e164d27994cfe440af6f514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.693372 13734 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:53.720288 13606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5406eb61fd534169b3c1645f80144779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.700618 13735 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:53.708385 13607 tablet_service.cc:1511] Processing CreateTablet for tablet 86a0c010e8cc4b828fec8e8547e37902 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:55:53.727591 13607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86a0c010e8cc4b828fec8e8547e37902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.729336 13603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68566bdf08224971a44a28240a8bf9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.731137 13676 tablet_service.cc:1511] Processing CreateTablet for tablet 973866cfc22e48ea931cb1a6fb16872f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:55:53.732515 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973866cfc22e48ea931cb1a6fb16872f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.734107 13524 tablet_service.cc:1511] Processing CreateTablet for tablet be11851ed11044eea9f0fc3dbdbe0599 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:55:53.733076 13678 tablet_service.cc:1511] Processing CreateTablet for tablet 1c9f20cfde5343c9b91c4e2adc5320ce (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:55:53.735431 13524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be11851ed11044eea9f0fc3dbdbe0599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.708326 13674 tablet_service.cc:1511] Processing CreateTablet for tablet f2ce7a788b8c40629fc1921a4286d084 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:55:53.711990 13609 tablet_service.cc:1511] Processing CreateTablet for tablet be11851ed11044eea9f0fc3dbdbe0599 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:55:53.736722 13609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be11851ed11044eea9f0fc3dbdbe0599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.739073 13600 tablet_service.cc:1511] Processing CreateTablet for tablet 82077fd60c1346d7b057a65a778ee2f5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:55:53.740350 13600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82077fd60c1346d7b057a65a778ee2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.747231 13527 tablet_service.cc:1511] Processing CreateTablet for tablet 82077fd60c1346d7b057a65a778ee2f5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:55:53.748697 13527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82077fd60c1346d7b057a65a778ee2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.749275 13678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c9f20cfde5343c9b91c4e2adc5320ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.752336 13733 tablet_bootstrap.cc:654] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:53.757575 13677 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9bdd77e564429c9c0434b358f48302 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:55:53.759737 13674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ce7a788b8c40629fc1921a4286d084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.686340 13529 tablet_service.cc:1511] Processing CreateTablet for tablet 86a0c010e8cc4b828fec8e8547e37902 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:55:53.765811 13529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86a0c010e8cc4b828fec8e8547e37902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.772634 13735 tablet_bootstrap.cc:654] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:53.773576 13677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9bdd77e564429c9c0434b358f48302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.781306 13683 tablet_service.cc:1511] Processing CreateTablet for tablet 0863151795784affb539e2b7a55fc2b8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:55:53.778754 13734 ts_tablet_manager.cc:1434] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.004s
I20260709 12:55:53.779332 13644 heartbeater.cc:499] Master 127.12.30.126:40047 was elected leader, sending a full tablet report...
I20260709 12:55:53.782648 13683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0863151795784affb539e2b7a55fc2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.790988 13526 tablet_service.cc:1511] Processing CreateTablet for tablet f125a6a76f974409807014b9ddaa1b8c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:55:53.792493 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f125a6a76f974409807014b9ddaa1b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.795235 13734 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:53.799837 13602 tablet_service.cc:1511] Processing CreateTablet for tablet f125a6a76f974409807014b9ddaa1b8c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:55:53.801553 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f125a6a76f974409807014b9ddaa1b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.837281 13531 tablet_service.cc:1511] Processing CreateTablet for tablet 1c9f20cfde5343c9b91c4e2adc5320ce (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:55:53.838336 13734 tablet_bootstrap.cc:654] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:53.837314 13524 tablet_service.cc:1511] Processing CreateTablet for tablet 973866cfc22e48ea931cb1a6fb16872f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:55:53.832762 13530 tablet_service.cc:1511] Processing CreateTablet for tablet f2ce7a788b8c40629fc1921a4286d084 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:55:53.844291 13600 tablet_service.cc:1511] Processing CreateTablet for tablet f2ce7a788b8c40629fc1921a4286d084 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:55:53.845690 13600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ce7a788b8c40629fc1921a4286d084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.849138 13605 tablet_service.cc:1511] Processing CreateTablet for tablet 973866cfc22e48ea931cb1a6fb16872f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:55:53.849700 13733 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:53.850224 13733 ts_tablet_manager.cc:1403] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.223s	user 0.011s	sys 0.002s
I20260709 12:55:53.850948 13605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973866cfc22e48ea931cb1a6fb16872f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.854313 13733 raft_consensus.cc:359] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:53.855124 13733 raft_consensus.cc:385] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:53.855414 13733 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:53.855615 13609 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9bdd77e564429c9c0434b358f48302 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:55:53.863634 13738 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:53.864180 13738 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:53.865998 13735 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:53.866638 13735 ts_tablet_manager.cc:1403] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.170s	user 0.009s	sys 0.004s
I20260709 12:55:53.860385 13533 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9bdd77e564429c9c0434b358f48302 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:55:53.841980 13531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c9f20cfde5343c9b91c4e2adc5320ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.873658 13735 raft_consensus.cc:359] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:53.875351 13609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9bdd77e564429c9c0434b358f48302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.877475 13600 tablet_service.cc:1511] Processing CreateTablet for tablet 0863151795784affb539e2b7a55fc2b8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:55:53.877868 13681 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa892a8dfec433d8aa859d442e3e309 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:55:53.895324 13681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa892a8dfec433d8aa859d442e3e309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.895010 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 1a689534693d4469befa622d6b4917e3 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:55:53.896461 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a689534693d4469befa622d6b4917e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.884855 13735 raft_consensus.cc:385] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:53.900035 13738 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:53.900148 13735 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:53.907549 13605 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa892a8dfec433d8aa859d442e3e309 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:55:53.908849 13600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0863151795784affb539e2b7a55fc2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.913087 13735 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:53.917379 13735 ts_tablet_manager.cc:1434] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 12:55:53.919200 13735 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:53.918807 13532 tablet_service.cc:1511] Processing CreateTablet for tablet 0863151795784affb539e2b7a55fc2b8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:55:53.924152 13527 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa892a8dfec433d8aa859d442e3e309 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:55:53.927407 13740 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:53.928023 13740 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:53.935050 13530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ce7a788b8c40629fc1921a4286d084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.945926 13525 tablet_service.cc:1511] Processing CreateTablet for tablet 1a689534693d4469befa622d6b4917e3 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:55:53.919307 13608 tablet_service.cc:1511] Processing CreateTablet for tablet 1c9f20cfde5343c9b91c4e2adc5320ce (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:55:53.905392 13733 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:53.972500 13533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9bdd77e564429c9c0434b358f48302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.973425 13532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0863151795784affb539e2b7a55fc2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.974090 13527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa892a8dfec433d8aa859d442e3e309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.977695 13601 tablet_service.cc:1511] Processing CreateTablet for tablet 1a689534693d4469befa622d6b4917e3 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:55:53.984402 13738 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.004683 13738 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:53.965377 13605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa892a8dfec433d8aa859d442e3e309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:53.956888 13740 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.012297 13734 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:54.021243 13525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a689534693d4469befa622d6b4917e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.028870 13524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973866cfc22e48ea931cb1a6fb16872f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.032410 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a689534693d4469befa622d6b4917e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.049629 13734 ts_tablet_manager.cc:1403] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.255s	user 0.021s	sys 0.010s
I20260709 12:55:54.056800 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:54.057631 13619 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:54.058979 13526 tablet_service.cc:1511] Processing CreateTablet for tablet 6af1705dc4aa4e738a8416263f5af754 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:55:54.060256 13738 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:54.064183 13608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c9f20cfde5343c9b91c4e2adc5320ce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:55:54.067745 13427 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 45683.3us
I20260709 12:55:54.070900 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:54.070948 13733 ts_tablet_manager.cc:1434] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.220s	user 0.014s	sys 0.000s
I20260709 12:55:54.071640 13734 raft_consensus.cc:359] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.072754 13734 raft_consensus.cc:385] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.073190 13734 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
W20260709 12:55:54.073454 13509 leader_election.cc:343] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.075572 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:54.076114 13659 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:55:54.077144 13659 leader_election.cc:343] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.080232 13738 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:54.080643 13738 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:54.081023 13738 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:54.082924 13733 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:54.098564 13734 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.101351 13734 ts_tablet_manager.cc:1434] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.048s	user 0.010s	sys 0.000s
I20260709 12:55:54.103204 13734 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
W20260709 12:55:54.103596 13428 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 67252.4us
W20260709 12:55:54.106174 13425 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 44112us
I20260709 12:55:54.106844 13603 tablet_service.cc:1511] Processing CreateTablet for tablet 6af1705dc4aa4e738a8416263f5af754 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:55:54.108232 13603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6af1705dc4aa4e738a8416263f5af754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.110136 13735 tablet_bootstrap.cc:654] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.110987 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6af1705dc4aa4e738a8416263f5af754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.126068 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:54.127866 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:54.127929 13528 tablet_service.cc:1511] Processing CreateTablet for tablet be389b2236484464bc3e9f558d7b4230 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:55:54.130209 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be389b2236484464bc3e9f558d7b4230. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:55:54.132510 13658 leader_election.cc:343] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:54.133808 13658 leader_election.cc:343] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.134245 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.135722 13679 tablet_service.cc:1511] Processing CreateTablet for tablet be389b2236484464bc3e9f558d7b4230 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:55:54.137140 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be389b2236484464bc3e9f558d7b4230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.146560 13676 tablet_service.cc:1511] Processing CreateTablet for tablet 6af1705dc4aa4e738a8416263f5af754 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:55:54.154441 13753 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.156145 13604 tablet_service.cc:1511] Processing CreateTablet for tablet 6b48fb5973b84b06a9ad858a9b3967f8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:55:54.157694 13604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b48fb5973b84b06a9ad858a9b3967f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.160094 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6af1705dc4aa4e738a8416263f5af754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.168804 13607 tablet_service.cc:1511] Processing CreateTablet for tablet be389b2236484464bc3e9f558d7b4230 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:55:54.170279 13607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be389b2236484464bc3e9f558d7b4230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.168812 13529 tablet_service.cc:1511] Processing CreateTablet for tablet 6b48fb5973b84b06a9ad858a9b3967f8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:55:54.173899 13529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b48fb5973b84b06a9ad858a9b3967f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.200807 13734 tablet_bootstrap.cc:654] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.205729 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:54.206627 13735 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:54.207108 13735 ts_tablet_manager.cc:1403] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.288s	user 0.022s	sys 0.047s
W20260709 12:55:54.207460 13506 leader_election.cc:343] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.207962 13506 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.209087 13740 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.209599 13733 tablet_bootstrap.cc:654] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.210322 13735 raft_consensus.cc:359] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.211091 13735 raft_consensus.cc:385] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.211393 13735 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.216486 13738 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:54.216624 13735 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.217602 13740 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.218057 13740 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.219162 13738 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:54.219878 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:54.220126 13740 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.222906 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:54.223140 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:54.233348 13619 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:54.234562 13681 tablet_service.cc:1511] Processing CreateTablet for tablet 6b48fb5973b84b06a9ad858a9b3967f8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
W20260709 12:55:54.235329 13658 leader_election.cc:343] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.235886 13681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b48fb5973b84b06a9ad858a9b3967f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.223840 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:54.236760 13693 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
W20260709 12:55:54.238523 13509 leader_election.cc:343] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.239595 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.241336 13740 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.241811 13740 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.243772 13740 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.244247 13740 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:54.244568 13740 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:54.244899 13740 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:54.249663 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:54.250370 13531 tablet_service.cc:1511] Processing CreateTablet for tablet e7d1ab18f48a4689857964b7200b72c9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
W20260709 12:55:54.251344 13506 leader_election.cc:343] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.252082 13531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d1ab18f48a4689857964b7200b72c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.252602 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:54.254081 13509 leader_election.cc:343] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.254505 13509 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.264097 13757 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.267517 13679 tablet_service.cc:1511] Processing CreateTablet for tablet e7d1ab18f48a4689857964b7200b72c9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:55:54.268977 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d1ab18f48a4689857964b7200b72c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.272656 13605 tablet_service.cc:1511] Processing CreateTablet for tablet e7d1ab18f48a4689857964b7200b72c9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:55:54.274407 13605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d1ab18f48a4689857964b7200b72c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.288405 13619 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:54.288546 13601 tablet_service.cc:1511] Processing CreateTablet for tablet c827d49183b8406b9adff9666f6d83a9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:55:54.289618 13676 tablet_service.cc:1511] Processing CreateTablet for tablet c827d49183b8406b9adff9666f6d83a9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:55:54.290241 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c827d49183b8406b9adff9666f6d83a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.290140 13659 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:54.290767 13735 ts_tablet_manager.cc:1434] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.083s	user 0.012s	sys 0.016s
I20260709 12:55:54.291102 13738 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:54.291142 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c827d49183b8406b9adff9666f6d83a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.303651 13734 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:54.292342 13738 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:54.306419 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 85eb0ac09bf7466899637997aae42111 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:55:54.309494 13734 ts_tablet_manager.cc:1403] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.207s	user 0.003s	sys 0.015s
I20260709 12:55:54.310252 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85eb0ac09bf7466899637997aae42111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.309890 13738 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:54.311257 13530 tablet_service.cc:1511] Processing CreateTablet for tablet 9d739d3dda4e4370b33be67fca1c66d5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:55:54.312525 13530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d739d3dda4e4370b33be67fca1c66d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.295166 13533 tablet_service.cc:1511] Processing CreateTablet for tablet 85eb0ac09bf7466899637997aae42111 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:55:54.313516 13533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85eb0ac09bf7466899637997aae42111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.314373 13602 tablet_service.cc:1511] Processing CreateTablet for tablet 9d739d3dda4e4370b33be67fca1c66d5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:55:54.315685 13683 tablet_service.cc:1511] Processing CreateTablet for tablet 9d739d3dda4e4370b33be67fca1c66d5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:55:54.301748 13527 tablet_service.cc:1511] Processing CreateTablet for tablet c827d49183b8406b9adff9666f6d83a9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:55:54.316962 13683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d739d3dda4e4370b33be67fca1c66d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.317090 13527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c827d49183b8406b9adff9666f6d83a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.303524 13608 tablet_service.cc:1511] Processing CreateTablet for tablet 85eb0ac09bf7466899637997aae42111 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:55:54.315594 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d739d3dda4e4370b33be67fca1c66d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.324265 13608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85eb0ac09bf7466899637997aae42111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.330597 13734 raft_consensus.cc:359] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.291863 13735 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:54.295890 13740 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.353948 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:54.355340 13509 leader_election.cc:343] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.357362 13733 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:54.357576 13531 tablet_service.cc:1511] Processing CreateTablet for tablet 32b8156356cf4b709a2903d6861a117c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:55:54.357973 13733 ts_tablet_manager.cc:1403] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.275s	user 0.017s	sys 0.009s
I20260709 12:55:54.358901 13531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b8156356cf4b709a2903d6861a117c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.361124 13604 tablet_service.cc:1511] Processing CreateTablet for tablet 32b8156356cf4b709a2903d6861a117c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:55:54.361604 13733 raft_consensus.cc:359] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.362465 13604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b8156356cf4b709a2903d6861a117c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.362851 13734 raft_consensus.cc:385] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.363240 13734 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.367867 13680 tablet_service.cc:1511] Processing CreateTablet for tablet 32b8156356cf4b709a2903d6861a117c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:55:54.369244 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b8156356cf4b709a2903d6861a117c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.362545 13733 raft_consensus.cc:385] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.373585 13526 tablet_service.cc:1511] Processing CreateTablet for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:55:54.376959 13601 tablet_service.cc:1511] Processing CreateTablet for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:55:54.381234 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:54.381096 13679 tablet_service.cc:1511] Processing CreateTablet for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:55:54.397114 13693 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:54.381793 13740 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.381846 13533 tablet_service.cc:1511] Processing CreateTablet for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:55:54.397697 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.364177 13734 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.389037 13605 tablet_service.cc:1511] Processing CreateTablet for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:55:54.397717 13607 tablet_service.cc:1511] Processing CreateTablet for tablet 5c25c2050d4547c39924ceb739727ae8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:55:54.389734 13735 tablet_bootstrap.cc:654] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.392782 13681 tablet_service.cc:1511] Processing CreateTablet for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:55:54.393895 13528 tablet_service.cc:1511] Processing CreateTablet for tablet 5c25c2050d4547c39924ceb739727ae8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:55:54.405385 13440 catalog_manager.cc:5697] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:54.396529 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.400504 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.402403 13683 tablet_service.cc:1511] Processing CreateTablet for tablet 5c25c2050d4547c39924ceb739727ae8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:55:54.409544 13683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c25c2050d4547c39924ceb739727ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.409772 13529 tablet_service.cc:1511] Processing CreateTablet for tablet f9f9e2a62b8a498d93662b589c6a6870 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:55:54.411562 13529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f9e2a62b8a498d93662b589c6a6870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.412209 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c25c2050d4547c39924ceb739727ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.415249 13605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.405735 13734 ts_tablet_manager.cc:1434] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.096s	user 0.003s	sys 0.003s
I20260709 12:55:54.395540 13733 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.416733 13733 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.418956 13733 ts_tablet_manager.cc:1434] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.001s
I20260709 12:55:54.416827 13734 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:54.424237 13533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.432240 13607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c25c2050d4547c39924ceb739727ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.432016 13603 tablet_service.cc:1511] Processing CreateTablet for tablet f9f9e2a62b8a498d93662b589c6a6870 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:55:54.433355 13603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f9e2a62b8a498d93662b589c6a6870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.419878 13733 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:54.443125 13693 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:54.444451 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.445736 13740 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:54.446720 13740 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:54.447511 13740 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.449051 13733 tablet_bootstrap.cc:654] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.448741 13679 tablet_service.cc:1511] Processing CreateTablet for tablet f9f9e2a62b8a498d93662b589c6a6870 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:55:54.450104 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f9e2a62b8a498d93662b589c6a6870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.452792 13734 tablet_bootstrap.cc:654] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.469316 13676 tablet_service.cc:1511] Processing CreateTablet for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:55:54.470646 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.472227 13741 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.472362 13681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.472729 13741 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.481995 13530 tablet_service.cc:1511] Processing CreateTablet for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:55:54.484643 13530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.486479 13529 tablet_service.cc:1511] Processing CreateTablet for tablet 3f54dde33fa845098e64673b576d780c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:55:54.486945 13608 tablet_service.cc:1511] Processing CreateTablet for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:55:54.487754 13529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f54dde33fa845098e64673b576d780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.488147 13608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.478322 13741 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.525763 13527 tablet_service.cc:1511] Processing CreateTablet for tablet fa97d994ab11478b9f735ced13bc6d48 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:55:54.527190 13527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa97d994ab11478b9f735ced13bc6d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.542737 13677 tablet_service.cc:1511] Processing CreateTablet for tablet 3f54dde33fa845098e64673b576d780c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:55:54.544091 13677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f54dde33fa845098e64673b576d780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.547127 13602 tablet_service.cc:1511] Processing CreateTablet for tablet fa97d994ab11478b9f735ced13bc6d48 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:55:54.547995 13741 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.548484 13741 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.550690 13741 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.552232 13735 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:54.552716 13735 ts_tablet_manager.cc:1403] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.261s	user 0.019s	sys 0.007s
I20260709 12:55:54.553571 13733 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:54.554035 13733 ts_tablet_manager.cc:1403] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.134s	user 0.012s	sys 0.035s
I20260709 12:55:54.555253 13604 tablet_service.cc:1511] Processing CreateTablet for tablet 3f54dde33fa845098e64673b576d780c (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:55:54.558568 13528 tablet_service.cc:1511] Processing CreateTablet for tablet bcd2fb69161943208c2aef45acb3c33d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:55:54.559870 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd2fb69161943208c2aef45acb3c33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.560243 13674 tablet_service.cc:1511] Processing CreateTablet for tablet 019c73868e744f4eba4ac7d97d34a137 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:55:54.561472 13674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 019c73868e744f4eba4ac7d97d34a137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.566092 13680 tablet_service.cc:1511] Processing CreateTablet for tablet fa97d994ab11478b9f735ced13bc6d48 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:55:54.567518 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa97d994ab11478b9f735ced13bc6d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.573779 13683 tablet_service.cc:1511] Processing CreateTablet for tablet bcd2fb69161943208c2aef45acb3c33d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 12:55:54.574734 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:54.575148 13683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd2fb69161943208c2aef45acb3c33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.574817 13526 tablet_service.cc:1511] Processing CreateTablet for tablet d9eb8a2c5d7846099d30c0839aefc5a9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:55:54.575332 13605 tablet_service.cc:1511] Processing CreateTablet for tablet 019c73868e744f4eba4ac7d97d34a137 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:55:54.576009 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa97d994ab11478b9f735ced13bc6d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.577700 13604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f54dde33fa845098e64673b576d780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.573975 13601 tablet_service.cc:1511] Processing CreateTablet for tablet bcd2fb69161943208c2aef45acb3c33d (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:55:54.576083 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9eb8a2c5d7846099d30c0839aefc5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.586114 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd2fb69161943208c2aef45acb3c33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.592415 13605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 019c73868e744f4eba4ac7d97d34a137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.593647 13607 tablet_service.cc:1511] Processing CreateTablet for tablet d9eb8a2c5d7846099d30c0839aefc5a9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:55:54.595052 13607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9eb8a2c5d7846099d30c0839aefc5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.547155 13533 tablet_service.cc:1511] Processing CreateTablet for tablet 019c73868e744f4eba4ac7d97d34a137 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:55:54.604514 13530 tablet_service.cc:1511] Processing CreateTablet for tablet 56f36011554a4cd78d7b17b92a823c8f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:55:54.587657 13733 raft_consensus.cc:359] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.608316 13733 raft_consensus.cc:385] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.608637 13733 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.609427 13733 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.555564 13735 raft_consensus.cc:359] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.601280 13533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 019c73868e744f4eba4ac7d97d34a137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.612648 13735 raft_consensus.cc:385] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.613024 13735 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.616199 13530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f36011554a4cd78d7b17b92a823c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.618129 13676 tablet_service.cc:1511] Processing CreateTablet for tablet 56f36011554a4cd78d7b17b92a823c8f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:55:54.619498 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f36011554a4cd78d7b17b92a823c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.611879 13733 ts_tablet_manager.cc:1434] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
I20260709 12:55:54.620962 13733 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:54.627198 13733 tablet_bootstrap.cc:654] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.629328 13608 tablet_service.cc:1511] Processing CreateTablet for tablet 56f36011554a4cd78d7b17b92a823c8f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:55:54.630669 13608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f36011554a4cd78d7b17b92a823c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.632318 13680 tablet_service.cc:1511] Processing CreateTablet for tablet d9eb8a2c5d7846099d30c0839aefc5a9 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:55:54.633965 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9eb8a2c5d7846099d30c0839aefc5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.613777 13735 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.660003 13735 ts_tablet_manager.cc:1434] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.107s	user 0.009s	sys 0.000s
I20260709 12:55:54.661108 13735 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:54.672405 13738 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.672698 13531 tablet_service.cc:1511] Processing CreateTablet for tablet 048528e413ac4c67a1cfd3dfe70673c7 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:55:54.673141 13738 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.674008 13531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 048528e413ac4c67a1cfd3dfe70673c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.676916 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:54.680438 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:54.687668 13601 tablet_service.cc:1511] Processing CreateTablet for tablet 048528e413ac4c67a1cfd3dfe70673c7 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:55:54.688998 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 048528e413ac4c67a1cfd3dfe70673c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.690745 13439 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:54.695149 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:54.692281 13659 leader_election.cc:343] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:54.696055 13658 leader_election.cc:343] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:54.697721 13584 leader_election.cc:343] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.698360 13658 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.698562 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:54.699263 13753 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:55:54.700165 13584 leader_election.cc:343] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.701165 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:54.681939 13734 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:54.704838 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:54.711674 13734 ts_tablet_manager.cc:1403] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.295s	user 0.023s	sys 0.032s
I20260709 12:55:54.711500 13676 tablet_service.cc:1511] Processing CreateTablet for tablet 048528e413ac4c67a1cfd3dfe70673c7 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:55:54.712842 13676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 048528e413ac4c67a1cfd3dfe70673c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.713474 13529 tablet_service.cc:1511] Processing CreateTablet for tablet 86d7b5c530f14daaade417c2e3903072 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 12:55:54.710672 13582 leader_election.cc:343] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.714692 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:54.715466 13741 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.714707 13734 raft_consensus.cc:359] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.716481 13529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d7b5c530f14daaade417c2e3903072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.716813 13734 raft_consensus.cc:385] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.717275 13734 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.682267 13738 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:55:54.740705 13582 leader_election.cc:343] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.741417 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:54.745639 13771 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.748792 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 86d7b5c530f14daaade417c2e3903072 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:55:54.750551 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d7b5c530f14daaade417c2e3903072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.755447 13740 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.756007 13740 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.756735 13601 tablet_service.cc:1511] Processing CreateTablet for tablet 86d7b5c530f14daaade417c2e3903072 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:55:54.758104 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d7b5c530f14daaade417c2e3903072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.758177 13740 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.759502 13738 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:54.762966 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:55:54.764624 13506 leader_election.cc:343] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:54.784765 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:54.718245 13734 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.795194 13525 tablet_service.cc:1511] Processing CreateTablet for tablet 1c2d2db0b5674796bbd0426953d99538 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:55:54.797024 13602 tablet_service.cc:1511] Processing CreateTablet for tablet 1c2d2db0b5674796bbd0426953d99538 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:55:54.802490 13735 tablet_bootstrap.cc:654] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.798421 13741 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.803323 13525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2d2db0b5674796bbd0426953d99538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.803112 13741 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.805781 13741 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:54.807421 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:54.810511 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2d2db0b5674796bbd0426953d99538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.801355 13733 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:54.813885 13733 ts_tablet_manager.cc:1403] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.193s	user 0.031s	sys 0.056s
I20260709 12:55:54.797497 13679 tablet_service.cc:1511] Processing CreateTablet for tablet 1c2d2db0b5674796bbd0426953d99538 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:55:54.818413 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2d2db0b5674796bbd0426953d99538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.824949 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:54.828985 13733 raft_consensus.cc:359] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.829780 13733 raft_consensus.cc:385] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.836941 13528 tablet_service.cc:1511] Processing CreateTablet for tablet 75988f1a2dda4bd6962cd5f3ad1c304a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:55:54.838500 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75988f1a2dda4bd6962cd5f3ad1c304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.834262 13733 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.839388 13733 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.841729 13733 ts_tablet_manager.cc:1434] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.028s	user 0.001s	sys 0.005s
I20260709 12:55:54.849278 13607 tablet_service.cc:1511] Processing CreateTablet for tablet 75988f1a2dda4bd6962cd5f3ad1c304a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260709 12:55:54.860481 13509 leader_election.cc:343] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.860962 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.862159 13735 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:54.862097 13740 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.862885 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:54.863672 13693 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:55:54.864972 13582 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:54.865079 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 75988f1a2dda4bd6962cd5f3ad1c304a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:55:54.865751 13741 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:54.866647 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75988f1a2dda4bd6962cd5f3ad1c304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.867174 13735 ts_tablet_manager.cc:1403] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.206s	user 0.018s	sys 0.008s
I20260709 12:55:54.872972 13601 tablet_service.cc:1511] Processing CreateTablet for tablet 1d73171142594388972ed22eae44f95f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:55:54.870055 13734 ts_tablet_manager.cc:1434] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.158s	user 0.006s	sys 0.003s
I20260709 12:55:54.871956 13526 tablet_service.cc:1511] Processing CreateTablet for tablet 1d73171142594388972ed22eae44f95f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:55:54.873821 13741 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.874501 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d73171142594388972ed22eae44f95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.875651 13733 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:54.876698 13734 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:54.868095 13607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75988f1a2dda4bd6962cd5f3ad1c304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.883419 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d73171142594388972ed22eae44f95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.893857 13529 tablet_service.cc:1511] Processing CreateTablet for tablet e634cc18febe4680bb70a3673cabbae4 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:55:54.894989 13679 tablet_service.cc:1511] Processing CreateTablet for tablet 1d73171142594388972ed22eae44f95f (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:55:54.905099 13733 tablet_bootstrap.cc:654] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.906138 13601 tablet_service.cc:1511] Processing CreateTablet for tablet e634cc18febe4680bb70a3673cabbae4 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:55:54.907429 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e634cc18febe4680bb70a3673cabbae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.912892 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d73171142594388972ed22eae44f95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.915788 13734 tablet_bootstrap.cc:654] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:54.919320 13740 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:54.923202 13529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e634cc18febe4680bb70a3673cabbae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.888420 13735 raft_consensus.cc:359] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.929610 13735 raft_consensus.cc:385] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.929930 13735 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.934053 13683 tablet_service.cc:1511] Processing CreateTablet for tablet e634cc18febe4680bb70a3673cabbae4 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:55:54.941094 13683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e634cc18febe4680bb70a3673cabbae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.945711 13680 tablet_service.cc:1511] Processing CreateTablet for tablet f97a6462cd4341ddbd7058ce9537c1eb (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:55:54.939677 13526 tablet_service.cc:1511] Processing CreateTablet for tablet f97a6462cd4341ddbd7058ce9537c1eb (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:55:54.947008 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97a6462cd4341ddbd7058ce9537c1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.947105 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97a6462cd4341ddbd7058ce9537c1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.930949 13735 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.956549 13733 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:54.957213 13733 ts_tablet_manager.cc:1403] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.002s
I20260709 12:55:54.957351 13734 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:54.957834 13734 ts_tablet_manager.cc:1403] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.081s	user 0.006s	sys 0.006s
I20260709 12:55:54.960268 13733 raft_consensus.cc:359] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.961040 13733 raft_consensus.cc:385] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.955734 13601 tablet_service.cc:1511] Processing CreateTablet for tablet 835fe463242d40e5b82ddb5898323fc5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:55:54.960695 13734 raft_consensus.cc:359] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.961503 13734 raft_consensus.cc:385] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:54.961808 13734 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.962008 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835fe463242d40e5b82ddb5898323fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.962654 13734 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.955075 13528 tablet_service.cc:1511] Processing CreateTablet for tablet 835fe463242d40e5b82ddb5898323fc5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:55:54.964706 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835fe463242d40e5b82ddb5898323fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.945034 13602 tablet_service.cc:1511] Processing CreateTablet for tablet f97a6462cd4341ddbd7058ce9537c1eb (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:55:54.967470 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97a6462cd4341ddbd7058ce9537c1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.969689 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 835fe463242d40e5b82ddb5898323fc5 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:55:54.971043 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835fe463242d40e5b82ddb5898323fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:54.971436 13738 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:54.972193 13738 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.975478 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:54.977314 13658 leader_election.cc:343] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.977869 13738 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:54.978188 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:54.979713 13659 leader_election.cc:343] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:54.980011 13733 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:54.980284 13659 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:54.981112 13733 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:54.982185 13753 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:54.983139 13733 ts_tablet_manager.cc:1434] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.026s	user 0.000s	sys 0.004s
I20260709 12:55:54.984967 13734 ts_tablet_manager.cc:1434] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:55:54.985967 13734 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:54.995522 13735 ts_tablet_manager.cc:1434] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.123s	user 0.006s	sys 0.001s
I20260709 12:55:54.996610 13735 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:55.003744 13740 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:55.006810 13525 tablet_service.cc:1511] Processing CreateTablet for tablet 3f7e1f6a8119482c9f8056bb615edfa8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:55:54.992166 13733 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.008358 13525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7e1f6a8119482c9f8056bb615edfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.024307 13735 tablet_bootstrap.cc:654] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.025516 13733 tablet_bootstrap.cc:654] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.029644 13734 tablet_bootstrap.cc:654] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.033648 13602 tablet_service.cc:1511] Processing CreateTablet for tablet 3f7e1f6a8119482c9f8056bb615edfa8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:55:55.037771 13602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7e1f6a8119482c9f8056bb615edfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.041633 13741 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.042565 13741 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.048163 13741 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.052477 13526 tablet_service.cc:1511] Processing CreateTablet for tablet 921257a2480244e88f3ff036fb3be8d2 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:55:55.053758 13526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 921257a2480244e88f3ff036fb3be8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.054467 13753 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.055562 13753 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.056195 13682 tablet_service.cc:1511] Processing CreateTablet for tablet 3f7e1f6a8119482c9f8056bb615edfa8 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:55:55.057546 13682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7e1f6a8119482c9f8056bb615edfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.059006 13738 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.059684 13738 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.060384 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.060676 13733 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.061132 13733 ts_tablet_manager.cc:1403] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.003s
I20260709 12:55:55.061009 13542 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:55.060951 13680 tablet_service.cc:1511] Processing CreateTablet for tablet 921257a2480244e88f3ff036fb3be8d2 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:55:55.061872 13738 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.062306 13680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 921257a2480244e88f3ff036fb3be8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.063553 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:55.063696 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.064436 13619 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:55.065346 13607 tablet_service.cc:1511] Processing CreateTablet for tablet 921257a2480244e88f3ff036fb3be8d2 (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:55:55.066613 13607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 921257a2480244e88f3ff036fb3be8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.066994 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:55.067708 13693 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:55.069576 13601 tablet_service.cc:1511] Processing CreateTablet for tablet c665edf3d7834e18a67df3f75b30856a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:55:55.070717 13601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c665edf3d7834e18a67df3f75b30856a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.070617 13528 tablet_service.cc:1511] Processing CreateTablet for tablet c665edf3d7834e18a67df3f75b30856a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:55:55.072134 13741 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.072348 13528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c665edf3d7834e18a67df3f75b30856a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.072516 13741 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.072976 13741 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.075004 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.077831 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.078722 13542 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
W20260709 12:55:55.080488 13658 leader_election.cc:343] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.081362 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:55.083261 13659 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:55.084585 13738 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.085000 13738 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.085453 13738 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.087018 13679 tablet_service.cc:1511] Processing CreateTablet for tablet c665edf3d7834e18a67df3f75b30856a (DEFAULT_TABLE table=test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:55:55.088202 13679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c665edf3d7834e18a67df3f75b30856a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:55:55.088213 13753 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.092564 13753 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.092897 13753 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.093218 13753 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.064232 13733 raft_consensus.cc:359] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.104820 13733 raft_consensus.cc:385] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:55:55.127274 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 44161.5us
I20260709 12:55:55.131552 13738 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.133728 13738 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.125804 13733 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.126389 13741 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.139559 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:55.140173 13542 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.141041 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:55.141913 13617 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 1: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:55:55.142160 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:55.141021 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.143046 13619 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 1: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:55:55.147101 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:55.147486 13542 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:55.147982 13693 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:55:55.148929 13741 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.149127 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:55.150190 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:55.151190 13741 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:55:55.151867 13733 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.153460 13738 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:55.154152 13738 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:55.160073 13733 ts_tablet_manager.cc:1434] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.099s	user 0.009s	sys 0.000s
I20260709 12:55:55.161036 13733 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.155290 13738 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.166930 13753 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.170567 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:55.172075 13734 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
W20260709 12:55:55.172042 13658 leader_election.cc:343] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.176680 13735 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:55.177093 13734 ts_tablet_manager.cc:1403] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.191s	user 0.015s	sys 0.007s
I20260709 12:55:55.179571 13733 tablet_bootstrap.cc:654] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.180269 13734 raft_consensus.cc:359] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.180626 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:55.181005 13734 raft_consensus.cc:385] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.181177 13618 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.181344 13734 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.177173 13735 ts_tablet_manager.cc:1403] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.181s	user 0.024s	sys 0.022s
I20260709 12:55:55.185981 13735 raft_consensus.cc:359] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.186748 13735 raft_consensus.cc:385] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.187047 13735 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.189579 13735 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.191869 13735 ts_tablet_manager.cc:1434] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.001s
I20260709 12:55:55.192916 13735 tablet_bootstrap.cc:492] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:55.193794 13618 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:55.195313 13659 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:55.196446 13782 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:55.203076 13753 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.206027 13782 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:55.206938 13735 tablet_bootstrap.cc:654] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.207305 13782 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.210630 13734 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.218647 13780 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.219386 13780 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.220659 13734 ts_tablet_manager.cc:1434] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:55:55.221758 13734 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:55.223976 13780 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.224701 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:55.225569 13693 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:55.226437 13439 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:55.227344 13506 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:55.228885 13780 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.229369 13780 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.229801 13780 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.234266 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:55.235617 13509 leader_election.cc:343] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.241918 13733 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.242512 13733 ts_tablet_manager.cc:1403] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.082s	user 0.029s	sys 0.030s
I20260709 12:55:55.243367 13780 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.244223 13734 tablet_bootstrap.cc:654] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.245494 13733 raft_consensus.cc:359] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.246217 13733 raft_consensus.cc:385] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.246559 13733 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.247591 13733 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.248831 13735 tablet_bootstrap.cc:492] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:55.249317 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:55.249976 13733 ts_tablet_manager.cc:1434] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:55:55.250897 13733 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
W20260709 12:55:55.250752 13509 leader_election.cc:343] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.249341 13735 ts_tablet_manager.cc:1403] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.019s
I20260709 12:55:55.254242 13735 raft_consensus.cc:359] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:55.255158 13735 raft_consensus.cc:385] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.255465 13735 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.255856 13734 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:55.256371 13734 ts_tablet_manager.cc:1403] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.002s
I20260709 12:55:55.257831 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:55.258124 13733 tablet_bootstrap.cc:654] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.258466 13693 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.259334 13734 raft_consensus.cc:359] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.260301 13734 raft_consensus.cc:385] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.260604 13734 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.261389 13734 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.263640 13734 ts_tablet_manager.cc:1434] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:55:55.264886 13734 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:55.267149 13780 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.268545 13693 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:55.268482 13438 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:55.270059 13506 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:55.271193 13780 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:55.272106 13734 tablet_bootstrap.cc:654] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.272375 13780 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:55.273288 13780 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.276170 13735 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:55.278970 13735 ts_tablet_manager.cc:1434] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:55:55.279174 13733 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.279706 13733 ts_tablet_manager.cc:1403] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.010s
I20260709 12:55:55.279989 13735 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:55.282493 13733 raft_consensus.cc:359] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.283326 13733 raft_consensus.cc:385] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.283661 13733 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.284708 13733 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.286979 13733 ts_tablet_manager.cc:1434] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:55:55.287869 13733 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.294209 13733 tablet_bootstrap.cc:654] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.298980 13735 tablet_bootstrap.cc:654] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.300210 13440 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:55.310593 13734 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:55.311242 13734 ts_tablet_manager.cc:1403] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.015s
I20260709 12:55:55.314826 13734 raft_consensus.cc:359] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.315733 13734 raft_consensus.cc:385] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.316874 13734 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.317595 13734 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.318899 13733 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.319340 13733 ts_tablet_manager.cc:1403] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.011s
I20260709 12:55:55.322064 13733 raft_consensus.cc:359] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.322697 13733 raft_consensus.cc:385] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.322939 13733 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.323208 13734 ts_tablet_manager.cc:1434] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:55:55.323652 13733 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.324180 13734 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:55.325645 13733 ts_tablet_manager.cc:1434] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:55:55.326782 13733 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.331869 13735 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:55.332427 13735 ts_tablet_manager.cc:1403] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.053s	user 0.022s	sys 0.014s
I20260709 12:55:55.334017 13734 tablet_bootstrap.cc:654] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.334645 13735 raft_consensus.cc:359] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.335199 13735 raft_consensus.cc:385] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.335502 13735 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.336274 13735 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.340068 13782 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.341135 13733 tablet_bootstrap.cc:654] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.340710 13782 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.344432 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.345284 13542 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:55.346796 13735 ts_tablet_manager.cc:1434] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.004s
I20260709 12:55:55.346900 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:55.348232 13738 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.348744 13738 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.349182 13738 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.351414 13735 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:55.356145 13782 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.357137 13733 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.357646 13733 ts_tablet_manager.cc:1403] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.001s
I20260709 12:55:55.357805 13735 tablet_bootstrap.cc:654] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.358248 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.359108 13618 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:55.360775 13733 raft_consensus.cc:359] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.361487 13733 raft_consensus.cc:385] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.361827 13733 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.362528 13733 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.363962 13738 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.364719 13733 ts_tablet_manager.cc:1434] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:55:55.365665 13733 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.366078 13738 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.367165 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:55.367959 13542 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.372427 13733 tablet_bootstrap.cc:654] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.373876 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:55.374629 13618 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.382517 13733 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.383092 13733 ts_tablet_manager.cc:1403] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 12:55:55.385862 13733 raft_consensus.cc:359] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.386646 13733 raft_consensus.cc:385] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.386984 13733 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.388115 13733 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.390283 13733 ts_tablet_manager.cc:1434] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:55:55.391418 13733 tablet_bootstrap.cc:492] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.396687 13733 tablet_bootstrap.cc:654] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.400718 13733 tablet_bootstrap.cc:492] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.401119 13733 ts_tablet_manager.cc:1403] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:55:55.402922 13733 raft_consensus.cc:359] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.403420 13733 raft_consensus.cc:385] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.403636 13733 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.404309 13733 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.407423 13733 ts_tablet_manager.cc:1434] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:55:55.408654 13733 tablet_bootstrap.cc:492] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.410917 13542 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:55.417047 13733 tablet_bootstrap.cc:654] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.421442 13618 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:55.423154 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:55.424286 13738 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:55.424929 13738 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:55.426040 13738 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.430557 13733 tablet_bootstrap.cc:492] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.431161 13733 ts_tablet_manager.cc:1403] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 12:55:55.434801 13733 raft_consensus.cc:359] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.435727 13733 raft_consensus.cc:385] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.436224 13733 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.437460 13733 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.440382 13733 ts_tablet_manager.cc:1434] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 12:55:55.441673 13733 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.450680 13733 tablet_bootstrap.cc:654] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.457255 13439 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:55.464805 13734 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:55.465413 13734 ts_tablet_manager.cc:1403] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.141s	user 0.041s	sys 0.052s
I20260709 12:55:55.467139 13735 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:55.467667 13735 ts_tablet_manager.cc:1403] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.116s	user 0.033s	sys 0.055s
I20260709 12:55:55.467959 13733 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.468400 13733 ts_tablet_manager.cc:1403] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.004s
I20260709 12:55:55.468425 13734 raft_consensus.cc:359] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.469236 13734 raft_consensus.cc:385] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.469540 13734 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.470326 13734 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.470430 13735 raft_consensus.cc:359] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.471468 13735 raft_consensus.cc:385] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.471004 13733 raft_consensus.cc:359] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.472064 13735 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.472280 13733 raft_consensus.cc:385] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.472649 13733 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.473012 13734 ts_tablet_manager.cc:1434] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:55:55.473510 13733 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.474859 13734 tablet_bootstrap.cc:492] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:55.475620 13733 ts_tablet_manager.cc:1434] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:55:55.476670 13733 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.473038 13735 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.481722 13734 tablet_bootstrap.cc:654] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.482486 13733 tablet_bootstrap.cc:654] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.487749 13735 ts_tablet_manager.cc:1434] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:55:55.493767 13738 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.494518 13738 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.498063 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.498281 13738 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.499118 13735 tablet_bootstrap.cc:492] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:55.499577 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:55.500689 13658 leader_election.cc:343] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:55.501752 13659 leader_election.cc:343] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.502188 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:55.503278 13738 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:55.507280 13735 tablet_bootstrap.cc:654] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.511312 13734 tablet_bootstrap.cc:492] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:55.511765 13734 ts_tablet_manager.cc:1403] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.015s
I20260709 12:55:55.513134 13733 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.513729 13733 ts_tablet_manager.cc:1403] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.016s
I20260709 12:55:55.514077 13741 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.514510 13741 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.516263 13741 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.517409 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.517822 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:55.514026 13734 raft_consensus.cc:359] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.518827 13542 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:55.518980 13734 raft_consensus.cc:385] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.519472 13734 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.517287 13733 raft_consensus.cc:359] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.519973 13733 raft_consensus.cc:385] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.520164 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:55.520361 13733 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.520328 13734 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.521126 13741 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.521493 13741 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.521920 13741 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.521540 13733 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.522773 13693 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:55.528519 13735 tablet_bootstrap.cc:492] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:55.528559 13733 ts_tablet_manager.cc:1434] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:55:55.529098 13735 ts_tablet_manager.cc:1403] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.008s
I20260709 12:55:55.529657 13733 tablet_bootstrap.cc:492] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:55.531694 13735 raft_consensus.cc:359] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.532392 13735 raft_consensus.cc:385] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.532684 13735 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.533445 13735 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.536190 13734 ts_tablet_manager.cc:1434] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.008s
I20260709 12:55:55.537583 13734 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:55.539510 13733 tablet_bootstrap.cc:654] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.543488 13734 tablet_bootstrap.cc:654] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.545569 13738 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.546036 13738 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.546933 13735 ts_tablet_manager.cc:1434] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:55:55.554250 13738 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.558636 13735 tablet_bootstrap.cc:492] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:55.556483 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.565213 13618 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:55.567423 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:55.569577 13735 tablet_bootstrap.cc:654] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.573205 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.574113 13542 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:55.577950 13738 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.578378 13738 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.578716 13738 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:55.579422 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:55.584049 13788 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.584717 13788 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.590545 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.594277 13735 tablet_bootstrap.cc:492] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:55.594921 13735 ts_tablet_manager.cc:1403] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.010s
I20260709 12:55:55.595106 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:55.596602 13658 leader_election.cc:343] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.597476 13782 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.598099 13782 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.598654 13735 raft_consensus.cc:359] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.599575 13735 raft_consensus.cc:385] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:55.599958 13735 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:55.601006 13735 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.601550 13782 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.604017 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:55.605854 13658 leader_election.cc:343] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.587544 13788 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.608968 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.610980 13735 ts_tablet_manager.cc:1434] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:55:55.612231 13735 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:55.613315 13543 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:55.607753 13659 leader_election.cc:343] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.587554 13542 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.640528 13659 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:55:55.645411 13659 leader_election.cc:343] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.647190 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:55.619967 13809 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.648232 13782 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:55.648595 13809 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.641703 13735 tablet_bootstrap.cc:654] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:55.653136 13809 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.655084 13782 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.655419 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.656921 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:55.655440 13741 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.657832 13693 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:55.660241 13582 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:55:55.662490 13584 leader_election.cc:343] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.660486 13741 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.663534 13741 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.664834 13741 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.665071 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:55.646050 13788 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:55.666064 13541 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.666824 13741 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.655781 13782 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.680291 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:55.681193 13693 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.684168 13788 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.684957 13788 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.692044 13782 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.693672 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.694959 13618 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:55.696884 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:55.697108 13741 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.700268 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.701233 13540 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:55.703269 13753 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.703730 13753 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.704226 13753 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.704528 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:55:55.706820 13584 leader_election.cc:343] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.709123 13741 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.722328 13753 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.733134 13782 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:55.743122 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:55.746340 13509 consensus_peers.cc:597] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:55.753298 13814 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.753857 13814 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.757320 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.757468 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:55.758080 13619 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
W20260709 12:55:55.758904 13506 leader_election.cc:343] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.759451 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:55:55.756181 13814 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.761004 13814 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:55.761349 13814 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:55.761683 13814 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.773010 13782 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:55.774329 13814 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.778514 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:55.781875 13659 leader_election.cc:343] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.785986 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:55.786614 13693 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
W20260709 12:55:55.787436 13506 leader_election.cc:343] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.788296 13815 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:55.789727 13814 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.830430 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:55.831558 13741 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:55.832163 13741 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:55:55.833201 13741 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:55.853619 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 94329.7us
I20260709 12:55:55.854398 13541 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:55.856189 13788 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.857234 13788 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.857710 13788 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.858029 13740 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.858526 13740 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.856137 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:55.859622 13618 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.863476 13740 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:55.864102 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:55.868494 13788 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:55:55.868959 13658 leader_election.cc:343] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.868363 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.869444 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:55.870970 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.871907 13816 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:55:55.873190 13659 leader_election.cc:343] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.873385 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:55.874435 13740 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:55:55.875452 13738 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.877995 13738 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.879261 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:55.880048 13615 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.882333 13753 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:55.885535 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:55.886273 13693 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.893141 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:55.894047 13540 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.896950 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:55.896296 13782 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:55:55.903097 13539 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:55.906131 13740 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:55.899710 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:55.898002 13615 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:55.933184 13539 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:55.912170 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:55.951177 13617 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:55.945508 13658 leader_election.cc:343] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:55.947134 13540 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:55.962783 13738 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:55.964169 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:55.981514 13780 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:55:55.982290 13780 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:55:55.983994 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:55.984979 13782 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:55.985697 13782 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:55.987373 13618 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:55.987980 13782 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.988615 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:55:55.989831 13740 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:55.990334 13740 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:55.991181 13740 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:55.996709 13734 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:55.997243 13734 ts_tablet_manager.cc:1403] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.460s	user 0.138s	sys 0.204s
I20260709 12:55:55.997942 13733 tablet_bootstrap.cc:492] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:55.998483 13733 ts_tablet_manager.cc:1403] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.469s	user 0.099s	sys 0.159s
I20260709 12:55:56.000314 13753 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.001933 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.007357 13735 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:56.008051 13735 ts_tablet_manager.cc:1403] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.396s	user 0.085s	sys 0.192s
I20260709 12:55:55.974406 13738 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.014371 13817 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.014890 13817 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.017201 13817 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.023384 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:55.963754 13439 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
W20260709 12:55:56.024803 13509 leader_election.cc:343] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.026295 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.029277 13735 raft_consensus.cc:359] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.029819 13780 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:56.029989 13735 raft_consensus.cc:385] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.030292 13735 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.030827 13753 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:56.030925 13735 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.033128 13735 ts_tablet_manager.cc:1434] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 12:55:56.033181 13738 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:56.034142 13735 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
W20260709 12:55:56.010313 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 69598us
I20260709 12:55:56.043473 13734 raft_consensus.cc:359] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.044405 13734 raft_consensus.cc:385] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.044850 13734 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.045796 13734 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:56.046057 13506 leader_election.cc:343] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.046715 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:55:56.046108 13821 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:55:56.047968 13814 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.048966 13693 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:56.055397 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.057325 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:55:56.060397 13659 leader_election.cc:343] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:56.056596 13823 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:55:56.061720 13782 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.064724 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.064796 13782 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:56.070664 13617 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:56.073450 13733 raft_consensus.cc:359] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.074308 13733 raft_consensus.cc:385] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.074631 13733 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.075757 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.077205 13740 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:55:56.075547 13733 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.079752 13782 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.088949 13734 ts_tablet_manager.cc:1434] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.091s	user 0.012s	sys 0.000s
I20260709 12:55:56.089857 13733 ts_tablet_manager.cc:1434] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.010s
I20260709 12:55:56.091159 13733 tablet_bootstrap.cc:492] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:56.095381 13734 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:56.112565 13582 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:56.113044 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.113739 13741 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.114171 13741 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:55:56.116158 13741 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.117561 13735 tablet_bootstrap.cc:654] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.122138 13738 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:55:56.126009 13658 leader_election.cc:343] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.126737 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.127732 13738 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.128669 13735 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:56.129197 13735 ts_tablet_manager.cc:1403] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.095s	user 0.010s	sys 0.006s
I20260709 12:55:56.131749 13734 tablet_bootstrap.cc:654] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.132612 13733 tablet_bootstrap.cc:654] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.144032 13735 raft_consensus.cc:359] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.144815 13738 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.144968 13735 raft_consensus.cc:385] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.145262 13735 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.145290 13738 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.145787 13440 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:56.150405 13735 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.154364 13735 ts_tablet_manager.cc:1434] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:55:56.160712 13735 tablet_bootstrap.cc:492] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:56.168867 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.177479 13741 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.178062 13741 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.179056 13734 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:56.179682 13734 ts_tablet_manager.cc:1403] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.090s	user 0.008s	sys 0.013s
W20260709 12:55:56.182797 13659 leader_election.cc:343] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.182888 13734 raft_consensus.cc:359] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.183780 13734 raft_consensus.cc:385] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.183954 13741 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.184144 13734 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.184917 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.185007 13734 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.187299 13734 ts_tablet_manager.cc:1434] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:55:56.188614 13734 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:56.161146 13738 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:56.198769 13440 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:56.201273 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.200953 13440 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:56.201951 13542 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:56.201309 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.204356 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:56.204358 13733 tablet_bootstrap.cc:492] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:56.205382 13741 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.207973 13733 ts_tablet_manager.cc:1403] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.117s	user 0.018s	sys 0.033s
I20260709 12:55:56.205040 13780 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.208725 13780 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.211063 13733 raft_consensus.cc:359] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.211879 13733 raft_consensus.cc:385] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.212252 13733 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.212795 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.213161 13780 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.212896 13733 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.213450 13692 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:56.214548 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.215389 13733 ts_tablet_manager.cc:1434] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 12:55:56.216418 13509 leader_election.cc:343] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.216379 13735 tablet_bootstrap.cc:654] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.206050 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.217315 13734 tablet_bootstrap.cc:654] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.217175 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.218485 13780 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:56.218904 13780 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:56.219226 13780 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.216409 13733 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:56.225713 13733 tablet_bootstrap.cc:654] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.226596 13780 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.227633 13738 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:56.228103 13738 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:56.228581 13738 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.230603 13735 tablet_bootstrap.cc:492] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:56.230487 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:56.231115 13735 ts_tablet_manager.cc:1403] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.076s	user 0.008s	sys 0.004s
W20260709 12:55:56.232198 13509 leader_election.cc:343] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.233321 13780 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.234074 13735 raft_consensus.cc:359] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.234752 13735 raft_consensus.cc:385] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.234768 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:56.235143 13735 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.235512 13692 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.236018 13735 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.241894 13738 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.242682 13735 ts_tablet_manager.cc:1434] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 12:55:56.243949 13735 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:56.245126 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:56.245754 13542 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.245833 13692 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:56.246474 13734 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:56.247313 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.247308 13734 ts_tablet_manager.cc:1403] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.059s	user 0.019s	sys 0.013s
I20260709 12:55:56.249034 13780 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.249969 13780 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:56.250089 13438 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:56.251461 13780 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.257617 13738 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:56.260476 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:56.262326 13659 leader_election.cc:343] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.268008 13542 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:56.270073 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.274940 13735 tablet_bootstrap.cc:654] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.289008 13734 raft_consensus.cc:359] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.290750 13734 raft_consensus.cc:385] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.291174 13734 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.291354 13741 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.290201 13738 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.292021 13738 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:56.291841 13741 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.292966 13738 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.294044 13741 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.293975 13734 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.296120 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.296875 13734 ts_tablet_manager.cc:1434] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.002s
I20260709 12:55:56.297866 13734 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
W20260709 12:55:56.298523 13584 leader_election.cc:343] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.297271 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.304659 13692 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:56.306923 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:56.308930 13440 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:56.309918 13809 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:56.310415 13809 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:56.310799 13809 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.311064 13733 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:56.311635 13733 ts_tablet_manager.cc:1403] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.095s	user 0.019s	sys 0.008s
I20260709 12:55:56.315260 13733 raft_consensus.cc:359] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.317003 13733 raft_consensus.cc:385] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.317353 13733 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.318996 13733 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.321275 13733 ts_tablet_manager.cc:1434] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:55:56.322283 13733 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:56.312692 13741 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.324245 13741 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.329738 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:55:56.331310 13582 leader_election.cc:343] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.331773 13618 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.333482 13809 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.335578 13809 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.328738 13741 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.340843 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:56.344082 13692 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:56.368733 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 47249.1us
I20260709 12:55:56.374557 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.375206 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:56.372365 13809 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:55:56.377254 13584 leader_election.cc:343] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.377810 13584 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:55:56.379022 13584 leader_election.cc:343] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.379531 13741 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.381065 13734 tablet_bootstrap.cc:654] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.385876 13735 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:56.386384 13735 ts_tablet_manager.cc:1403] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.143s	user 0.033s	sys 0.037s
I20260709 12:55:56.381783 13692 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:56.393888 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:56.397440 13735 raft_consensus.cc:359] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.398504 13735 raft_consensus.cc:385] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.396739 13847 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.398962 13735 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.400039 13847 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:55:56.398332 13439 catalog_manager.cc:5697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:56.409814 13780 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.410473 13780 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.403376 13847 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.417235 13735 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.418527 13733 tablet_bootstrap.cc:654] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.420310 13735 ts_tablet_manager.cc:1434] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.015s
I20260709 12:55:56.424238 13735 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:56.427186 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:56.428745 13509 leader_election.cc:343] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.435856 13814 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.435956 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.436524 13814 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:56.437587 13506 leader_election.cc:343] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.438050 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.438781 13814 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.439271 13740 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.440719 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.442566 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.443607 13692 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
W20260709 12:55:56.444648 13509 leader_election.cc:343] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.445932 13734 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:56.446314 13734 ts_tablet_manager.cc:1403] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.149s	user 0.014s	sys 0.000s
I20260709 12:55:56.434535 13780 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.450992 13506 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.451820 13740 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:56.452178 13740 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:56.452507 13740 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.453117 13735 tablet_bootstrap.cc:654] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.459091 13740 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.453934 13809 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:55:56.463759 13439 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:56.471215 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:56.472772 13509 leader_election.cc:343] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.477725 13733 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:56.478193 13733 ts_tablet_manager.cc:1403] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.156s	user 0.010s	sys 0.014s
I20260709 12:55:56.478495 13734 raft_consensus.cc:359] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.480810 13734 raft_consensus.cc:385] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.481169 13734 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.484453 13740 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.487357 13734 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.490362 13734 ts_tablet_manager.cc:1434] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.002s
I20260709 12:55:56.490864 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:56.502213 13692 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.503553 13733 raft_consensus.cc:359] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.502485 13734 tablet_bootstrap.cc:492] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:56.500864 13740 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:56.508111 13733 raft_consensus.cc:385] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.516239 13733 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.515144 13735 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:56.517457 13735 ts_tablet_manager.cc:1403] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.002s
I20260709 12:55:56.518013 13733 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.526190 13733 ts_tablet_manager.cc:1434] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:55:56.511703 13692 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:56.532685 13733 tablet_bootstrap.cc:492] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:56.532907 13506 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.534747 13740 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.535247 13740 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:56.536109 13740 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.543533 13733 tablet_bootstrap.cc:654] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.547982 13788 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:56.523289 13734 tablet_bootstrap.cc:654] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.520165 13735 raft_consensus.cc:359] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.549614 13788 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:56.550264 13735 raft_consensus.cc:385] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.550540 13735 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.551437 13847 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.551191 13735 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.552006 13847 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.553601 13735 ts_tablet_manager.cc:1434] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.001s
I20260709 12:55:56.554251 13847 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.555051 13735 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:56.554977 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.530460 13693 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.571868 13539 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.577320 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.578138 13692 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:55:56.579685 13582 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:56.580539 13855 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.604830 13809 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:56.614420 13788 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:56.621016 13809 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:56.626358 13855 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.627195 13733 tablet_bootstrap.cc:492] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:56.627792 13733 ts_tablet_manager.cc:1403] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.095s	user 0.018s	sys 0.009s
I20260709 12:55:56.631072 13733 raft_consensus.cc:359] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.631778 13733 raft_consensus.cc:385] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.632076 13733 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.628017 13740 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:55:56.634220 13733 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.634747 13735 tablet_bootstrap.cc:654] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.642012 13733 ts_tablet_manager.cc:1434] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:55:56.643370 13733 tablet_bootstrap.cc:492] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:56.662428 13440 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
W20260709 12:55:56.664748 13509 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:56.665500 13788 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:55:56.670948 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:56.666064 13788 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.674664 13788 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:56.676554 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.676751 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:56.678042 13658 leader_election.cc:343] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.678246 13788 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:55:56.679636 13659 leader_election.cc:343] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.680718 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.690325 13733 tablet_bootstrap.cc:654] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.685811 13788 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.717980 13734 tablet_bootstrap.cc:492] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:56.718462 13734 ts_tablet_manager.cc:1403] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.216s	user 0.016s	sys 0.016s
I20260709 12:55:56.723497 13782 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:56.722045 13740 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.727262 13740 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.732168 13734 raft_consensus.cc:359] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.735373 13734 raft_consensus.cc:385] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.735790 13734 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.736714 13734 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.739795 13740 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.740864 13734 ts_tablet_manager.cc:1434] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.001s
I20260709 12:55:56.741858 13734 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:56.741150 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.760095 13809 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.765193 13809 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.765486 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.766409 13691 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 1: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:55:56.767386 13809 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.768780 13735 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:56.769280 13735 ts_tablet_manager.cc:1403] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.214s	user 0.022s	sys 0.005s
I20260709 12:55:56.774791 13780 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:56.776239 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:56.778187 13506 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.776298 13735 raft_consensus.cc:359] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.779810 13538 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:56.781059 13735 raft_consensus.cc:385] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.781348 13735 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.782482 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.784523 13735 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:56.785401 13582 leader_election.cc:343] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.787616 13584 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:55:56.788774 13855 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:56.789126 13855 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:56.789470 13855 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.792776 13817 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.798015 13735 ts_tablet_manager.cc:1434] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 12:55:56.799397 13735 tablet_bootstrap.cc:492] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:56.807725 13733 tablet_bootstrap.cc:492] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:56.808274 13733 ts_tablet_manager.cc:1403] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.165s	user 0.015s	sys 0.013s
I20260709 12:55:56.813043 13733 raft_consensus.cc:359] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.813731 13733 raft_consensus.cc:385] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.813982 13733 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.825609 13733 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.835089 13733 ts_tablet_manager.cc:1434] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.006s
I20260709 12:55:56.836020 13733 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:56.838083 13855 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.840341 13817 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.842269 13855 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.843590 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:56.844265 13542 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.845392 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:55:56.846596 13582 leader_election.cc:343] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.852375 13734 tablet_bootstrap.cc:654] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.855093 13542 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:56.856423 13584 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:55:56.857781 13847 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.858278 13847 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:55:56.859063 13847 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.859831 13735 tablet_bootstrap.cc:654] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.874768 13780 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.875314 13780 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.877573 13780 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:55:56.877432 13659 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:56.880784 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.881724 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:56.882356 13693 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
W20260709 12:55:56.884312 13509 leader_election.cc:343] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.885752 13506 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.886814 13733 tablet_bootstrap.cc:654] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:56.886886 13780 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:56.887490 13780 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:56.888638 13780 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.898452 13788 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:56.899133 13788 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.899813 13780 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.901410 13788 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:56.902369 13780 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:56.903128 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:56.904656 13509 leader_election.cc:343] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.905207 13735 tablet_bootstrap.cc:492] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:56.905680 13735 ts_tablet_manager.cc:1403] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.106s	user 0.008s	sys 0.004s
I20260709 12:55:56.908042 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:56.910652 13734 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:56.910838 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:56.911257 13734 ts_tablet_manager.cc:1403] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.170s	user 0.022s	sys 0.024s
I20260709 12:55:56.911423 13693 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:56.915627 13735 raft_consensus.cc:359] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.916369 13735 raft_consensus.cc:385] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.916625 13735 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
W20260709 12:55:56.919658 13659 leader_election.cc:343] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.921149 13734 raft_consensus.cc:359] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.922048 13734 raft_consensus.cc:385] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:56.922372 13734 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:56.923302 13734 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.920737 13735 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.925637 13735 ts_tablet_manager.cc:1434] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 12:55:56.926206 13855 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:56.931110 13780 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:56.935366 13735 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:56.936172 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:56.938175 13658 leader_election.cc:343] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:56.938637 13658 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.939672 13693 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:56.940210 13788 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:56.950827 13506 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:56.957864 13780 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:56.958505 13780 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:56.959535 13780 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:56.966074 13440 catalog_manager.cc:5697] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:57.039938 13734 ts_tablet_manager.cc:1434] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.128s	user 0.007s	sys 0.005s
I20260709 12:55:57.041267 13734 tablet_bootstrap.cc:492] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
W20260709 12:55:57.045068 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 96457.1us
I20260709 12:55:57.048334 13733 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.048792 13733 ts_tablet_manager.cc:1403] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.213s	user 0.035s	sys 0.030s
I20260709 12:55:57.051219 13735 tablet_bootstrap.cc:654] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.055469 13733 raft_consensus.cc:359] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.056303 13733 raft_consensus.cc:385] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.060573 13441 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:57.056806 13733 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.066027 13733 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.068053 13733 ts_tablet_manager.cc:1434] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:55:57.068758 13847 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:57.070899 13733 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.089452 13788 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.094053 13733 tablet_bootstrap.cc:654] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.093667 13788 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.096274 13734 tablet_bootstrap.cc:654] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.096755 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:57.098272 13658 leader_election.cc:343] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.098712 13788 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.109992 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:57.111696 13659 leader_election.cc:343] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.112213 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:57.113158 13780 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:57.113333 13788 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:57.118819 13734 tablet_bootstrap.cc:492] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:57.119273 13734 ts_tablet_manager.cc:1403] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.004s
I20260709 12:55:57.123790 13733 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.124313 13733 ts_tablet_manager.cc:1403] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.005s
I20260709 12:55:57.126576 13735 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:57.127172 13735 ts_tablet_manager.cc:1403] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.192s	user 0.010s	sys 0.006s
I20260709 12:55:57.127081 13733 raft_consensus.cc:359] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.128182 13733 raft_consensus.cc:385] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.122648 13734 raft_consensus.cc:359] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.128508 13733 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.129249 13733 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.130335 13735 raft_consensus.cc:359] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.131214 13735 raft_consensus.cc:385] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.131525 13735 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.132498 13733 ts_tablet_manager.cc:1434] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:55:57.133400 13733 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.133180 13735 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.139322 13733 tablet_bootstrap.cc:654] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.141321 13735 ts_tablet_manager.cc:1434] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:55:57.142285 13735 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:57.146627 13788 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.147660 13735 tablet_bootstrap.cc:654] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.147130 13788 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.151000 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:57.151666 13619 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:57.154304 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.155114 13539 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:57.156098 13733 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.156528 13733 ts_tablet_manager.cc:1403] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.010s
I20260709 12:55:57.156842 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:57.157904 13782 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:57.158316 13782 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:57.158746 13782 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.158603 13788 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.128741 13734 raft_consensus.cc:385] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.160164 13734 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.159843 13733 raft_consensus.cc:359] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.160547 13733 raft_consensus.cc:385] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.160816 13733 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.160861 13734 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.162747 13734 ts_tablet_manager.cc:1434] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.000s
I20260709 12:55:57.165702 13782 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.166894 13734 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:57.168521 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:57.165681 13733 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.169085 13539 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.172214 13733 ts_tablet_manager.cc:1434] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:55:57.172981 13782 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.173645 13733 tablet_bootstrap.cc:492] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.173992 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:57.177582 13619 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.178560 13539 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:57.179759 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:57.182125 13619 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:57.183120 13733 tablet_bootstrap.cc:654] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.177058 13734 tablet_bootstrap.cc:654] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.185907 13782 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:57.186638 13735 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:57.187304 13782 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:57.188565 13782 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.187844 13735 ts_tablet_manager.cc:1403] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.004s
I20260709 12:55:57.191821 13733 tablet_bootstrap.cc:492] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.192485 13733 ts_tablet_manager.cc:1403] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.003s
I20260709 12:55:57.194591 13734 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:57.194721 13741 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.195050 13734 ts_tablet_manager.cc:1403] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 12:55:57.194679 13735 raft_consensus.cc:359] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.195103 13733 raft_consensus.cc:359] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.195319 13741 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.195693 13735 raft_consensus.cc:385] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.195948 13733 raft_consensus.cc:385] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.196183 13735 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.196271 13733 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.197408 13735 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.197747 13733 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.199704 13741 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.198886 13734 raft_consensus.cc:359] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.215126 13734 raft_consensus.cc:385] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.215587 13734 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.200201 13733 ts_tablet_manager.cc:1434] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:55:57.199338 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.218706 13733 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.201074 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:57.219823 13693 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:55:57.221192 13582 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:57.222504 13847 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.223690 13735 ts_tablet_manager.cc:1434] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 12:55:57.224997 13735 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:57.223323 13734 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.230053 13734 ts_tablet_manager.cc:1434] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:55:57.230181 13847 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:57.231451 13734 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:57.236676 13733 tablet_bootstrap.cc:654] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.237552 13735 tablet_bootstrap.cc:654] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.238636 13734 tablet_bootstrap.cc:654] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.251643 13733 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.252333 13733 ts_tablet_manager.cc:1403] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.003s
I20260709 12:55:57.256618 13733 raft_consensus.cc:359] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.257664 13733 raft_consensus.cc:385] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.258020 13733 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.258683 13440 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:57.261014 13733 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.262167 13734 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:57.262625 13734 ts_tablet_manager.cc:1403] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.006s
I20260709 12:55:57.263616 13733 ts_tablet_manager.cc:1434] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 12:55:57.264659 13733 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.265426 13734 raft_consensus.cc:359] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.272794 13733 tablet_bootstrap.cc:654] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.275017 13735 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:57.275520 13735 ts_tablet_manager.cc:1403] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.026s
I20260709 12:55:57.266109 13734 raft_consensus.cc:385] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.277748 13734 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.278354 13735 raft_consensus.cc:359] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.279119 13735 raft_consensus.cc:385] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.279532 13735 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.278935 13734 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.282203 13734 ts_tablet_manager.cc:1434] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:55:57.283521 13734 tablet_bootstrap.cc:492] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:57.284327 13735 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.290007 13735 ts_tablet_manager.cc:1434] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:55:57.290982 13735 tablet_bootstrap.cc:492] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:57.294629 13734 tablet_bootstrap.cc:654] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.296746 13735 tablet_bootstrap.cc:654] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.310762 13780 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.315552 13780 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.317601 13735 tablet_bootstrap.cc:492] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:57.318152 13735 ts_tablet_manager.cc:1403] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.004s
I20260709 12:55:57.318413 13780 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.318702 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:57.320007 13733 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
W20260709 12:55:57.320147 13509 leader_election.cc:343] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.320469 13733 ts_tablet_manager.cc:1403] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.024s
I20260709 12:55:57.321017 13735 raft_consensus.cc:359] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.321717 13735 raft_consensus.cc:385] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.321573 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:57.322069 13735 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.322330 13693 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:57.322889 13735 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.323019 13847 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:57.323518 13733 raft_consensus.cc:359] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.324196 13733 raft_consensus.cc:385] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.324458 13733 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.325163 13735 ts_tablet_manager.cc:1434] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:55:57.325045 13733 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.326033 13735 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:57.325932 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:57.326946 13733 ts_tablet_manager.cc:1434] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:55:57.327847 13733 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.330583 13734 tablet_bootstrap.cc:492] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:57.331146 13734 ts_tablet_manager.cc:1403] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.048s	user 0.006s	sys 0.018s
I20260709 12:55:57.332494 13847 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.333001 13847 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.334501 13733 tablet_bootstrap.cc:654] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.335265 13847 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.334630 13734 raft_consensus.cc:359] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.337167 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:57.336122 13734 raft_consensus.cc:385] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.337921 13693 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:57.338037 13734 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.339031 13734 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.340979 13735 tablet_bootstrap.cc:654] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.341423 13734 ts_tablet_manager.cc:1434] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:55:57.344094 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:57.345194 13741 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:57.345592 13741 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:57.346000 13741 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.346387 13734 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:57.356386 13733 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.357020 13733 ts_tablet_manager.cc:1403] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.015s
W20260709 12:55:57.393082 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 66954.4us
I20260709 12:55:57.400959 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.402899 13542 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:57.407183 13740 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:57.406745 13741 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.407878 13740 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:57.408478 13740 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.410929 13741 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.411795 13780 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.417578 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:57.418569 13693 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.420194 13780 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.413619 13741 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.427522 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:57.437500 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:57.438400 13540 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:57.448637 13509 leader_election.cc:343] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.450093 13733 raft_consensus.cc:359] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.450914 13733 raft_consensus.cc:385] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.451249 13733 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.451330 13735 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:57.453781 13733 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.447510 13741 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.456455 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:57.492295 13691 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:57.479839 13847 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.479282 13788 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.495785 13847 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.503824 13788 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.506827 13788 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.457170 13735 ts_tablet_manager.cc:1403] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.131s	user 0.012s	sys 0.013s
I20260709 12:55:57.512509 13741 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.513413 13733 ts_tablet_manager.cc:1434] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.156s	user 0.017s	sys 0.002s
I20260709 12:55:57.514931 13733 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.516177 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.456570 13692 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.471398 13780 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:55:57.518221 13584 leader_election.cc:343] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.522293 13734 tablet_bootstrap.cc:654] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.518206 13740 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.525951 13740 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.507433 13788 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.527486 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:57.526856 13788 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.528146 13619 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.529763 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.531695 13693 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:57.532426 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.545331 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:57.544447 13542 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:55:57.550366 13809 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:57.550971 13809 raft_consensus.cc:697] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:55:57.552232 13809 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.552969 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:57.553990 13900 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:57.554374 13900 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:57.554710 13900 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.556450 13741 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:57.543812 13733 tablet_bootstrap.cc:654] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.528575 13847 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.564690 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.533164 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:57.534236 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:57.569510 13690 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:57.569887 13691 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
W20260709 12:55:57.569468 13658 leader_election.cc:343] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.535099 13506 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:57.537353 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:57.534222 13540 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:57.572283 13687 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.541497 13735 raft_consensus.cc:359] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.572885 13735 raft_consensus.cc:385] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.573210 13735 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.573835 13735 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.574333 13780 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:57.574703 13780 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:57.575091 13780 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.577023 13735 ts_tablet_manager.cc:1434] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.001s
I20260709 12:55:57.578471 13735 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:57.581269 13817 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:57.583464 13543 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:55:57.572515 13582 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:57.585108 13733 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.585711 13741 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:57.586191 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:57.586898 13741 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:57.587406 13741 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.587489 13855 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:57.587836 13733 ts_tablet_manager.cc:1403] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.005s
I20260709 12:55:57.588369 13855 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:57.589156 13855 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.592864 13733 raft_consensus.cc:359] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.596768 13815 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.596457 13741 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.585803 13735 tablet_bootstrap.cc:654] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.600946 13733 raft_consensus.cc:385] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.600847 13815 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.602334 13733 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.603946 13815 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.605445 13741 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.603891 13733 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.609302 13733 ts_tablet_manager.cc:1434] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 12:55:57.614351 13733 tablet_bootstrap.cc:492] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.614017 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:57.619694 13817 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:57.614467 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.615710 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:57.615839 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:57.621621 13690 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.619721 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:57.624250 13540 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.533242 13788 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.601418 13687 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:57.551571 13619 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:57.633416 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:57.634557 13817 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:57.635166 13817 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:57.636301 13817 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.639634 13733 tablet_bootstrap.cc:654] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:57.649865 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:57.611408 13900 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.669411 13900 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.664829 13788 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.671487 13788 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.674042 13788 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.674739 13441 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
W20260709 12:55:57.686647 13658 leader_election.cc:343] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.691527 13440 catalog_manager.cc:5697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:57.698079 13540 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:57.699967 13782 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:57.700541 13782 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.701220 13690 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:57.702872 13782 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:57.658551 13855 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:57.710613 13659 leader_election.cc:343] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:57.711706 13659 leader_election.cc:343] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.709499 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:57.711390 13735 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:57.713099 13735 ts_tablet_manager.cc:1403] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.135s	user 0.009s	sys 0.029s
I20260709 12:55:57.714521 13780 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.712383 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:57.719710 13788 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:57.715670 13809 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:57.723874 13809 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:55:57.725008 13809 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.730712 13733 tablet_bootstrap.cc:492] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:57.731225 13733 ts_tablet_manager.cc:1403] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.117s	user 0.013s	sys 0.021s
I20260709 12:55:57.732396 13734 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:57.733399 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:57.733724 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.734051 13543 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.736285 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:57.738098 13780 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.740993 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:57.741395 13855 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:57.731853 13735 raft_consensus.cc:359] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.743337 13735 raft_consensus.cc:385] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.744625 13734 ts_tablet_manager.cc:1403] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.399s	user 0.029s	sys 0.041s
W20260709 12:55:57.744923 13509 leader_election.cc:343] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.747807 13734 raft_consensus.cc:359] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.748559 13734 raft_consensus.cc:385] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.748838 13734 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.749509 13734 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.752961 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:55:57.753300 13543 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:55:57.751752 13734 ts_tablet_manager.cc:1434] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
W20260709 12:55:57.754805 13658 leader_election.cc:343] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.754086 13733 raft_consensus.cc:359] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.953670 13733 raft_consensus.cc:385] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:57.954254 13733 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.743765 13735 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:57.812839 13440 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
W20260709 12:55:57.766989 13584 leader_election.cc:343] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:57.958496 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 191547us
I20260709 12:55:57.959658 13734 tablet_bootstrap.cc:492] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:57.959903 13735 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.955366 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:57.963949 13733 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.964419 13690 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.967262 13618 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:57.754805 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:55:57.873953 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:57.975359 13687 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:57.954206 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 198755us
I20260709 12:55:57.979835 13733 ts_tablet_manager.cc:1434] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.248s	user 0.008s	sys 0.003s
W20260709 12:55:57.984205 13658 leader_election.cc:343] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:57.990660 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:57.992408 13788 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:57.992286 13900 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:57.992946 13788 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:55:57.994263 13733 tablet_bootstrap.cc:492] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:57.993956 13788 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:57.994266 13734 tablet_bootstrap.cc:654] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.004529 13735 ts_tablet_manager.cc:1434] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.291s	user 0.012s	sys 0.002s
I20260709 12:55:58.015268 13687 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:55:58.019155 13690 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:58.020789 13506 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.022020 13814 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:58.022504 13735 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:58.022647 13814 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:58.024474 13814 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.039369 13734 tablet_bootstrap.cc:492] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:58.040081 13734 ts_tablet_manager.cc:1403] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.081s	user 0.012s	sys 0.013s
I20260709 12:55:58.043134 13735 tablet_bootstrap.cc:654] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.043764 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:58.033838 13847 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:55:58.045092 13855 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:58.045557 13855 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:55:58.046064 13780 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:58.046451 13855 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.104415 13809 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
W20260709 12:55:57.758527 13659 leader_election.cc:343] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.125844 13659 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.136070 13734 raft_consensus.cc:359] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.137055 13734 raft_consensus.cc:385] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.137388 13734 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.137988 13733 tablet_bootstrap.cc:654] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.138147 13734 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.141314 13734 ts_tablet_manager.cc:1434] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.101s	user 0.035s	sys 0.042s
I20260709 12:55:58.142527 13734 tablet_bootstrap.cc:492] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:58.145682 13735 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.146133 13735 ts_tablet_manager.cc:1403] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.124s	user 0.009s	sys 0.007s
I20260709 12:55:58.149995 13734 tablet_bootstrap.cc:654] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.151312 13899 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.155833 13735 raft_consensus.cc:359] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.156472 13441 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:58.161413 13899 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.163859 13899 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.163725 13741 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:58.167546 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.167718 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.168325 13691 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
W20260709 12:55:58.171361 13509 leader_election.cc:343] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.171850 13899 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:58.173650 13506 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.174908 13899 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:58.175274 13899 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:58.175601 13899 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.177455 13788 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:55:58.180725 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 425265us
I20260709 12:55:58.184617 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:58.189038 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.189482 13735 raft_consensus.cc:385] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.189788 13735 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.173105 13440 catalog_manager.cc:5697] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:58.188707 13899 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.191694 13735 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:58.232019 13659 leader_election.cc:343] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.257655 13817 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:55:58.258754 13659 leader_election.cc:343] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.259087 13735 ts_tablet_manager.cc:1434] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.000s
I20260709 12:55:58.259297 13659 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.216753 13809 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 12:55:58.235003 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:58.213546 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.261808 13687 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.255059 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:58.262816 13788 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:58.264638 13788 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:55:58.267470 13509 leader_election.cc:343] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.258361 13817 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.260443 13735 tablet_bootstrap.cc:492] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:58.192034 13734 tablet_bootstrap.cc:492] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:58.255888 13741 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.203240 13916 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.276294 13916 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.276595 13741 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.262933 13733 tablet_bootstrap.cc:492] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.275290 13734 ts_tablet_manager.cc:1403] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.133s	user 0.004s	sys 0.011s
I20260709 12:55:58.278271 13733 ts_tablet_manager.cc:1403] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.284s	user 0.018s	sys 0.007s
I20260709 12:55:58.280052 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.284034 13741 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.286358 13734 raft_consensus.cc:359] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.287243 13734 raft_consensus.cc:385] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.287557 13734 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
W20260709 12:55:58.288251 13658 leader_election.cc:343] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.288532 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.289727 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:55:58.290042 13509 leader_election.cc:343] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.291694 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.295439 13692 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:58.296845 13506 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.297785 13780 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:58.298139 13780 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:58.298462 13780 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.237730 13899 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.291265 13817 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.289367 13735 tablet_bootstrap.cc:654] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
W20260709 12:55:58.311568 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 54592us
I20260709 12:55:58.312004 13734 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.313797 13733 raft_consensus.cc:359] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.314268 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.314488 13733 raft_consensus.cc:385] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.314656 13734 ts_tablet_manager.cc:1434] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:55:58.314788 13733 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.315553 13733 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.316411 13780 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.322785 13687 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:58.316228 13916 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:58.324882 13506 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.325752 13618 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.324430 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:58.327078 13733 ts_tablet_manager.cc:1434] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 12:55:58.327930 13817 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:58.328155 13733 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:58.328462 13817 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:55:58.329744 13509 leader_election.cc:343] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.337781 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.338801 13817 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:58.342247 13659 leader_election.cc:343] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.343042 13659 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.344257 13782 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.345757 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:58.350226 13780 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.352875 13734 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:58.357668 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:58.358305 13687 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.377758 13930 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.385236 13930 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.367827 13687 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:58.391582 13930 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:55:58.392223 13659 leader_election.cc:343] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.393643 13506 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.399290 13659 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.400825 13817 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:58.401372 13817 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:58.402269 13782 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.401279 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.406371 13543 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:55:58.407193 13615 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.406899 13817 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.410395 13734 tablet_bootstrap.cc:654] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.412456 13782 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.413026 13782 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.413988 13735 tablet_bootstrap.cc:492] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.414448 13735 ts_tablet_manager.cc:1403] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.154s	user 0.023s	sys 0.024s
I20260709 12:55:58.415321 13782 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:58.419684 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.420275 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.420269 13735 raft_consensus.cc:359] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.421329 13735 raft_consensus.cc:385] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.421707 13735 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
W20260709 12:55:58.421803 13658 leader_election.cc:343] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.422468 13735 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.422925 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.423686 13584 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:58.424949 13735 ts_tablet_manager.cc:1434] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
W20260709 12:55:58.425372 13659 leader_election.cc:343] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.426417 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.427120 13735 tablet_bootstrap.cc:492] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:58.428082 13916 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.430394 13855 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.438522 13441 catalog_manager.cc:5697] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:58.440794 13782 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:58.444473 13929 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.445612 13733 tablet_bootstrap.cc:654] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
W20260709 12:55:58.450381 13584 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:58.455003 13782 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.455638 13782 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.458206 13782 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:58.462394 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:58.464264 13658 leader_election.cc:343] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.469897 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.470697 13780 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:55:58.471491 13659 leader_election.cc:343] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.472018 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.473222 13782 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.450398 13735 tablet_bootstrap.cc:654] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.495539 13735 tablet_bootstrap.cc:492] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.496081 13735 ts_tablet_manager.cc:1403] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.010s
I20260709 12:55:58.503850 13735 raft_consensus.cc:359] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.504758 13735 raft_consensus.cc:385] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.505053 13735 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
W20260709 12:55:58.512065 13509 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:58.516672 13735 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.517359 13734 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:58.517905 13734 ts_tablet_manager.cc:1403] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.166s	user 0.015s	sys 0.012s
I20260709 12:55:58.518447 13733 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.518890 13733 ts_tablet_manager.cc:1403] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.191s	user 0.037s	sys 0.033s
I20260709 12:55:58.498106 13440 catalog_manager.cc:5697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:55:58.521631 13735 ts_tablet_manager.cc:1434] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:55:58.522655 13735 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:58.525059 13733 raft_consensus.cc:359] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.525717 13733 raft_consensus.cc:385] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.525996 13733 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.526656 13733 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.528757 13735 tablet_bootstrap.cc:654] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.529048 13733 ts_tablet_manager.cc:1434] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 12:55:58.530148 13733 tablet_bootstrap.cc:492] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:58.533607 13734 raft_consensus.cc:359] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.534356 13734 raft_consensus.cc:385] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.534642 13734 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.535339 13734 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.538147 13815 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:58.541206 13733 tablet_bootstrap.cc:654] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.544552 13734 ts_tablet_manager.cc:1434] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:55:58.545703 13734 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:58.546446 13735 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.546924 13735 ts_tablet_manager.cc:1403] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 12:55:58.549628 13735 raft_consensus.cc:359] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.550346 13735 raft_consensus.cc:385] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.550623 13735 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.552932 13735 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.555199 13735 ts_tablet_manager.cc:1434] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:55:58.555315 13733 tablet_bootstrap.cc:492] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.555845 13733 ts_tablet_manager.cc:1403] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.002s
I20260709 12:55:58.556316 13735 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:58.552888 13734 tablet_bootstrap.cc:654] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.558478 13780 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.558964 13780 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.561038 13780 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.561589 13733 raft_consensus.cc:359] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.562153 13733 raft_consensus.cc:385] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.562418 13733 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.562908 13733 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.563277 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.565356 13733 ts_tablet_manager.cc:1434] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:55:58.566394 13733 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:58.563906 13687 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:58.567878 13438 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:58.569407 13506 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:58.570413 13814 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:58.570760 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:58.572353 13509 leader_election.cc:343] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.576314 13780 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.576771 13780 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.570793 13814 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:58.578001 13814 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.578688 13780 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.587949 13733 tablet_bootstrap.cc:654] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.591596 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.593974 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.594798 13687 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
W20260709 12:55:58.594924 13509 leader_election.cc:343] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.601876 13506 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.604928 13899 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:58.605461 13899 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:58.605901 13899 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.608619 13814 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.618722 13899 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.624413 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:58.624997 13691 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.624923 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:58.626837 13509 leader_election.cc:343] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.630510 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:58.632498 13509 leader_election.cc:343] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.634198 13735 tablet_bootstrap.cc:654] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.629469 13899 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.640578 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:58.641224 13687 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.642212 13691 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:58.648536 13734 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:58.649042 13734 ts_tablet_manager.cc:1403] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.037s
I20260709 12:55:58.648553 13814 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.649830 13506 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.652752 13817 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:58.653290 13817 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:58.653898 13734 raft_consensus.cc:359] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.654641 13734 raft_consensus.cc:385] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.654944 13734 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.654135 13817 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.655594 13687 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:58.657063 13506 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.658480 13780 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:58.658915 13780 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:58.659143 13733 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.659564 13733 ts_tablet_manager.cc:1403] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.093s	user 0.015s	sys 0.008s
I20260709 12:55:58.659698 13780 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.664716 13733 raft_consensus.cc:359] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.665454 13733 raft_consensus.cc:385] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.655654 13734 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.665741 13733 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.666718 13733 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.669166 13733 ts_tablet_manager.cc:1434] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:55:58.670225 13733 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:58.671789 13855 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:58.676237 13733 tablet_bootstrap.cc:654] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.678375 13916 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.678848 13916 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.681607 13916 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:58.682472 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.683461 13734 ts_tablet_manager.cc:1434] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
I20260709 12:55:58.685211 13735 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.687543 13735 ts_tablet_manager.cc:1403] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.131s	user 0.005s	sys 0.008s
W20260709 12:55:58.685658 13658 leader_election.cc:343] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.683902 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.689626 13916 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:55:58.690402 13659 leader_election.cc:343] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.690627 13916 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.690883 13659 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.691984 13782 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.692718 13916 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:58.694780 13735 raft_consensus.cc:359] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.695248 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.696493 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:58.696861 13658 leader_election.cc:343] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:58.697896 13659 leader_election.cc:343] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.698308 13659 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.695487 13735 raft_consensus.cc:385] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.699636 13735 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.701004 13735 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.699334 13916 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.703033 13735 ts_tablet_manager.cc:1434] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:55:58.704197 13735 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:58.704563 13734 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:58.714341 13440 catalog_manager.cc:5697] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:58.719311 13735 tablet_bootstrap.cc:654] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.723039 13733 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.736997 13733 ts_tablet_manager.cc:1403] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.067s	user 0.023s	sys 0.016s
I20260709 12:55:58.740777 13439 catalog_manager.cc:5697] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:58.744721 13733 raft_consensus.cc:359] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.745494 13733 raft_consensus.cc:385] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.745752 13733 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.746413 13733 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.724819 13734 tablet_bootstrap.cc:654] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.753826 13733 ts_tablet_manager.cc:1434] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.002s
I20260709 12:55:58.755004 13733 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:58.761746 13735 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.762269 13735 ts_tablet_manager.cc:1403] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 12:55:58.763965 13733 tablet_bootstrap.cc:654] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.763979 13780 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.764544 13780 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.765213 13735 raft_consensus.cc:359] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.765944 13735 raft_consensus.cc:385] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.766242 13735 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.766655 13780 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.766907 13735 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.773797 13735 ts_tablet_manager.cc:1434] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:55:58.774550 13847 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.775027 13847 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.776005 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.776808 13687 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:58.774569 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.777012 13847 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.779488 13735 tablet_bootstrap.cc:492] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
W20260709 12:55:58.779671 13509 leader_election.cc:343] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.782881 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.783870 13817 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:58.784257 13817 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:58.784595 13817 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.785123 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.786473 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.788978 13687 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:55:58.791657 13582 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:58.791440 13817 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.792519 13847 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.795542 13734 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:58.796090 13734 ts_tablet_manager.cc:1403] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.092s	user 0.021s	sys 0.011s
I20260709 12:55:58.798709 13734 raft_consensus.cc:359] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.799530 13734 raft_consensus.cc:385] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.799839 13734 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.800545 13734 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.802086 13734 ts_tablet_manager.cc:1434] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:55:58.803282 13734 tablet_bootstrap.cc:492] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:58.806819 13733 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.807286 13733 ts_tablet_manager.cc:1403] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.011s
I20260709 12:55:58.809871 13733 raft_consensus.cc:359] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.810545 13733 raft_consensus.cc:385] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.810600 13734 tablet_bootstrap.cc:654] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.810868 13733 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.819478 13733 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.822535 13847 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.825495 13733 ts_tablet_manager.cc:1434] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:55:58.826576 13733 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:58.829124 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:58.831012 13509 leader_election.cc:343] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.836987 13817 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.839627 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:58.840361 13687 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.841784 13735 tablet_bootstrap.cc:654] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.844105 13733 tablet_bootstrap.cc:654] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.855376 13733 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.855835 13733 ts_tablet_manager.cc:1403] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.004s
I20260709 12:55:58.861704 13687 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:58.862896 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.864290 13780 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:58.864902 13780 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:58.864742 13733 raft_consensus.cc:359] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.865465 13733 raft_consensus.cc:385] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.865736 13733 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.865926 13780 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.866398 13733 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.868535 13733 ts_tablet_manager.cc:1434] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:55:58.869486 13733 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:58.863379 13735 tablet_bootstrap.cc:492] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.871031 13735 ts_tablet_manager.cc:1403] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.092s	user 0.020s	sys 0.000s
I20260709 12:55:58.874315 13782 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.875053 13733 tablet_bootstrap.cc:654] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.874979 13782 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.879098 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:58.880849 13658 leader_election.cc:343] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.881719 13782 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:58.883565 13734 tablet_bootstrap.cc:492] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:58.883492 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:58.884212 13734 ts_tablet_manager.cc:1403] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.081s	user 0.022s	sys 0.053s
W20260709 12:55:58.884797 13659 leader_election.cc:343] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.885288 13659 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.886323 13814 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:58.886337 13782 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:58.885902 13735 raft_consensus.cc:359] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.890599 13735 raft_consensus.cc:385] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.890885 13735 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.892751 13735 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.893707 13814 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.894135 13814 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.897163 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:58.898849 13509 leader_election.cc:343] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.900640 13814 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.903683 13735 ts_tablet_manager.cc:1434] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 12:55:58.904254 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.905151 13691 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:58.906366 13900 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.906973 13900 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.909237 13734 raft_consensus.cc:359] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.909667 13900 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:58.909965 13734 raft_consensus.cc:385] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.910356 13734 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.911141 13734 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.911895 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:58.913334 13734 ts_tablet_manager.cc:1434] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 12:55:58.914371 13506 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.915138 13734 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:58.915702 13817 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:58.916245 13817 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:58.916690 13817 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:58.920648 13658 leader_election.cc:343] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.921491 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:58.922799 13659 leader_election.cc:343] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.923216 13659 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:58.923832 13899 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:58.926339 13440 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:58.934024 13734 tablet_bootstrap.cc:654] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.934267 13899 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:58.936089 13782 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:55:58.939870 13509 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:58.943068 13817 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.949780 13817 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.951395 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:58.953310 13509 leader_election.cc:343] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:58.956195 13734 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:58.956703 13734 ts_tablet_manager.cc:1403] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.003s
I20260709 12:55:58.959420 13734 raft_consensus.cc:359] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.960273 13734 raft_consensus.cc:385] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.960598 13734 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.961398 13734 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.964073 13734 ts_tablet_manager.cc:1434] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:55:58.958894 13735 tablet_bootstrap.cc:492] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:58.971732 13734 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:58.977291 13735 tablet_bootstrap.cc:654] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.982402 13733 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:58.982863 13733 ts_tablet_manager.cc:1403] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.114s	user 0.029s	sys 0.052s
I20260709 12:55:58.985409 13733 raft_consensus.cc:359] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.986060 13733 raft_consensus.cc:385] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.986349 13733 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.986987 13733 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.987519 13847 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:58.987999 13847 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.990140 13847 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:58.990739 13735 tablet_bootstrap.cc:492] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:58.991202 13735 ts_tablet_manager.cc:1403] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 12:55:58.991724 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:58.991952 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:58.992368 13691 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:58.992705 13692 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:55:58.993793 13735 raft_consensus.cc:359] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.994462 13735 raft_consensus.cc:385] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:58.994728 13735 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:58.995337 13735 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:58.997332 13735 ts_tablet_manager.cc:1434] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:55:58.998073 13734 tablet_bootstrap.cc:654] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:58.998741 13733 ts_tablet_manager.cc:1434] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:55:58.999701 13733 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.001798 13691 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:59.005127 13847 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.005439 13506 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.005584 13847 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.008154 13961 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:59.008814 13961 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:59.010218 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.009928 13961 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.012102 13847 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.017145 13735 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.027957 13733 tablet_bootstrap.cc:654] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.028421 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:59.029605 13734 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:59.030272 13734 ts_tablet_manager.cc:1403] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.063s	user 0.008s	sys 0.005s
I20260709 12:55:59.030359 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:59.031596 13735 tablet_bootstrap.cc:654] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.033272 13734 raft_consensus.cc:359] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.034842 13734 raft_consensus.cc:385] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.034977 13615 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.035223 13734 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.035300 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:59.036633 13847 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.036435 13734 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.042527 13734 ts_tablet_manager.cc:1434] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:55:59.045706 13900 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.045857 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:59.046217 13900 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.047109 13734 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
W20260709 12:55:59.053426 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:55:59.043254 13847 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.054483 13542 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.057037 13734 tablet_bootstrap.cc:654] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.043123 13439 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:59.072147 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:59.072487 13900 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:55:59.074409 13658 leader_election.cc:343] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.075660 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:59.077711 13659 leader_election.cc:343] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.078150 13659 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.079030 13916 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.085300 13733 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.085950 13733 ts_tablet_manager.cc:1403] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.020s
I20260709 12:55:59.089146 13733 raft_consensus.cc:359] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.089903 13733 raft_consensus.cc:385] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.090161 13733 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.097126 13733 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.098384 13855 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.104590 13961 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.105268 13961 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.107847 13961 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.109655 13733 ts_tablet_manager.cc:1434] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.003s
I20260709 12:55:59.111048 13733 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.112270 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.112927 13616 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.113382 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.113982 13687 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.121675 13509 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:59.122777 13780 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:59.123108 13780 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:59.123409 13780 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.127416 13733 tablet_bootstrap.cc:654] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.129426 13780 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.131155 13780 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.134541 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:59.135119 13616 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.140358 13900 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.140854 13900 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.143676 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:59.145056 13658 leader_election.cc:343] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.145345 13733 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.145937 13733 ts_tablet_manager.cc:1403] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.000s
I20260709 12:55:59.149632 13733 raft_consensus.cc:359] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.150605 13733 raft_consensus.cc:385] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.151657 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:59.153155 13659 leader_election.cc:343] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.153321 13616 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:59.153584 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.154404 13509 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:55:59.154469 13916 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.157226 13899 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:59.157750 13899 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:59.150954 13733 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.158567 13899 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.160531 13900 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:59.167483 13733 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.169428 13733 ts_tablet_manager.cc:1434] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:55:59.170377 13733 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.171269 13961 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:59.179428 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:59.180043 13687 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.182500 13735 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.182966 13735 ts_tablet_manager.cc:1403] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.166s	user 0.039s	sys 0.022s
I20260709 12:55:59.185473 13733 tablet_bootstrap.cc:654] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.196133 13687 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:59.198266 13780 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:59.203492 13735 raft_consensus.cc:359] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.204468 13735 raft_consensus.cc:385] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.204816 13735 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.205519 13735 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.207616 13735 ts_tablet_manager.cc:1434] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:55:59.203171 13439 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:59.213445 13780 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.213967 13780 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.213497 13735 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.215700 13780 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.217157 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.222411 13692 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.221222 13734 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:59.223644 13734 ts_tablet_manager.cc:1403] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.177s	user 0.049s	sys 0.067s
I20260709 12:55:59.229156 13506 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:59.231056 13780 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:59.231463 13780 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:59.231673 13780 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.234575 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.240371 13900 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.240773 13735 tablet_bootstrap.cc:654] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.240895 13900 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:59.244316 13509 leader_election.cc:343] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:59.245469 13509 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:55:59.248699 13734 raft_consensus.cc:359] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.249250 13734 raft_consensus.cc:385] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.249648 13734 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.250524 13734 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.252046 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:59.253321 13658 leader_election.cc:343] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.253113 13817 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:59.254467 13780 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.256809 13780 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.258737 13733 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.258858 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:59.259250 13733 ts_tablet_manager.cc:1403] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.089s	user 0.001s	sys 0.011s
I20260709 12:55:59.259654 13691 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.262058 13733 raft_consensus.cc:359] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.262676 13733 raft_consensus.cc:385] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.262923 13733 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.264370 13733 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.266398 13733 ts_tablet_manager.cc:1434] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 12:55:59.269168 13734 ts_tablet_manager.cc:1434] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 12:55:59.270141 13735 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.270839 13735 ts_tablet_manager.cc:1403] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.000s
I20260709 12:55:59.271199 13734 tablet_bootstrap.cc:492] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:59.276610 13735 raft_consensus.cc:359] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.277405 13735 raft_consensus.cc:385] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.277745 13735 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.278405 13735 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.280938 13735 ts_tablet_manager.cc:1434] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:55:59.281564 13961 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:59.281972 13735 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.282994 13734 tablet_bootstrap.cc:654] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.270267 13733 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.284607 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:59.286036 13659 leader_election.cc:343] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.286489 13659 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.287249 13900 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:59.287495 13788 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.288239 13735 tablet_bootstrap.cc:654] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.290026 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:59.271088 13691 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:59.293040 13506 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:59.294222 13817 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:59.294735 13817 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:59.297755 13817 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.304382 13733 tablet_bootstrap.cc:654] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.311962 13735 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.313616 13735 ts_tablet_manager.cc:1403] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.008s
I20260709 12:55:59.316947 13735 raft_consensus.cc:359] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.317920 13735 raft_consensus.cc:385] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.318327 13735 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
W20260709 12:55:59.312716 13509 leader_election.cc:343] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.321712 13735 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.306167 13616 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.323792 13735 ts_tablet_manager.cc:1434] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:55:59.330325 13735 tablet_bootstrap.cc:492] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.341610 13735 tablet_bootstrap.cc:654] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.341753 13439 catalog_manager.cc:5697] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:55:59.346120 13817 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:59.346760 13734 tablet_bootstrap.cc:492] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:59.347267 13734 ts_tablet_manager.cc:1403] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.004s
I20260709 12:55:59.350567 13733 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.351022 13733 ts_tablet_manager.cc:1403] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.081s	user 0.016s	sys 0.008s
I20260709 12:55:59.353726 13733 raft_consensus.cc:359] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.354403 13733 raft_consensus.cc:385] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.354666 13733 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.355407 13733 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.357513 13733 ts_tablet_manager.cc:1434] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:55:59.355857 13734 raft_consensus.cc:359] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.359185 13734 raft_consensus.cc:385] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.359447 13734 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.360103 13734 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.361107 13735 tablet_bootstrap.cc:492] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.361720 13735 ts_tablet_manager.cc:1403] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.002s
I20260709 12:55:59.362087 13734 ts_tablet_manager.cc:1434] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.004s
I20260709 12:55:59.363126 13734 tablet_bootstrap.cc:492] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:59.365170 13733 tablet_bootstrap.cc:492] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.368562 13735 raft_consensus.cc:359] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.369530 13735 raft_consensus.cc:385] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.369874 13735 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.370757 13735 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.371534 13900 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.372328 13900 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.373008 13735 ts_tablet_manager.cc:1434] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:55:59.374326 13900 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:59.374698 13733 tablet_bootstrap.cc:654] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.375422 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.375784 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:59.377264 13658 leader_election.cc:343] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.378862 13735 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.379688 13734 tablet_bootstrap.cc:654] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
W20260709 12:55:59.383147 13659 leader_election.cc:343] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.383651 13659 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.384591 13900 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.386569 13735 tablet_bootstrap.cc:654] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.387260 13733 tablet_bootstrap.cc:492] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.387817 13733 ts_tablet_manager.cc:1403] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.001s
I20260709 12:55:59.391709 13733 raft_consensus.cc:359] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.392835 13733 raft_consensus.cc:385] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.393250 13733 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.394193 13733 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.396685 13733 ts_tablet_manager.cc:1434] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:55:59.397500 13735 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.397929 13733 tablet_bootstrap.cc:492] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.397959 13735 ts_tablet_manager.cc:1403] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.006s
I20260709 12:55:59.400653 13735 raft_consensus.cc:359] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.401350 13735 raft_consensus.cc:385] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.401635 13735 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.402292 13735 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.404628 13735 ts_tablet_manager.cc:1434] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:55:59.404913 13733 tablet_bootstrap.cc:654] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.405575 13735 tablet_bootstrap.cc:492] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.414100 13735 tablet_bootstrap.cc:654] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.420238 13733 tablet_bootstrap.cc:492] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.420962 13733 ts_tablet_manager.cc:1403] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.010s
I20260709 12:55:59.423790 13735 tablet_bootstrap.cc:492] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.424285 13735 ts_tablet_manager.cc:1403] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.006s
I20260709 12:55:59.423979 13733 raft_consensus.cc:359] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.424835 13733 raft_consensus.cc:385] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.425176 13733 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.426096 13733 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.427431 13735 raft_consensus.cc:359] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.428210 13735 raft_consensus.cc:385] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.428376 13733 ts_tablet_manager.cc:1434] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:55:59.428473 13735 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.429365 13735 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.429848 13733 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.431658 13735 ts_tablet_manager.cc:1434] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:55:59.432973 13855 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.433459 13855 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.436189 13733 tablet_bootstrap.cc:654] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.437178 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:59.437448 13855 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.437572 13735 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.438596 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.439824 13616 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.441337 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:59.445153 13735 tablet_bootstrap.cc:654] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.448262 13855 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.449656 13817 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:59.457398 13735 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.457398 13733 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.457754 13780 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.459048 13733 ts_tablet_manager.cc:1403] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.003s
I20260709 12:55:59.458750 13780 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.457974 13735 ts_tablet_manager.cc:1403] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.007s
I20260709 12:55:59.460839 13780 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.462587 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.462765 13735 raft_consensus.cc:359] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.462913 13733 raft_consensus.cc:359] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.463444 13735 raft_consensus.cc:385] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.463766 13733 raft_consensus.cc:385] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.463757 13735 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.464283 13733 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.464625 13735 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.465173 13733 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.466704 13735 ts_tablet_manager.cc:1434] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:55:59.467659 13735 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:55:59.467746 13733 ts_tablet_manager.cc:1434] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:55:59.468818 13733 tablet_bootstrap.cc:492] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.469399 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.473872 13735 tablet_bootstrap.cc:654] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.463277 13691 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.476215 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:59.477351 13961 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:59.477744 13961 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:59.478048 13961 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.479660 13733 tablet_bootstrap.cc:654] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.480069 13899 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:59.481688 13734 tablet_bootstrap.cc:492] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:59.482121 13734 ts_tablet_manager.cc:1403] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.119s	user 0.045s	sys 0.020s
W20260709 12:55:59.484320 13509 leader_election.cc:343] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.488471 13734 raft_consensus.cc:359] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.491770 13734 raft_consensus.cc:385] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.492551 13734 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.494141 13734 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.499286 13734 ts_tablet_manager.cc:1434] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:55:59.503315 13734 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:59.517349 13616 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.523080 13734 tablet_bootstrap.cc:654] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.527313 13961 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.533062 13817 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.533672 13817 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.536474 13734 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:59.536674 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.539690 13734 ts_tablet_manager.cc:1403] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.000s
I20260709 12:55:59.540555 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.541240 13691 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.542300 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:55:59.546665 13734 raft_consensus.cc:359] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.548592 13899 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:59.551906 13899 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:59.552790 13899 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:59.554149 13509 leader_election.cc:343] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.563309 13899 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.566535 13734 raft_consensus.cc:385] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.567807 13734 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.571363 13899 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.569492 13734 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.574146 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:59.574788 13687 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.576648 13734 ts_tablet_manager.cc:1434] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:55:59.579456 13734 tablet_bootstrap.cc:492] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:59.590526 13780 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.591024 13780 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.594169 13780 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.596117 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:55:59.597461 13509 leader_election.cc:343] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.599805 13734 tablet_bootstrap.cc:654] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:55:59.599881 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:59.602631 13509 leader_election.cc:343] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.606184 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.606861 13690 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.608006 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.608848 13814 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:59.609205 13814 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:59.609499 13814 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.580505 13900 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.612325 13734 tablet_bootstrap.cc:492] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:55:59.612151 13900 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.613060 13780 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.613485 13780 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.615207 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.616410 13780 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.616291 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:59.617174 13659 leader_election.cc:343] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.617705 13900 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:55:59.618352 13509 leader_election.cc:343] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.618794 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.619406 13690 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.620653 13506 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.621649 13780 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:55:59.622005 13780 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:55:59.622347 13780 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.626318 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:59.627938 13658 leader_election.cc:343] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.628369 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.628612 13780 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.629666 13782 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.638160 13855 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.638069 13687 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:59.638959 13855 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.641695 13855 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.638932 13817 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.645256 13817 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:55:59.642949 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.659341 13788 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.659873 13788 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.647433 13961 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.665349 13734 ts_tablet_manager.cc:1403] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.000s
I20260709 12:55:59.661959 13788 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:59.671527 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:59.671685 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.672362 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:59.672912 13616 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:55:59.673163 13659 leader_election.cc:343] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:59.676783 13658 leader_election.cc:343] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.677333 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:55:59.685001 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52726.1us
I20260709 12:55:59.686378 13780 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.687945 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:55:59.688090 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:55:59.688846 13788 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.689787 13780 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:55:59.690196 13780 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:55:59.690575 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:59.690889 13780 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:55:59.692729 13509 leader_election.cc:343] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.698385 13735 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:55:59.698772 13735 ts_tablet_manager.cc:1403] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.231s	user 0.067s	sys 0.101s
I20260709 12:55:59.696388 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:59.700551 13692 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.701817 13734 raft_consensus.cc:359] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.702513 13734 raft_consensus.cc:385] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.702673 13733 tablet_bootstrap.cc:492] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:55:59.702843 13734 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.703096 13733 ts_tablet_manager.cc:1403] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.234s	user 0.032s	sys 0.051s
I20260709 12:55:59.691860 13616 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:59.703642 13734 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.704815 13961 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.706548 13733 raft_consensus.cc:359] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.707161 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:55:59.711160 13733 raft_consensus.cc:385] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.711597 13733 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.712492 13855 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:55:59.712399 13733 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.708045 13735 raft_consensus.cc:359] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.710668 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:55:59.714556 13817 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:55:59.714910 13691 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:55:59.714588 13733 ts_tablet_manager.cc:1434] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:55:59.720463 13788 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.709481 13961 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.721053 13788 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.721644 13734 ts_tablet_manager.cc:1434] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.000s
I20260709 12:55:59.722412 13961 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.722548 13734 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:55:59.722754 13900 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.723222 13900 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.723852 13735 raft_consensus.cc:385] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:55:59.724223 13735 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:55:59.724072 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:55:59.725741 13814 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.732601 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:55:59.733963 13658 leader_election.cc:343] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:55:59.735515 13509 leader_election.cc:343] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:55:59.736003 13900 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:59.736984 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.790508 13733 tablet_bootstrap.cc:492] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:55:59.742780 13438 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:00.033926 13735 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:55:59.803582 13509 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:00.038087 13780 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:00.039047 13780 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:00.040719 13780 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:00.043251 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 239753us
I20260709 12:55:59.737262 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:55:59.949451 13719 tablet.cc:2406] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:55:59.787827 13788 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:59.861703 13900 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:00.060189 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:00.060320 13900 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.062418 13900 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:55:59.770664 13899 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:55:59.789425 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:55:59.738554 13814 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:55:59.745755 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:55:59.830484 13692 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:55:59.784889 13855 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:00.038542 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 90637.7us
I20260709 12:55:59.832998 13691 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:00.068195 13735 ts_tablet_manager.cc:1434] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.369s	user 0.019s	sys 0.004s
I20260709 12:55:59.916779 13847 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:00.100056 13690 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:55:59.832561 14001 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:00.110991 13899 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.118667 13847 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.110280 13855 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.222576 13506 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:00.115507 14001 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.226042 14001 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.226326 13733 tablet_bootstrap.cc:654] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:00.227005 13619 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:00.275136 13658 leader_election.cc:343] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:56:00.484689 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 209603us
I20260709 12:56:00.485950 13734 tablet_bootstrap.cc:654] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:00.425273 13788 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:00.486747 13788 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.489107 13899 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:00.367589 14005 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:56:00.368162 14006 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:56:00.478508 13735 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:00.366779 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:00.477859 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 255380us
I20260709 12:56:00.505085 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:00.505965 13539 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:00.506548 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:00.507047 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:00.233170 13814 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:00.235870 13847 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.512312 13788 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:00.510679 13855 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.510691 13543 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:56:00.534587 13788 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:00.542487 14003 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:00.545272 13780 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:00.551177 14004 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:56:00.551656 13733 tablet_bootstrap.cc:492] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:00.535633 13619 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:00.553898 13733 ts_tablet_manager.cc:1403] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.764s	user 0.027s	sys 0.043s
I20260709 12:56:00.545729 13780 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.571018 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:00.575501 13780 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.580538 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:00.588685 14009 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:00.589321 14009 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:00.589725 14009 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:00.547668 13788 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.568146 13814 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:00.596930 13733 raft_consensus.cc:359] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.598745 13733 raft_consensus.cc:385] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:00.599051 13733 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:00.599479 13735 tablet_bootstrap.cc:654] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:00.601549 13733 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:00.606105 13658 leader_election.cc:343] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:00.611409 13733 ts_tablet_manager.cc:1434] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.052s	user 0.003s	sys 0.005s
W20260709 12:56:00.609305 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 60648.8us
I20260709 12:56:00.612762 13814 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.628082 13615 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:00.643775 13788 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:00.635524 13613 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:00.648885 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:00.649626 13688 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:00.656687 14003 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:00.657235 14003 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:00.668561 13659 leader_election.cc:343] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:00.673276 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:00.668006 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:00.679685 13693 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
W20260709 12:56:00.687820 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 47939.1us
I20260709 12:56:00.692787 14017 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:00.711556 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:00.712255 13539 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:56:00.715521 13506 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:00.716312 13734 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:00.716815 13734 ts_tablet_manager.cc:1403] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.994s	user 0.031s	sys 0.028s
I20260709 12:56:00.721832 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:00.722302 14016 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:00.722724 14016 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:00.723084 14016 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:00.723666 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:00.719164 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:00.725426 13687 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:00.732470 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:00.732926 14015 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:00.734449 14008 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:00.735714 13814 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:00.736135 13814 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:00.736493 13814 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:00.738749 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:00.739418 13689 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:00.733402 14015 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:00.743678 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:00.743899 14015 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:00.734848 14008 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:00.748373 14008 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:00.750986 13438 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.027s	sys 0.015s
W20260709 12:56:00.751403 13438 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.027s	sys 0.015s
I20260709 12:56:00.741570 14017 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:00.760357 13780 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:00.738699 14018 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:56:00.756412 14013 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:56:00.766765 14007 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:00.766822 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:00.767359 14007 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.780934 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:00.789561 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 123) took 1054 ms. Trace:
I20260709 12:56:00.763263 14017 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.790112 14003 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.811174 13735 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:00.811694 13735 ts_tablet_manager.cc:1403] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.333s	user 0.018s	sys 0.010s
I20260709 12:56:00.794790 14007 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.768450 14018 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:00.830699 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 162165us
W20260709 12:56:00.831415 13659 leader_election.cc:343] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:00.831832 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:00.790093 13619 rpcz_store.cc:276] 0709 12:55:59.735165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:55:59.735304 (+   139us) service_pool.cc:224] Handling call
0709 12:56:00.789366 (+1054062us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540013b0d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:00.789481 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367301504}
I20260709 12:56:00.829084 13899 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:56:00.849951 13734 raft_consensus.cc:359] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.832388 14007 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:00.852327 13734 raft_consensus.cc:385] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:00.853677 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:00.853829 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:00.822372 14017 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:00.841531 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:00.841019 14021 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:00.838392 13788 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:00.856426 13855 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:00.856029 13734 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:00.862943 14003 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.866710 13734 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.869788 13855 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.866456 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:00.868264 13788 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.847183 13689 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:00.878227 13734 ts_tablet_manager.cc:1434] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.161s	user 0.005s	sys 0.005s
I20260709 12:56:00.883839 13734 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:00.884951 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:00.866345 13814 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:00.896286 13509 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:56:00.896984 13509 leader_election.cc:343] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:00.900024 13735 raft_consensus.cc:359] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.900756 13735 raft_consensus.cc:385] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:00.901018 13735 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:00.901654 13735 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.902684 13930 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:00.903240 13930 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:00.866636 14007 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.903861 13438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49880 (request call id 80) took 1168 ms. Trace:
I20260709 12:56:00.903800 13735 ts_tablet_manager.cc:1434] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.092s	user 0.003s	sys 0.000s
W20260709 12:56:00.906648 13509 leader_election.cc:343] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:00.882926 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 128) took 1086 ms. Trace:
I20260709 12:56:00.886979 13855 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.916816 13735 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:00.963795 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:00.912407 13689 rpcz_store.cc:276] 0709 12:55:59.794339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:55:59.794489 (+   150us) service_pool.cc:224] Handling call
0709 12:56:00.144026 (+349537us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:56:00.726855 (+582829us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:56:00.877752 (+150897us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55398400}
I20260709 12:56:00.967941 14001 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:00.938197 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 130) took 1027 ms. Trace:
I20260709 12:56:00.971170 14024 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:00.903623 14016 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.972544 14007 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:00.939785 14015 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.879657 14009 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:00.979310 14011 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:00.980834 13788 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:00.988258 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45232.9us
I20260709 12:56:00.981194 13685 rpcz_store.cc:276] 0709 12:55:59.882880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:55:59.883185 (+   305us) service_pool.cc:224] Handling call
0709 12:56:00.080208 (+197023us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:00.812789 (+732581us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:00.855721 (+ 42932us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68df8c 00007fa02b68def6 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:00.872592 (+ 16871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:00.881304 (+  8712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:00.882698 (+  1394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:00.885758 (+  3060us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:00.892193 (+  6435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:00.895603 (+  3410us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:00.909732 (+ 14129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34622720}
W20260709 12:56:00.988752 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:56:00.989812 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:00.991199 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:00.993085 14003 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:00.993764 14003 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:00.939133 14018 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.000419 14011 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.002329 13814 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:01.011312 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 102345us
I20260709 12:56:01.014019 13688 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 1: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:01.001219 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.006127 14007 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:01.021533 14007 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.024645 14007 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.016848 14015 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.013600 14008 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.028674 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:01.030730 13684 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.026033 14003 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.029008 13930 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:00.996878 14024 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.045089 14024 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.047173 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:01.065990 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49575.9us
I20260709 12:56:01.066624 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.073937 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:01.074828 13692 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 1: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:01.076021 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.079674 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.080386 13541 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:56:01.080509 14009 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.089089 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.090734 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.092446 13780 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:56:01.093593 13509 leader_election.cc:343] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.095073 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:01.095219 13509 leader_election.cc:343] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:00.909976 13438 rpcz_store.cc:276] 0709 12:55:59.735093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:55:59.739787 (+  4694us) service_pool.cc:224] Handling call
0709 12:56:00.903766 (+1163979us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:56:00.548221 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:56:00.548754 (+   533us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:56:00.548787 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:56:00.548804 (+    17us) tablet.cc:662] Decoding operations
0709 12:56:00.549125 (+   321us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:56:00.549169 (+    44us) write_op.cc:641] Partition lock acquired for write op
0709 12:56:00.549193 (+    24us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:56:00.549366 (+   173us) tablet.cc:701] Row locks acquired
0709 12:56:00.549384 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:56:00.549503 (+   119us) write_op.cc:270] Start()
0709 12:56:00.549619 (+   116us) write_op.cc:276] Timestamp: P: 1783601760549464 usec, L: 0
0709 12:56:00.549641 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:56:00.550223 (+   582us) log.cc:844] Serialized 537 byte log entry
0709 12:56:00.742623 (+192400us) op_driver.cc:464] REPLICATION: finished
0709 12:56:00.876569 (+133946us) write_op.cc:301] APPLY: starting
0709 12:56:00.880648 (+  4079us) tablet.cc:1367] starting BulkCheckPresence
0709 12:56:00.886251 (+  5603us) tablet.cc:1370] finished BulkCheckPresence
0709 12:56:00.898500 (+ 12249us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:56:00.898786 (+   286us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:56:00.898942 (+   156us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:56:00.898994 (+    52us) write_op.cc:312] APPLY: finished
0709 12:56:00.899215 (+   221us) log.cc:844] Serialized 22 byte log entry
0709 12:56:00.900024 (+   809us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:00.900107 (+    83us) write_op.cc:454] Released schema lock
0709 12:56:00.900146 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:56:00.900871 (+   725us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:00.900910 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":122191,"apply.run_cpu_time_us":2169,"apply.run_wall_time_us":29509,"num_ops":1,"prepare.queue_time_us":498494,"prepare.run_cpu_time_us":17656,"prepare.run_wall_time_us":76061,"raft.queue_time_us":81379,"raft.run_cpu_time_us":34392,"raft.run_wall_time_us":289609,"replication_time_us":182883,"spinlock_wait_cycles":6464768,"thread_start_us":905709,"threads_started":4,"wal-append.queue_time_us":85538}]]}
I20260709 12:56:01.071017 14011 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:01.103261 13659 leader_election.cc:343] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.106279 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.098635 14009 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:01.107528 14009 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:01.107872 14009 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.109767 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.111766 13733 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:01.091570 13855 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:01.112322 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.112792 14008 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.101354 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.118121 13582 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.121505 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.121881 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:01.122186 14040 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.122495 13685 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:01.095752 14015 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:56:01.101008 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:01.100759 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:01.127354 13509 leader_election.cc:343] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.128151 13509 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.118297 13684 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:01.116350 14016 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.130465 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:01.128149 14040 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.128584 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.135774 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:01.129914 13899 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.140614 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.148044 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:01.148658 14009 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.151491 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.152375 13541 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:01.159495 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.144495 14008 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.161017 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.162478 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:01.163626 13539 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:01.164057 13536 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:01.165792 13659 leader_election.cc:343] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.166353 13735 tablet_bootstrap.cc:654] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.155023 14009 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.166635 14008 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.172847 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.181739 13734 tablet_bootstrap.cc:654] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
W20260709 12:56:01.183655 13509 leader_election.cc:343] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.175531 14008 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:01.188877 13509 leader_election.cc:343] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.192764 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:01.193709 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.196453 14008 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.196627 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.196928 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.201588 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.203064 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.202813 13439 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:56:01.205081 13899 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:01.205554 13899 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:01.205765 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.211761 13733 tablet_bootstrap.cc:654] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.197099 14008 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.224368 14002 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.226267 13617 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.226480 14008 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.225033 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:01.206461 13899 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:01.231233 13658 leader_election.cc:343] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.192145 13536 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:01.235059 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.236091 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:01.242357 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.241667 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.244859 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.253786 13659 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:01.255147 13659 leader_election.cc:343] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.255694 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.264009 13734 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:01.264678 13734 ts_tablet_manager.cc:1403] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.381s	user 0.028s	sys 0.030s
I20260709 12:56:01.253089 14009 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:01.257860 14031 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:01.261294 13788 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.289337 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:01.289947 13693 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.290623 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.295145 14018 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:01.295568 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:01.297550 13659 leader_election.cc:343] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.297910 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:01.298761 13659 leader_election.cc:343] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.298998 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:01.299487 13659 leader_election.cc:343] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.299808 13659 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.302814 13584 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.303624 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.307399 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.298532 14018 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:01.310021 13930 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.304242 14007 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:01.304328 13782 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.304373 13847 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:01.304512 14017 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.313812 14035 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:56:01.316035 14042 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:56:01.317406 14008 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.317929 14008 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:01.318956 14043 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:56:01.311846 14014 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:56:01.324083 14002 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:01.326900 14008 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.327391 13693 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
W20260709 12:56:01.334224 13658 leader_election.cc:343] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.332262 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.339778 13506 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.339124 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:01.342687 13734 raft_consensus.cc:359] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.343395 13734 raft_consensus.cc:385] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.343669 13734 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.356010 13734 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.360370 13734 ts_tablet_manager.cc:1434] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.002s
I20260709 12:56:01.356035 13899 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:01.358168 14008 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:56:01.363694 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 55135.9us
I20260709 12:56:01.364398 13735 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:01.365901 13735 ts_tablet_manager.cc:1403] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.454s	user 0.013s	sys 0.037s
I20260709 12:56:01.366451 13734 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
W20260709 12:56:01.368047 13659 leader_election.cc:343] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.368551 13659 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:01.369446 13659 leader_election.cc:343] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.369771 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:01.370559 13659 leader_election.cc:343] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.365202 13899 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:01.366742 14008 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:01.374290 14009 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.373041 14021 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.377846 14008 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.368827 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:01.381870 13659 leader_election.cc:343] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.385613 13899 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:01.414283 13658 leader_election.cc:343] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.414710 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.415804 14017 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.425334 13438 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:01.429965 13733 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:01.430521 13733 ts_tablet_manager.cc:1403] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.319s	user 0.005s	sys 0.014s
I20260709 12:56:01.430274 13438 catalog_manager.cc:5697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:01.441344 13733 raft_consensus.cc:359] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.448807 13733 raft_consensus.cc:385] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.449215 13733 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.449991 13733 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.455570 13735 raft_consensus.cc:359] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.456277 13735 raft_consensus.cc:385] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.456525 13735 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.464133 13735 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.473620 13735 ts_tablet_manager.cc:1434] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.106s	user 0.006s	sys 0.000s
I20260709 12:56:01.482883 13735 tablet_bootstrap.cc:492] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:01.482671 13439 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:01.486320 13899 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:01.489153 13733 ts_tablet_manager.cc:1434] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.004s
I20260709 12:56:01.494382 13733 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:01.505798 13735 tablet_bootstrap.cc:654] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.506842 13734 tablet_bootstrap.cc:654] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.518289 13733 tablet_bootstrap.cc:654] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.522903 13441 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:01.525511 13441 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:01.529619 13734 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:01.530138 13734 ts_tablet_manager.cc:1403] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.164s	user 0.006s	sys 0.007s
I20260709 12:56:01.526950 13733 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:01.530987 13733 ts_tablet_manager.cc:1403] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.001s
I20260709 12:56:01.533684 13733 raft_consensus.cc:359] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.534325 13733 raft_consensus.cc:385] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.534598 13733 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.535279 13733 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.526950 13735 tablet_bootstrap.cc:492] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:01.536192 13735 ts_tablet_manager.cc:1403] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.054s	user 0.006s	sys 0.007s
I20260709 12:56:01.538722 13735 raft_consensus.cc:359] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.541123 13735 raft_consensus.cc:385] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.541396 13734 raft_consensus.cc:359] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.542078 13734 raft_consensus.cc:385] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.542320 13734 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.542928 13734 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.541374 13735 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.544363 13733 ts_tablet_manager.cc:1434] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:56:01.544402 13735 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.546975 13735 ts_tablet_manager.cc:1434] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:56:01.547937 13735 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:01.548892 14015 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:01.553236 13734 ts_tablet_manager.cc:1434] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:56:01.555233 13734 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:01.561740 13733 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:01.570912 13733 tablet_bootstrap.cc:654] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.594413 13788 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.594933 13788 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.597106 13788 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.598991 13734 tablet_bootstrap.cc:654] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.599452 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.611958 14051 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.601239 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.612471 14051 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.613337 13733 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
W20260709 12:56:01.613468 13658 leader_election.cc:343] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:56:01.614436 13659 leader_election.cc:343] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.614526 14051 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.614847 13659 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.615509 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.616825 13733 ts_tablet_manager.cc:1403] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.052s	user 0.006s	sys 0.009s
I20260709 12:56:01.617834 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.616744 14008 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.607038 13735 tablet_bootstrap.cc:654] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.620712 13582 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.621872 14018 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.623834 13733 raft_consensus.cc:359] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.626120 13733 raft_consensus.cc:385] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.626441 13733 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.627343 13733 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.631539 13733 ts_tablet_manager.cc:1434] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:56:01.632608 14018 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.632615 13733 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:01.633215 14018 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.635380 14018 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.637862 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.639074 13734 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:01.639287 13691 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:01.639599 13734 ts_tablet_manager.cc:1403] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.085s	user 0.008s	sys 0.005s
I20260709 12:56:01.643096 13734 raft_consensus.cc:359] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.644145 13734 raft_consensus.cc:385] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.644476 13734 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.645140 13734 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.646432 14008 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.649564 14008 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.651791 14008 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.652766 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.653932 13734 ts_tablet_manager.cc:1434] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 12:56:01.656121 13734 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:01.658638 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:01.647646 13733 tablet_bootstrap.cc:654] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
W20260709 12:56:01.654503 13658 leader_election.cc:343] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.661724 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.662608 13536 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:56:01.663586 13659 leader_election.cc:343] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.664113 13659 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.664870 13734 tablet_bootstrap.cc:654] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.664778 13788 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.668545 13584 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.679128 14018 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.686658 13735 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:01.687242 13735 ts_tablet_manager.cc:1403] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.140s	user 0.007s	sys 0.004s
I20260709 12:56:01.694003 13735 raft_consensus.cc:359] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.710249 13734 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
W20260709 12:56:01.712517 13509 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:01.716997 13735 raft_consensus.cc:385] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.717260 13734 ts_tablet_manager.cc:1403] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.015s
I20260709 12:56:01.712393 14018 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.717391 13735 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.719952 13735 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.721074 13734 raft_consensus.cc:359] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.721778 13734 raft_consensus.cc:385] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.722047 13734 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.722793 13734 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.724234 14015 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:01.727356 13735 ts_tablet_manager.cc:1434] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.004s
I20260709 12:56:01.728417 13735 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:01.730814 13734 ts_tablet_manager.cc:1434] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:56:01.732066 13734 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:01.734838 13735 tablet_bootstrap.cc:654] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.739847 13733 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:01.740216 13734 tablet_bootstrap.cc:654] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.740403 13733 ts_tablet_manager.cc:1403] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.026s
I20260709 12:56:01.743424 13733 raft_consensus.cc:359] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.744189 13733 raft_consensus.cc:385] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.744529 13733 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.745340 13733 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.747852 13733 ts_tablet_manager.cc:1434] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:56:01.749161 13733 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:01.756430 13734 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:01.756943 13734 ts_tablet_manager.cc:1403] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.008s
I20260709 12:56:01.759949 13734 raft_consensus.cc:359] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.760687 13734 raft_consensus.cc:385] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.761006 13734 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.764263 13735 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:01.764750 13735 ts_tablet_manager.cc:1403] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.007s
I20260709 12:56:01.761722 13734 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.765491 13613 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.767409 13734 ts_tablet_manager.cc:1434] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:56:01.767486 13735 raft_consensus.cc:359] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.768335 13735 raft_consensus.cc:385] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.768682 13735 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.768898 13734 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:01.769381 13735 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.772110 14024 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.772739 13733 tablet_bootstrap.cc:654] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.772686 14024 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.776782 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.777768 13540 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:56:01.779846 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:01.780927 14008 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:01.781533 14008 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:01.782027 14008 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.788403 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:01.789865 13659 leader_election.cc:343] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.790457 14024 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.796146 13735 ts_tablet_manager.cc:1434] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 12:56:01.798197 13735 tablet_bootstrap.cc:492] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:01.799046 13734 tablet_bootstrap.cc:654] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.801883 14024 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.802603 14024 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.806484 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.807386 13540 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 0.
I20260709 12:56:01.810151 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:01.811213 13788 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:01.811689 13788 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:01.812112 13788 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:01.814388 13509 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:01.815385 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:01.816836 13659 leader_election.cc:343] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.818984 14018 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.819501 14018 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.821763 14018 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.814405 14024 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.823187 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.824599 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.831878 13899 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.833063 13899 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.834599 13584 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:01.835690 13899 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.839617 14017 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:01.840804 14008 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.838135 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:01.842901 13509 leader_election.cc:343] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.835727 14018 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.844915 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:01.846141 13659 leader_election.cc:343] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.847081 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:01.847867 13540 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.854693 13788 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.857268 13788 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.858780 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:01.859472 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:01.859468 13536 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.861498 13616 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:01.869338 13659 leader_election.cc:343] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.870574 14021 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:01.872474 14018 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.872984 14018 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.874989 14018 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.875000 13536 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:01.882031 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.883962 14031 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:01.884591 14031 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:01.885640 14031 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.887758 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.888525 13691 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:01.888965 14017 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.889623 14017 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.892658 14017 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.892794 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.893440 13536 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:56:01.897279 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:01.898733 13659 leader_election.cc:343] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.899266 13540 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:01.843797 14008 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:01.900228 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:01.901052 14007 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:01.901465 14007 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:01.901603 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.901836 14007 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.902220 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.903123 13687 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:56:01.903651 14014 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:01.904070 14014 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:01.904395 14014 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.908213 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.910140 13735 tablet_bootstrap.cc:654] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
W20260709 12:56:01.910648 13658 leader_election.cc:343] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.911069 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.911369 14014 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.909509 14007 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.913183 13733 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:01.913650 13733 ts_tablet_manager.cc:1403] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.165s	user 0.008s	sys 0.006s
I20260709 12:56:01.914140 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:01.915361 13509 leader_election.cc:343] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:01.918619 13734 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:01.922914 13734 ts_tablet_manager.cc:1403] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.154s	user 0.049s	sys 0.045s
I20260709 12:56:01.926017 13734 raft_consensus.cc:359] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.926719 13734 raft_consensus.cc:385] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.926980 13734 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.915714 14008 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:01.927879 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:01.928542 13687 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.930872 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.933215 14008 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:01.933692 14008 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:01.934475 14008 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.935206 13687 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:01.936302 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:01.936924 13691 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.939997 13733 raft_consensus.cc:359] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.940654 13733 raft_consensus.cc:385] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.940907 13733 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.941501 13733 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.915623 14007 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.927618 13734 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.945287 13734 ts_tablet_manager.cc:1434] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:56:01.948414 13734 tablet_bootstrap.cc:492] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:01.948686 13733 ts_tablet_manager.cc:1434] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.003s
I20260709 12:56:01.949642 13733 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:01.918910 14014 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.955971 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:01.956746 13540 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:01.958115 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:01.959113 13734 tablet_bootstrap.cc:654] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.964483 13691 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:01.964890 13733 tablet_bootstrap.cc:654] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:01.965646 13735 tablet_bootstrap.cc:492] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:01.965607 13582 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:01.966225 13735 ts_tablet_manager.cc:1403] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.168s	user 0.015s	sys 0.004s
I20260709 12:56:01.966434 14007 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:01.967190 14007 raft_consensus.cc:697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:01.968428 14051 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:01.968879 14051 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.969769 13735 raft_consensus.cc:359] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.970670 13735 raft_consensus.cc:385] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:01.970877 14051 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:01.971011 13735 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:01.968003 14007 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.971890 13735 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.976328 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:01.977073 13687 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:01.964583 13540 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:01.980908 13735 ts_tablet_manager.cc:1434] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 12:56:01.981638 13899 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:01.982143 13899 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:01.983333 13899 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:01.992354 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:01.982124 13735 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.005985 13734 tablet_bootstrap.cc:492] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.006459 13734 ts_tablet_manager.cc:1403] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.000s
I20260709 12:56:02.005633 13439 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:02.009837 13440 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:02.020776 13584 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:02.021981 14007 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.023293 13734 raft_consensus.cc:359] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.024099 13734 raft_consensus.cc:385] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.024376 13734 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.030385 13441 catalog_manager.cc:5697] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:02.032909 14007 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.036733 13735 tablet_bootstrap.cc:654] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.025030 13734 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.041785 13734 ts_tablet_manager.cc:1434] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:56:02.042762 13734 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.046437 13733 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:02.046952 13733 ts_tablet_manager.cc:1403] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.098s	user 0.024s	sys 0.010s
I20260709 12:56:02.050079 13733 raft_consensus.cc:359] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.050738 13733 raft_consensus.cc:385] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.051011 13733 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.051641 13733 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.053876 13733 ts_tablet_manager.cc:1434] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:56:02.054760 13733 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:02.061836 13734 tablet_bootstrap.cc:654] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.067899 13733 tablet_bootstrap.cc:654] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.070906 13735 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.071329 14014 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:02.073460 13735 ts_tablet_manager.cc:1403] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.000s
I20260709 12:56:02.076795 13441 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
W20260709 12:56:02.080380 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:56:02.081923 13735 raft_consensus.cc:359] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.082722 13735 raft_consensus.cc:385] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.083014 13735 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.083729 13735 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.094185 13733 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:02.094683 13733 ts_tablet_manager.cc:1403] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.003s
I20260709 12:56:02.097352 13733 raft_consensus.cc:359] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.098068 13733 raft_consensus.cc:385] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.098474 13733 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.099133 13733 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.100991 13733 ts_tablet_manager.cc:1434] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:56:02.101917 13733 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:02.107568 13733 tablet_bootstrap.cc:654] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.112018 13735 ts_tablet_manager.cc:1434] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:56:02.113692 13735 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.119994 13733 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:02.120484 13733 ts_tablet_manager.cc:1403] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:56:02.123131 13733 raft_consensus.cc:359] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.123788 13733 raft_consensus.cc:385] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.124076 13733 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.124861 13733 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.127074 13733 ts_tablet_manager.cc:1434] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:56:02.129364 13735 tablet_bootstrap.cc:654] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.135706 13733 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:02.142321 13733 tablet_bootstrap.cc:654] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.148612 14051 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.149430 14051 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.151376 13733 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:02.151993 13733 ts_tablet_manager.cc:1403] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.007s
I20260709 12:56:02.153471 14014 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.154089 14014 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.155230 13733 raft_consensus.cc:359] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.156309 13733 raft_consensus.cc:385] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.156571 13733 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.156855 14014 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.157707 13733 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.159729 13733 ts_tablet_manager.cc:1434] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:56:02.158880 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.165793 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:02.167492 13509 leader_election.cc:343] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.167650 13691 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.168857 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.169827 14003 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.170356 14003 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.170706 14003 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.170846 14051 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.159526 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.171891 13540 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:02.167524 13733 tablet_bootstrap.cc:492] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
I20260709 12:56:02.180776 13734 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.181706 13734 ts_tablet_manager.cc:1403] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.139s	user 0.032s	sys 0.043s
I20260709 12:56:02.181797 13733 tablet_bootstrap.cc:654] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.182145 14008 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.182608 14008 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.185159 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.185899 13691 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:02.185698 13734 raft_consensus.cc:359] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.186298 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.186597 13734 raft_consensus.cc:385] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.186970 13734 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.169790 13735 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.188138 13735 ts_tablet_manager.cc:1403] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.004s
I20260709 12:56:02.188222 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:02.187943 13734 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.189220 14007 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.190723 13735 raft_consensus.cc:359] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.191342 13735 raft_consensus.cc:385] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.191584 13735 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.193856 14003 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.196929 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:02.197113 14007 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:56:02.189282 13658 leader_election.cc:343] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.197433 14008 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:02.201573 13788 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.198928 13734 ts_tablet_manager.cc:1434] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:56:02.198766 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.202093 13788 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.202956 13734 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.197490 13691 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.207034 13733 tablet_bootstrap.cc:492] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:02.204262 13788 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:02.207870 13733 ts_tablet_manager.cc:1403] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.008s
W20260709 12:56:02.208495 13659 leader_election.cc:343] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.208923 13659 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.210832 13788 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.211624 13734 tablet_bootstrap.cc:654] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.204841 13735 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.215174 13733 raft_consensus.cc:359] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.205761 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.226261 13733 raft_consensus.cc:385] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.226591 13733 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.204771 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:02.227156 13658 leader_election.cc:343] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.217993 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.227385 13733 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.218922 13614 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:02.228164 13509 leader_election.cc:343] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.229064 13733 ts_tablet_manager.cc:1434] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:56:02.204921 14003 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.222190 13691 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.223783 13735 ts_tablet_manager.cc:1434] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 12:56:02.225926 13734 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.236011 13734 ts_tablet_manager.cc:1403] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.000s
I20260709 12:56:02.230088 13733 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Bootstrap starting.
W20260709 12:56:02.231266 13659 leader_election.cc:343] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.237654 13659 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.239127 13734 raft_consensus.cc:359] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.239675 14031 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.240209 13735 tablet_bootstrap.cc:492] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.237067 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.242223 14014 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:02.242697 14014 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:02.243500 14014 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.247133 13733 tablet_bootstrap.cc:654] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.233585 13788 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.251878 13788 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.253832 13735 tablet_bootstrap.cc:654] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.254278 13788 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:02.239795 13734 raft_consensus.cc:385] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.255528 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.255795 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.255743 13734 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
W20260709 12:56:02.257709 13658 leader_election.cc:343] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:56:02.258534 13659 leader_election.cc:343] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.258319 13734 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.259289 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.261318 13734 ts_tablet_manager.cc:1434] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:56:02.262495 13734 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.264622 13788 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.269117 13734 tablet_bootstrap.cc:654] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.283311 13734 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.284277 13734 ts_tablet_manager.cc:1403] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 12:56:02.285096 13733 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: No bootstrap required, opened a new log
I20260709 12:56:02.285640 13733 ts_tablet_manager.cc:1403] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.002s
I20260709 12:56:02.287794 13735 tablet_bootstrap.cc:492] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.288297 13735 ts_tablet_manager.cc:1403] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.000s
I20260709 12:56:02.289041 13733 raft_consensus.cc:359] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.289848 13733 raft_consensus.cc:385] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.290304 13733 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.291067 13733 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.294131 13733 ts_tablet_manager.cc:1434] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:56:02.291240 13735 raft_consensus.cc:359] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.295131 13735 raft_consensus.cc:385] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.295487 13735 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.296303 13735 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.297091 13734 raft_consensus.cc:359] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.297781 13734 raft_consensus.cc:385] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.298058 13734 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.298882 13734 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.300490 13735 ts_tablet_manager.cc:1434] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:56:02.301405 13734 ts_tablet_manager.cc:1434] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.004s
I20260709 12:56:02.302021 13735 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.304553 13734 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.304131 13440 catalog_manager.cc:5697] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:02.314836 13735 tablet_bootstrap.cc:654] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.317298 13734 tablet_bootstrap.cc:654] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.323096 13735 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.323567 13735 ts_tablet_manager.cc:1403] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.022s	user 0.002s	sys 0.009s
I20260709 12:56:02.323747 13734 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.324276 13734 ts_tablet_manager.cc:1403] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.002s
I20260709 12:56:02.325836 13735 raft_consensus.cc:359] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.326550 13735 raft_consensus.cc:385] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.326833 13735 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.327611 13735 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.327730 13734 raft_consensus.cc:359] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.328619 13734 raft_consensus.cc:385] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.328949 13734 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.329501 13614 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.329771 13735 ts_tablet_manager.cc:1434] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:56:02.330775 13735 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.329790 13734 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.332990 13734 ts_tablet_manager.cc:1434] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:56:02.333956 13734 tablet_bootstrap.cc:492] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.337266 13735 tablet_bootstrap.cc:654] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.337332 14014 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.337966 14014 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.338384 13788 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:02.341543 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.342209 13691 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.343209 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.343447 13506 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.344090 13616 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.344404 14003 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.344882 14003 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.345283 14003 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.348176 14014 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.353017 13734 tablet_bootstrap.cc:654] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.353318 14003 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.359189 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:02.360072 13619 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.360600 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:02.361375 13691 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.364236 14003 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.366439 13734 tablet_bootstrap.cc:492] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.367007 13734 ts_tablet_manager.cc:1403] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.004s
I20260709 12:56:02.369735 13734 raft_consensus.cc:359] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.370374 13734 raft_consensus.cc:385] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.370644 13734 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.371308 13734 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.374356 13734 ts_tablet_manager.cc:1434] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:56:02.375540 13734 tablet_bootstrap.cc:492] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.382210 13734 tablet_bootstrap.cc:654] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.399114 13788 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:02.405441 13691 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.410404 14031 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.419755 14031 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.422739 14031 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:02.428745 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:02.430969 13659 leader_election.cc:343] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.433192 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:02.444661 13658 leader_election.cc:343] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.445258 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.460822 13782 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.455636 13506 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.457068 14016 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:02.462040 14003 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:02.462787 14003 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:02.464033 14003 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.469403 13782 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.470307 13782 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.474774 13782 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:02.478961 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:02.484375 13659 leader_election.cc:343] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.492616 13619 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.497318 13619 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.459998 13616 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.517012 13735 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.517544 13735 ts_tablet_manager.cc:1403] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.187s	user 0.061s	sys 0.078s
I20260709 12:56:02.518311 13734 tablet_bootstrap.cc:492] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.520825 13734 ts_tablet_manager.cc:1403] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.145s	user 0.060s	sys 0.059s
I20260709 12:56:02.521307 13788 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:02.522650 14051 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:02.523896 13734 raft_consensus.cc:359] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.524657 13734 raft_consensus.cc:385] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.524971 13734 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.525616 13734 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.527956 13734 ts_tablet_manager.cc:1434] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:56:02.529284 13734 tablet_bootstrap.cc:492] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.532140 13782 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.532785 13782 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.533898 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.536213 14051 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.536710 14003 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.539883 14003 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.547528 14016 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:56:02.547348 13658 leader_election.cc:343] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.563413 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.564429 14031 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.545257 13735 raft_consensus.cc:359] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.585412 14051 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.588825 13782 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:02.589741 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:02.591231 13659 leader_election.cc:343] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.591518 14014 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.592010 14014 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.593945 14014 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.585038 13735 raft_consensus.cc:385] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.595032 14031 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:02.594915 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 70604.9us
I20260709 12:56:02.595613 14031 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.598222 14031 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:02.602113 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:02.603909 13659 leader_election.cc:343] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.604880 14016 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.605403 14016 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.605705 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.606251 13687 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.607364 14016 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.595219 13735 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.611019 13735 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.612740 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.615289 14031 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:02.617503 14018 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.618054 14018 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.621037 14018 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.621379 14003 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.623754 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.625506 13735 ts_tablet_manager.cc:1434] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.108s	user 0.004s	sys 0.013s
I20260709 12:56:02.625311 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.626117 13543 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:02.626724 14051 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.630309 13735 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.631494 13734 tablet_bootstrap.cc:654] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
W20260709 12:56:02.634528 13658 leader_election.cc:343] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.635223 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.639555 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.640614 13782 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.640708 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:02.641464 13658 leader_election.cc:343] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.641677 14016 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.643496 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.659054 14016 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.659322 13693 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.659533 14016 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.648839 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.654218 13735 tablet_bootstrap.cc:654] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.660408 13687 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.665091 13506 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.666925 14003 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.667379 14003 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.667214 14016 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.667807 14003 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.670497 13658 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.671739 14018 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:02.670800 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.673055 14107 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.673398 14107 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.673723 14107 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.682179 14003 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.683581 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:02.682168 14107 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.684122 13687 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.685832 14003 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:02.686697 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 48597.3us
I20260709 12:56:02.687705 14107 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.689767 14031 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.690624 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:02.690892 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.691192 13693 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.691349 14016 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.695191 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.695490 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.696010 13690 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:02.696223 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:02.696728 13691 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.703549 13734 tablet_bootstrap.cc:492] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.704613 13691 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.705781 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.707069 13734 ts_tablet_manager.cc:1403] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.178s	user 0.028s	sys 0.019s
I20260709 12:56:02.707124 14107 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:02.707868 14107 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:02.707345 13687 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.709187 13693 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.710181 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.711676 14003 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:02.712388 14003 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:02.713222 14003 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.716580 13506 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.717422 14104 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:02.717891 14104 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:02.718281 13440 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:02.718637 14104 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.719559 13734 raft_consensus.cc:359] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.724036 13734 raft_consensus.cc:385] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.724460 13734 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.708842 14107 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.725409 13734 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:02.734668 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 82038.5us
I20260709 12:56:02.736181 13584 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:02.738327 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.738818 13734 ts_tablet_manager.cc:1434] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:56:02.740219 14113 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:02.740823 14113 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:02.741425 14113 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:02.744501 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.745178 13614 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.749817 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:02.750406 13614 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.751039 14113 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.752552 13734 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.757939 13614 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.764711 13734 tablet_bootstrap.cc:654] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.765384 13735 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.765872 13735 ts_tablet_manager.cc:1403] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.136s	user 0.019s	sys 0.043s
I20260709 12:56:02.767036 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:02.769608 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:02.770166 13619 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.774626 13735 raft_consensus.cc:359] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.775337 13735 raft_consensus.cc:385] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.780786 13735 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.781540 13735 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.775218 13440 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:02.783676 13735 ts_tablet_manager.cc:1434] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:56:02.785136 13735 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.787333 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:02.789758 13619 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.791291 13616 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:02.794152 13509 leader_election.cc:343] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.794849 13735 tablet_bootstrap.cc:654] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.790840 14113 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.791071 13734 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.788121 13440 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:02.797617 13734 ts_tablet_manager.cc:1403] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.000s
W20260709 12:56:02.799723 13509 leader_election.cc:343] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.803295 13735 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.803050 13734 raft_consensus.cc:359] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.803721 13734 raft_consensus.cc:385] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.803767 13735 ts_tablet_manager.cc:1403] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.008s
I20260709 12:56:02.804114 13734 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.800781 13440 catalog_manager.cc:5697] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
W20260709 12:56:02.804893 13509 leader_election.cc:343] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.804791 13734 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.806447 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:02.806630 13735 raft_consensus.cc:359] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.807119 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:02.807859 14112 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:02.808472 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:02.807307 13735 raft_consensus.cc:385] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.809142 14112 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:02.809407 13734 ts_tablet_manager.cc:1434] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:56:02.809834 13735 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.810866 13735 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.812080 14003 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:02.813237 13735 ts_tablet_manager.cc:1434] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:56:02.813726 13734 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.814211 13735 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.820817 13734 tablet_bootstrap.cc:654] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.821480 13735 tablet_bootstrap.cc:654] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.824795 14104 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.825271 14104 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.834849 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.835551 13693 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.838121 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.839036 14003 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.839442 14003 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.839799 14003 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.846374 13734 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.846925 13734 ts_tablet_manager.cc:1403] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.000s
I20260709 12:56:02.849442 14003 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.852210 13734 raft_consensus.cc:359] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.852960 13734 raft_consensus.cc:385] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.852986 13735 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.853411 13734 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.853734 13735 ts_tablet_manager.cc:1403] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.012s
I20260709 12:56:02.854130 13734 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.854351 14106 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:02.856388 13734 ts_tablet_manager.cc:1434] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:56:02.857371 13734 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.856969 13735 raft_consensus.cc:359] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.858340 13735 raft_consensus.cc:385] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.858714 13735 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.859355 13735 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.861688 13735 ts_tablet_manager.cc:1434] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:56:02.862789 13735 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.864490 13734 tablet_bootstrap.cc:654] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.868050 14104 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.869324 13735 tablet_bootstrap.cc:654] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.890954 13735 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:02.891084 14003 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.891501 13735 ts_tablet_manager.cc:1403] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.029s	user 0.020s	sys 0.005s
I20260709 12:56:02.897038 13734 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:02.897565 13734 ts_tablet_manager.cc:1403] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.040s	user 0.022s	sys 0.011s
I20260709 12:56:02.900300 13734 raft_consensus.cc:359] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.901098 13734 raft_consensus.cc:385] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.901183 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:02.901408 13734 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:02.902087 13734 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.904605 13734 ts_tablet_manager.cc:1434] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:56:02.905612 13734 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:02.905674 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:02.909384 13509 leader_election.cc:343] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.910187 13735 raft_consensus.cc:359] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.910934 13735 raft_consensus.cc:385] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:02.911217 13735 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
W20260709 12:56:02.912539 13509 leader_election.cc:343] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:02.912676 13734 tablet_bootstrap.cc:654] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.916422 14112 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.916929 14112 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.918134 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:02.918716 13693 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.919091 14112 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.921382 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.921787 14104 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:02.922348 13543 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:56:02.924111 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:02.924857 14112 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.925227 14112 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.925590 14112 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.911862 13735 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.926955 13735 ts_tablet_manager.cc:1434] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 12:56:02.927866 13735 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:02.937053 13693 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:02.939299 13735 tablet_bootstrap.cc:654] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:02.943048 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:02.944828 14104 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:02.945722 14104 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:02.946835 14104 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.954519 14113 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.955245 14113 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.955780 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.956430 13693 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:56:02.958119 14113 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.974105 14112 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.974712 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:02.979218 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:02.980100 13543 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:02.985962 14106 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:02.986472 14106 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:02.992115 14106 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:02.993558 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:02.994486 13693 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:02.996814 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:02.997658 14106 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:02.998282 14106 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:02.998845 14106 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.019050 14112 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.020551 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:03.023531 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.024566 13693 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:03.032668 13734 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:03.033305 13734 ts_tablet_manager.cc:1403] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.128s	user 0.047s	sys 0.072s
I20260709 12:56:03.033080 14106 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.031648 13543 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:03.036751 13734 raft_consensus.cc:359] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.037393 13734 raft_consensus.cc:385] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:03.037662 13734 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:03.038421 13734 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.040364 13734 ts_tablet_manager.cc:1434] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:56:03.041239 13734 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:03.042902 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.043500 13688 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.046727 14015 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.047228 14015 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.049221 14015 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.053628 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.054303 13693 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
W20260709 12:56:03.055505 13509 leader_election.cc:343] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.057152 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:03.059262 13734 tablet_bootstrap.cc:654] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:03.061579 14015 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:03.062057 14015 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:03.062515 14015 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:03.069861 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:03.071458 13688 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:03.073999 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:03.074668 13619 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:03.075559 13582 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:03.079341 13735 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:03.079830 13735 ts_tablet_manager.cc:1403] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.152s	user 0.039s	sys 0.035s
I20260709 12:56:03.082862 14015 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.084586 14051 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.097630 14051 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:03.102631 13582 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:03.082381 13735 raft_consensus.cc:359] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.104287 13735 raft_consensus.cc:385] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:03.104712 13735 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:03.105657 13735 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.108022 14015 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.113456 14113 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:03.114276 14113 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:03.115414 14113 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.116948 14106 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.120921 14136 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.121414 14136 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.123598 14136 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.127414 13735 ts_tablet_manager.cc:1434] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 12:56:03.128495 13735 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:03.128785 14136 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.129354 14136 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.142807 14136 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.143224 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.143800 13693 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.145752 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:03.146341 13619 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:56:03.146528 13509 leader_election.cc:343] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.148046 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:03.147641 13439 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:03.152232 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.152936 13690 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:03.154672 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:03.155298 13616 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:03.163264 14008 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.163848 14008 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.166188 14008 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:03.167901 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.168573 13688 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:03.169811 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.170408 13684 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:03.171986 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:03.172413 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:03.172689 13543 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:03.172962 13616 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
W20260709 12:56:03.173224 13509 leader_election.cc:343] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.174124 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:03.174924 14031 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:03.174993 13619 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:03.175312 14031 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:03.175669 14031 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:03.182703 13734 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:03.183244 13734 ts_tablet_manager.cc:1403] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.142s	user 0.037s	sys 0.043s
I20260709 12:56:03.184015 14031 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.184995 13688 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:03.186024 13734 raft_consensus.cc:359] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.186715 13734 raft_consensus.cc:385] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:03.186966 13734 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:03.187067 13693 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:03.187646 13734 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.188771 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:03.189218 13734 ts_tablet_manager.cc:1434] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:56:03.189358 13619 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:03.190505 13734 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:03.192190 14135 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.192165 14134 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:03.192880 14135 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.194067 13509 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:03.195180 14135 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.195216 14137 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:03.196295 14137 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:03.196609 14137 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:03.196969 13734 tablet_bootstrap.cc:654] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:03.203374 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.204743 13686 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:03.205787 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:03.206964 14134 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:03.207598 14134 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:03.207948 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:03.208686 14136 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:03.209192 14136 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:03.209621 14136 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.215060 13614 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:03.208778 14134 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.227237 14113 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.227757 14113 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.228067 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:03.228129 14137 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.229949 14113 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.230974 14137 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.232059 13543 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 2: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:03.239846 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:03.222836 14136 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.242638 13619 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:03.243594 14136 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.247064 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:03.249591 14133 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:03.250128 14133 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:03.251104 14133 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.251616 14031 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:03.256258 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:03.257387 13613 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:03.249274 13735 tablet_bootstrap.cc:654] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:03.261349 14135 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:03.265290 13506 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:03.267118 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.268440 13688 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:56:03.267123 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:03.273061 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56814.1us
I20260709 12:56:03.273192 14014 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:03.273660 14014 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:03.273994 14014 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.274071 13441 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:03.276525 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:03.277436 13536 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 0.
I20260709 12:56:03.280198 13584 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:03.281033 14051 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:03.281435 14051 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:03.281756 14051 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.284338 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.284953 13684 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:03.293345 13509 leader_election.cc:343] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.294343 14135 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:03.295315 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.296300 13686 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:03.302094 14014 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.303342 14113 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.304033 14113 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.304868 14014 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.306625 14113 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.310536 13734 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:03.311053 13734 ts_tablet_manager.cc:1403] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.121s	user 0.041s	sys 0.069s
I20260709 12:56:03.311473 14051 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.308385 13684 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:03.314013 13734 raft_consensus.cc:359] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.315318 13734 raft_consensus.cc:385] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:03.315615 13734 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:03.316571 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:03.316874 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:03.317444 13540 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.317847 13734 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.319937 13734 ts_tablet_manager.cc:1434] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 12:56:03.320686 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:03.322320 13734 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:03.322645 14137 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:03.323268 14137 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:03.324777 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.324944 14135 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:03.325384 13684 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.324146 14137 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.326992 13540 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:03.328302 14113 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.329211 13584 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:03.328891 14113 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.329937 14112 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:03.331334 14113 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.332653 14051 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.337172 13509 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:03.342036 13659 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:03.359664 13509 leader_election.cc:343] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.362854 13735 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
W20260709 12:56:03.363138 13489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:56:03.363366 13735 ts_tablet_manager.cc:1403] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.235s	user 0.011s	sys 0.007s
I20260709 12:56:03.366338 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:03.355398 14112 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:03.359103 14106 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:56:03.364777 14141 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:56:03.364549 14031 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:03.371587 14031 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:03.358893 14140 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:56:03.365520 13899 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:56:03.372212 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.372514 14031 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.372960 14140 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:03.373137 13685 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:03.373392 14140 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:03.374300 14140 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.375952 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.376569 13692 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.367096 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:03.371138 14112 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:03.404259 13509 leader_election.cc:343] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.386915 14140 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:03.411504 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.421752 13688 raft_consensus.cc:2393] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:56:03.422870 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 66353.6us
I20260709 12:56:03.427883 13584 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:03.434168 13582 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:03.435824 14051 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.445607 13734 tablet_bootstrap.cc:654] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:03.469661 14151 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:03.517632 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 89830.8us
I20260709 12:56:03.473063 13684 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:03.530937 13735 raft_consensus.cc:359] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.531900 13735 raft_consensus.cc:385] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:03.534049 13506 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:03.532289 13735 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:03.535352 13735 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.501053 14136 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:56:03.542713 14016 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:03.554034 13692 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:03.553345 13899 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:56:03.570524 14051 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:03.564687 14016 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:03.589453 14016 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.604772 14135 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:56:03.621026 14151 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:03.627678 13735 ts_tablet_manager.cc:1434] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.264s	user 0.014s	sys 0.004s
I20260709 12:56:03.626806 14003 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.634500 14003 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.636646 14003 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.640075 13735 tablet_bootstrap.cc:492] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:03.652251 13734 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:03.652715 13734 ts_tablet_manager.cc:1403] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.331s	user 0.023s	sys 0.013s
I20260709 12:56:03.651074 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.634451 13439 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:03.658622 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:03.659168 13439 catalog_manager.cc:5697] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 2, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:03.660467 13439 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:03.662343 13686 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:03.667040 13617 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:56:03.669696 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:03.671376 13509 leader_election.cc:343] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.672912 14003 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:03.673372 14003 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:03.673730 14003 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.675261 14134 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:03.687808 14151 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.688360 14151 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.691126 14151 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.694388 14112 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:03.696763 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:03.700058 13735 tablet_bootstrap.cc:654] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
W20260709 12:56:03.700232 14157 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.004s
I20260709 12:56:03.701162 13734 raft_consensus.cc:359] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.702668 13734 raft_consensus.cc:385] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:03.702999 13734 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:03.703792 13734 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.704818 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.706209 13734 ts_tablet_manager.cc:1434] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 12:56:03.729527 14151 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:56:03.729907 14163 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:56:03.714579 13688 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:03.738404 13734 tablet_bootstrap.cc:492] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:03.749577 14112 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:03.750169 14112 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:03.751309 14160 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 12:56:03.753011 13685 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:56:03.754117 14112 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.754741 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:03.713811 14003 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.759765 14133 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:03.767764 13440 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 0 to 2, leader changed from <none> to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:03.772971 14134 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:03.779544 13734 tablet_bootstrap.cc:654] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:03.781411 14003 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:03.814723 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:03.815780 13693 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.824851 13582 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:03.841872 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:03.842406 13735 tablet_bootstrap.cc:492] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:03.842721 13684 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 1: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:03.842950 13735 ts_tablet_manager.cc:1403] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.203s	user 0.040s	sys 0.073s
I20260709 12:56:03.838660 13441 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 0 to 1, leader changed from <none> to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:03.846223 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:03.837314 14051 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:56:03.858211 13509 leader_election.cc:343] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:56:03.860944 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:03.857580 14106 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:56:03.863157 14112 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:03.868166 13693 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:03.872196 14112 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:03.874571 14051 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:56:03.875604 13543 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:56:03.886113 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:03.887521 14016 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:03.888005 14016 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:03.888765 13439 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:03.888780 14016 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.905004 13735 raft_consensus.cc:359] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.905767 13735 raft_consensus.cc:385] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:03.906037 13735 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:03.910564 13735 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:03.916047 13735 ts_tablet_manager.cc:1434] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.003s
I20260709 12:56:03.917014 13735 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Bootstrap starting.
I20260709 12:56:03.883518 14155 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:56:03.925900 14133 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:03.938971 13619 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:56:03.940919 14008 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:03.910003 14003 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 12:56:04.018491 14133 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.029072 14146 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.078274 14003 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.090771 13734 tablet_bootstrap.cc:492] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:04.091357 13734 ts_tablet_manager.cc:1403] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.353s	user 0.049s	sys 0.038s
I20260709 12:56:04.093868 14003 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:04.095896 13735 tablet_bootstrap.cc:654] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:04.097553 13734 raft_consensus.cc:359] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.103222 13734 raft_consensus.cc:385] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:04.103601 13734 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:04.107520 13734 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.110175 13734 ts_tablet_manager.cc:1434] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.001s
I20260709 12:56:04.112201 13734 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:04.118991 13440 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:04.120529 14003 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:04.121172 14003 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.128528 13734 tablet_bootstrap.cc:654] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:04.140069 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:04.141026 13611 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:04.146662 14003 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.148274 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:04.149379 14015 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:04.149860 14015 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:04.150281 14015 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:04.151415 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:04.152328 13685 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:04.164453 14146 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.166011 14015 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.169435 13735 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: No bootstrap required, opened a new log
I20260709 12:56:04.169962 13735 ts_tablet_manager.cc:1403] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent bootstrapping tablet: real 0.253s	user 0.009s	sys 0.002s
I20260709 12:56:04.169895 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:04.170936 13685 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:04.172179 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:04.172840 13611 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:04.189014 13735 raft_consensus.cc:359] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.189893 13735 raft_consensus.cc:385] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:04.190203 13735 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Initialized, Role: FOLLOWER
I20260709 12:56:04.190958 13735 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.187484 13611 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:04.168083 14015 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.204490 13685 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:04.233006 13735 ts_tablet_manager.cc:1434] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.004s
I20260709 12:56:04.235313 13734 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:04.236267 13734 ts_tablet_manager.cc:1403] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.125s	user 0.047s	sys 0.031s
I20260709 12:56:04.236172 14003 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:04.236765 14003 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.237178 13506 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:04.238718 14003 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.238881 13734 raft_consensus.cc:359] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.239601 13734 raft_consensus.cc:385] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:04.239856 13734 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:04.240211 14015 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:04.240676 14015 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:04.240507 13734 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.241441 14015 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.242975 13734 ts_tablet_manager.cc:1434] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:56:04.250572 13734 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Bootstrap starting.
I20260709 12:56:04.254907 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:04.255602 13685 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:04.256814 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:04.257470 13611 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 0.
I20260709 12:56:04.257089 13734 tablet_bootstrap.cc:654] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Neither blocks nor log segments found. Creating new log.
I20260709 12:56:04.258432 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:04.259155 14106 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:56:04.259618 14106 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:04.260059 14106 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:04.268154 14106 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.270846 14106 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.272313 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:04.273389 13686 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:04.280350 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:04.281164 13611 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:04.289439 13611 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:04.290449 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:04.290685 13686 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:04.291262 14015 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:56:04.292665 14015 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:04.293489 14015 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.304757 13440 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:04.309718 13734 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: No bootstrap required, opened a new log
I20260709 12:56:04.310160 13734 ts_tablet_manager.cc:1403] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.015s
I20260709 12:56:04.312383 13734 raft_consensus.cc:359] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.312875 13734 raft_consensus.cc:385] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:56:04.313122 13734 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Initialized, Role: FOLLOWER
I20260709 12:56:04.313715 13734 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.315753 13734 ts_tablet_manager.cc:1434] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:56:04.321250 13440 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:04.482147 13619 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:56:04.487200 14151 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:04.487772 14151 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.488013 14106 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.493578 14151 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.494930 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:04.496975 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:04.497704 13685 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:04.499121 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:04.504709 14151 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:04.506502 14151 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:04.507014 14151 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:04.513782 14106 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:56:04.516251 14151 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.524632 14151 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.525588 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:04.528265 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:04.533228 14106 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:04.533793 14106 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.535383 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:04.536444 14146 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:04.540042 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:04.540846 13619 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:04.542433 14106 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.544256 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:04.544970 13685 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:04.546124 13506 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:04.547780 14106 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:04.548699 14106 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:04.549053 14106 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:04.555860 14106 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.558346 14106 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.559320 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:04.559967 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:04.560853 13611 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:04.566849 13685 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:04.568073 13506 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:04.568856 14106 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:04.569365 14106 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:04.570212 14106 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.585784 13440 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 2, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:04.661752 14106 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.680423 14015 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.800278 14146 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:04.800808 14146 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.802843 14146 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.803596 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:04.803857 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:04.804525 13689 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:04.805622 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:04.806322 14146 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:04.806668 14146 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:04.806973 14146 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:04.811985 14146 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.813560 14146 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:04.814822 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:04.814431 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:04.815764 13689 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:04.820544 13689 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:04.821508 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:04.822209 14146 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:04.822630 14146 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:04.823415 14146 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.834052 13441 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:04.843564 14003 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.857795 14136 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:56:04.978665 13613 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:04.979951 14003 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:04.980671 14136 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:04.981012 14136 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:04.981617 14136 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:04.982720 14136 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:05.009332 13689 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:05.010808 14136 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:05.011534 14003 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:05.011848 14003 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:05.012511 14003 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:05.013424 14003 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:05.252228 13540 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:56:05.252511 13540 pending_rounds.cc:85] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 1
I20260709 12:56:05.252691 13540 pending_rounds.cc:107] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:56:05.252871 13540 raft_consensus.cc:2889] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:56:05.254032 14051 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:05.263442 13685 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:05.264894 14051 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:05.601078 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:56:05.630218 14201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:56:05.630389 14200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:56:05.632867 14203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:56:05.633615 12409 server_base.cc:1061] running on GCE node
I20260709 12:56:05.634686 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:56:05.634923 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:56:05.635133 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601765635116 us; error 0 us; skew 500 ppm
I20260709 12:56:05.635687 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:56:05.638641 12409 webserver.cc:533] Webserver started at http://127.12.30.68:35979/ using document root <none> and password file <none>
I20260709 12:56:05.639154 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:56:05.639375 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:56:05.639747 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:56:05.640952 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-3-root/instance:
uuid: "0f34cda059124f839711dbef61bb2c28"
format_stamp: "Formatted at 2026-07-09 12:56:05 on dist-test-slave-mlt7"
I20260709 12:56:05.646828 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:56:05.651716 14208 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:56:05.653486 12409 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:56:05.653904 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-3-root
uuid: "0f34cda059124f839711dbef61bb2c28"
format_stamp: "Formatted at 2026-07-09 12:56:05 on dist-test-slave-mlt7"
I20260709 12:56:05.654235 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:56:05.680388 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:56:05.681862 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:56:05.683400 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:56:05.686278 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:56:05.686467 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:56:05.686789 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:56:05.686971 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:56:05.751176 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.68:39527
I20260709 12:56:05.751307 14270 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.68:39527 every 8 connection(s)
I20260709 12:56:05.758190 12409 auto_leader_rebalancer-test.cc:384] add a tserver: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:05.787173 14271 heartbeater.cc:344] Connected to a master server at 127.12.30.126:40047
I20260709 12:56:05.787740 14271 heartbeater.cc:461] Registering TS with master...
I20260709 12:56:05.788589 14271 heartbeater.cc:507] Master 127.12.30.126:40047 requested a full tablet report, sending...
I20260709 12:56:05.790401 13441 ts_manager.cc:194] Registered new tserver with Master: 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:05.791663 13441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41440
I20260709 12:56:06.021667 13782 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:06.022187 13782 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:06.023928 13782 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:06.024535 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:06.024799 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:06.024983 13540 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:06.025452 13613 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:06.025991 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:06.026635 13782 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:06.026937 13782 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:06.027223 13782 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:06.031418 13782 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:06.033097 13782 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:06.034200 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:06.034746 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:06.035251 13613 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:06.041127 13536 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:06.042272 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:06.042994 13782 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:06.043427 13782 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:06.044128 13782 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:06.053319 13441 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:06.248095 14198 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:06.248629 14198 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:06.250941 14198 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:06.251797 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:06.252056 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:06.252745 13536 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:06.253677 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:06.254380 14198 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:06.254809 14198 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:06.255095 14198 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:06.260107 14198 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:06.261790 14198 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:06.262487 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:06.262643 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:06.263185 13536 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:06.268299 13536 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:06.269294 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:06.270004 14198 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:06.270457 14198 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:06.271147 14198 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:06.283601 13441 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:06.549297 13613 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:06.555497 13685 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:06.557828 14284 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:06.558911 13536 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:06.563125 14198 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:06.567711 14051 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.575474 13685 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:56:06.576785 13584 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973866cfc22e48ea931cb1a6fb16872f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:06.577642 13536 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.590276 13441 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:06.608909 13540 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:06.616214 13536 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:06.618108 13782 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:06.618965 14031 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:06.619341 14031 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:06.620126 14031 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:06.620586 13613 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:06.621311 14031 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:06.621879 14003 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:06.622597 13685 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:06.626406 14003 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:56:06.643954 13508 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75988f1a2dda4bd6962cd5f3ad1c304a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:06.643764 14015 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.650666 13611 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.650929 13689 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.673463 13439 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.685532 13689 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:56:06.686088 13689 pending_rounds.cc:85] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 1
I20260709 12:56:06.686318 13689 pending_rounds.cc:107] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:56:06.686519 13689 raft_consensus.cc:2889] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:56:06.687891 14051 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:56:06.707665 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 50388us
I20260709 12:56:06.707813 13693 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:06.743490 13536 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:06.745368 14198 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:06.745993 13611 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:06.747417 13782 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:06.751061 13543 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:06.753394 14031 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:06.769856 13782 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.782155 13543 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.793910 14271 heartbeater.cc:499] Master 127.12.30.126:40047 was elected leader, sending a full tablet report...
W20260709 12:56:06.810974 13658 consensus_peers.cc:597] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5db10903ab7d403ba303d72b7f26dfed. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:06.816507 13613 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.830554 13543 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:06.836573 13441 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:06.892083 13689 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:06.892419 13508 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b47f4cbc5b245adbdd9ebed9c3175f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:06.893092 13613 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:06.893411 14309 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:06.895413 14309 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:56:06.903952 14295 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.905650 13693 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.927222 13613 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:06.927802 13441 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:06.965341 14313 ts_tablet_manager.cc:933] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:06.968794 14313 tablet_copy_client.cc:323] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:06.983319 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 973866cfc22e48ea931cb1a6fb16872f from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:06.983857 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 973866cfc22e48ea931cb1a6fb16872f from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-973866cfc22e48ea931cb1a6fb16872f
I20260709 12:56:06.991034 13629 tablet_copy_source_session.cc:215] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:06.995122 14313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973866cfc22e48ea931cb1a6fb16872f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:07.006949 14313 tablet_copy_client.cc:806] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:07.007633 14313 tablet_copy_client.cc:670] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:07.012025 14313 tablet_copy_client.cc:538] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:07.021793 14313 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:07.049928 14313 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:07.051136 14313 tablet_bootstrap.cc:492] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:07.051982 14313 ts_tablet_manager.cc:1403] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.008s
I20260709 12:56:07.055305 14313 raft_consensus.cc:359] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.056231 14313 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:07.057106 14313 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.060902 14313 ts_tablet_manager.cc:1434] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:56:07.064052 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-973866cfc22e48ea931cb1a6fb16872f received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:07.064604 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-973866cfc22e48ea931cb1a6fb16872f on tablet 973866cfc22e48ea931cb1a6fb16872f with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:07.156944 14313 ts_tablet_manager.cc:933] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:07.159010 14313 tablet_copy_client.cc:323] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:07.175457 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 75988f1a2dda4bd6962cd5f3ad1c304a from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:07.176084 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 75988f1a2dda4bd6962cd5f3ad1c304a from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-75988f1a2dda4bd6962cd5f3ad1c304a
I20260709 12:56:07.183765 13554 tablet_copy_source_session.cc:215] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:07.187235 14313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75988f1a2dda4bd6962cd5f3ad1c304a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:07.203243 14313 tablet_copy_client.cc:806] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:07.204005 14313 tablet_copy_client.cc:670] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:07.212183 14313 tablet_copy_client.cc:538] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:07.222231 14313 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:07.241111 14313 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:07.242013 14313 tablet_bootstrap.cc:492] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:07.242717 14313 ts_tablet_manager.cc:1403] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.004s
I20260709 12:56:07.245590 14313 raft_consensus.cc:359] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.246411 14313 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:07.247032 14313 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.249212 14313 ts_tablet_manager.cc:1434] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:56:07.250765 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-75988f1a2dda4bd6962cd5f3ad1c304a received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:07.251232 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-75988f1a2dda4bd6962cd5f3ad1c304a on tablet 75988f1a2dda4bd6962cd5f3ad1c304a with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:07.319375 14313 ts_tablet_manager.cc:933] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:07.321045 14313 tablet_copy_client.cc:323] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:56:07.334967 13703 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet 5db10903ab7d403ba303d72b7f26dfed from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:56:07.335539 13703 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet 5db10903ab7d403ba303d72b7f26dfed from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-5db10903ab7d403ba303d72b7f26dfed
I20260709 12:56:07.342258 13703 tablet_copy_source_session.cc:215] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:07.345532 14313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db10903ab7d403ba303d72b7f26dfed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:07.358501 14313 tablet_copy_client.cc:806] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:07.359083 14313 tablet_copy_client.cc:670] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:07.364064 14313 tablet_copy_client.cc:538] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:07.371411 14313 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:07.390187 14324 ts_tablet_manager.cc:933] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:07.401005 14313 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:07.401881 14313 tablet_bootstrap.cc:492] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:07.402488 14313 ts_tablet_manager.cc:1403] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.017s
I20260709 12:56:07.405007 14246 raft_consensus.cc:1217] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:07.404946 14313 raft_consensus.cc:359] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.405767 14313 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:07.406322 14313 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.408313 14324 tablet_copy_client.cc:323] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:07.409489 14313 ts_tablet_manager.cc:1434] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:56:07.410432 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:07.410954 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-1b47f4cbc5b245adbdd9ebed9c3175f1
I20260709 12:56:07.411173 13703 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-5db10903ab7d403ba303d72b7f26dfed received from {username='slave'} at 127.0.0.1:40872
I20260709 12:56:07.411624 13703 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-5db10903ab7d403ba303d72b7f26dfed on tablet 5db10903ab7d403ba303d72b7f26dfed with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:07.420595 13554 tablet_copy_source_session.cc:215] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:07.423635 14324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:07.435595 14324 tablet_copy_client.cc:806] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:07.436555 14324 tablet_copy_client.cc:670] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:07.441847 14324 tablet_copy_client.cc:538] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:07.452159 14324 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:07.473783 14324 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:07.474756 14324 tablet_bootstrap.cc:492] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:07.475445 14324 ts_tablet_manager.cc:1403] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.024s	user 0.023s	sys 0.002s
I20260709 12:56:07.530543 14324 raft_consensus.cc:359] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.539040 14324 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:07.540959 14324 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:07.572858 14324 ts_tablet_manager.cc:1434] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.006s
I20260709 12:56:07.587203 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-1b47f4cbc5b245adbdd9ebed9c3175f1 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:07.589190 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-1b47f4cbc5b245adbdd9ebed9c3175f1 on tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:07.725839 14246 raft_consensus.cc:1217] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:07.776306 14246 raft_consensus.cc:1217] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:07.797186 14246 raft_consensus.cc:1217] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:07.892690 14284 raft_consensus.cc:1064] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:07.897305 14284 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:07.905009 14246 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:07.906837 14198 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:07.910717 13693 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:07.913749 13540 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:07.915488 14321 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:07.916690 14284 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:56:07.949216 14284 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:07.950804 13540 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:07.950575 14246 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:07.969888 13685 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:07.977349 13439 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:08.042183 13613 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:08.048471 13540 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.048655 14246 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.050401 14198 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:08.052412 14198 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:08.060675 14321 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.069823 13540 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.078670 14246 raft_consensus.cc:2955] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.081945 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 973866cfc22e48ea931cb1a6fb16872f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:08.086310 13439 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.097791 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 973866cfc22e48ea931cb1a6fb16872f with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:08.102130 14344 tablet_replica.cc:333] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:08.103338 14344 raft_consensus.cc:2243] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:08.104274 14344 raft_consensus.cc:2272] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:08.107839 14344 ts_tablet_manager.cc:1916] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:08.126590 14344 ts_tablet_manager.cc:1929] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:08.127259 14344 log.cc:1199] T 973866cfc22e48ea931cb1a6fb16872f P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/973866cfc22e48ea931cb1a6fb16872f
I20260709 12:56:08.130406 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 973866cfc22e48ea931cb1a6fb16872f (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:08.145262 14329 raft_consensus.cc:1064] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:08.147466 14329 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:08.154417 14246 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.155836 13693 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.156728 13619 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.157840 14332 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:08.161343 14331 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:08.163390 14295 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:08.180773 14331 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.181910 13685 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.205758 14015 raft_consensus.cc:1064] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:08.208876 14015 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:08.219403 14333 raft_consensus.cc:1064] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:08.219015 13438 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.222141 14333 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:08.229635 14246 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.242105 13613 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.251292 13540 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:08.279475 14245 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.283983 14329 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:08.299664 14332 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.317835 14246 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.319586 14244 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.321188 14031 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:08.320287 14245 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.325107 14329 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:08.329594 13540 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.332161 14031 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:56:08.346755 13613 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.360772 14334 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:08.365023 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 75988f1a2dda4bd6962cd5f3ad1c304a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:56:08.365710 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 141312us
I20260709 12:56:08.366997 13617 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.370806 14350 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:08.375883 13689 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:08.381505 13693 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.414033 14334 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.416504 14332 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:08.421653 13539 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.422284 13617 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.423396 13440 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.433809 14332 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:08.440764 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 75988f1a2dda4bd6962cd5f3ad1c304a with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:08.452623 13693 raft_consensus.cc:2955] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.462553 14356 tablet_replica.cc:333] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:08.469239 14244 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.494475 13440 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.498386 14356 raft_consensus.cc:2243] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:08.499262 14356 raft_consensus.cc:2272] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:08.503532 14356 ts_tablet_manager.cc:1916] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:08.510026 14332 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.527482 14356 ts_tablet_manager.cc:1929] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:08.528108 14356 log.cc:1199] T 75988f1a2dda4bd6962cd5f3ad1c304a P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/75988f1a2dda4bd6962cd5f3ad1c304a
I20260709 12:56:08.529958 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 75988f1a2dda4bd6962cd5f3ad1c304a (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:08.531495 13685 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:08.531742 13613 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.536659 14244 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.539057 13617 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.540577 14334 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:08.541415 14333 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:08.544817 13686 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.548475 14244 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.557371 13617 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.555461 14031 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.564400 13440 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.604892 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5db10903ab7d403ba303d72b7f26dfed with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:08.613098 13438 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:08.629192 13539 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:08.630625 14244 raft_consensus.cc:2955] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.636227 14245 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.636917 13685 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:08.638441 14332 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:08.639416 14331 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:08.646838 14295 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.660248 14245 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.669330 13685 raft_consensus.cc:2955] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:08.675773 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet 5db10903ab7d403ba303d72b7f26dfed with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:08.686487 14360 tablet_replica.cc:333] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:08.687666 14360 raft_consensus.cc:2243] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:08.688776 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:08.702418 13439 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:08.702669 14360 raft_consensus.cc:2272] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:08.715101 14360 ts_tablet_manager.cc:1916] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:08.716751 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:08.734397 14356 tablet_replica.cc:333] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:08.735590 14356 raft_consensus.cc:2243] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:08.736304 14356 raft_consensus.cc:2272] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:08.740644 14356 ts_tablet_manager.cc:1916] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:08.766245 14360 ts_tablet_manager.cc:1929] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:08.768128 14360 log.cc:1199] T 5db10903ab7d403ba303d72b7f26dfed P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/5db10903ab7d403ba303d72b7f26dfed
I20260709 12:56:08.769834 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet 5db10903ab7d403ba303d72b7f26dfed (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:08.779799 14356 ts_tablet_manager.cc:1929] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:08.781255 14356 log.cc:1199] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/1b47f4cbc5b245adbdd9ebed9c3175f1
I20260709 12:56:08.782876 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 1b47f4cbc5b245adbdd9ebed9c3175f1 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:09.791030 13693 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:09.799525 13611 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:09.799780 13537 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:09.800698 13658 consensus_peers.cc:597] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc9a7a0b2fe343e48ea3ff1bae778069. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:09.801155 14359 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:09.802403 14363 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:09.820328 14363 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.828207 13537 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.831277 13613 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.845279 13537 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:09.847255 13439 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:56:09.857925 13508 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f125a6a76f974409807014b9ddaa1b8c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:09.858310 13615 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:09.862659 14295 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:09.872931 13693 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:09.876394 14309 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.879123 13615 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.896126 14295 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:09.900239 13537 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:56:09.938252 13508 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f7e1f6a8119482c9f8056bb615edfa8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:09.942740 13615 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:09.942987 13438 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.944379 14375 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:09.943894 13686 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.954864 13693 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:09.973942 14331 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:09.975421 14375 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.977916 13615 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:09.991053 13613 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:09.996770 13438 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:56:10.010746 13584 consensus_peers.cc:597] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a315f47e47184081bd93e74c72dba511. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:10.015329 13693 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:10.017674 13537 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:10.017241 14284 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:10.022679 13693 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:10.023818 14284 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:10.034328 14284 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:10.041276 13693 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:10.043711 13537 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:10.060535 13438 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:10.302440 14385 ts_tablet_manager.cc:933] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:10.304459 14385 tablet_copy_client.cc:323] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:10.306169 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet f125a6a76f974409807014b9ddaa1b8c from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:10.306694 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet f125a6a76f974409807014b9ddaa1b8c from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-f125a6a76f974409807014b9ddaa1b8c
I20260709 12:56:10.314203 13554 tablet_copy_source_session.cc:215] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:10.317700 14385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f125a6a76f974409807014b9ddaa1b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:10.335736 14385 tablet_copy_client.cc:806] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:10.336377 14385 tablet_copy_client.cc:670] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:10.340296 14385 tablet_copy_client.cc:538] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:10.349399 14385 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:10.369180 14385 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:10.370077 14385 tablet_bootstrap.cc:492] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:10.370761 14385 ts_tablet_manager.cc:1403] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.001s
I20260709 12:56:10.372846 14388 ts_tablet_manager.cc:933] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:10.373611 14385 raft_consensus.cc:359] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.374632 14385 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:10.374955 14388 tablet_copy_client.cc:323] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:56:10.375288 14385 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.377066 13703 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet bc9a7a0b2fe343e48ea3ff1bae778069 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:56:10.377619 13703 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet bc9a7a0b2fe343e48ea3ff1bae778069 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-bc9a7a0b2fe343e48ea3ff1bae778069
I20260709 12:56:10.381872 14385 ts_tablet_manager.cc:1434] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.003s
I20260709 12:56:10.383754 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-f125a6a76f974409807014b9ddaa1b8c received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:10.384191 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-f125a6a76f974409807014b9ddaa1b8c on tablet f125a6a76f974409807014b9ddaa1b8c with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:10.391764 13703 tablet_copy_source_session.cc:215] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:10.395944 14388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9a7a0b2fe343e48ea3ff1bae778069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:10.414515 14388 tablet_copy_client.cc:806] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:10.415125 14388 tablet_copy_client.cc:670] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:10.420645 14388 tablet_copy_client.cc:538] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:10.430361 14388 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:10.447837 14388 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:10.448630 14388 tablet_bootstrap.cc:492] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:10.449177 14388 ts_tablet_manager.cc:1403] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.009s
I20260709 12:56:10.451395 14388 raft_consensus.cc:359] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.452042 14388 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:10.452586 14388 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.454636 14388 ts_tablet_manager.cc:1434] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:56:10.457962 13703 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-bc9a7a0b2fe343e48ea3ff1bae778069 received from {username='slave'} at 127.0.0.1:40872
I20260709 12:56:10.458395 13703 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-bc9a7a0b2fe343e48ea3ff1bae778069 on tablet bc9a7a0b2fe343e48ea3ff1bae778069 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:10.472589 14388 ts_tablet_manager.cc:933] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:10.480999 14385 ts_tablet_manager.cc:933] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:10.484095 14388 tablet_copy_client.cc:323] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:10.486562 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet a315f47e47184081bd93e74c72dba511 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:10.487337 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet a315f47e47184081bd93e74c72dba511 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-a315f47e47184081bd93e74c72dba511
I20260709 12:56:10.498566 13629 tablet_copy_source_session.cc:215] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:10.503394 14388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a315f47e47184081bd93e74c72dba511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:10.530035 14388 tablet_copy_client.cc:806] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:10.534085 14388 tablet_copy_client.cc:670] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:10.540518 14388 tablet_copy_client.cc:538] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:10.542671 14385 tablet_copy_client.cc:323] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:10.544543 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 3f7e1f6a8119482c9f8056bb615edfa8 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:10.545113 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 3f7e1f6a8119482c9f8056bb615edfa8 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-3f7e1f6a8119482c9f8056bb615edfa8
I20260709 12:56:10.550889 14388 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:10.557148 13554 tablet_copy_source_session.cc:215] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:10.564273 14385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7e1f6a8119482c9f8056bb615edfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:10.577852 14388 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:10.578806 14388 tablet_bootstrap.cc:492] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:10.579486 14388 ts_tablet_manager.cc:1403] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.007s
I20260709 12:56:10.584666 14388 raft_consensus.cc:359] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.585503 14388 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:10.586161 14388 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.588737 14385 tablet_copy_client.cc:806] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:10.589414 14385 tablet_copy_client.cc:670] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:10.593382 14385 tablet_copy_client.cc:538] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:10.595492 14388 ts_tablet_manager.cc:1434] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.003s
I20260709 12:56:10.599085 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-a315f47e47184081bd93e74c72dba511 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:10.599556 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-a315f47e47184081bd93e74c72dba511 on tablet a315f47e47184081bd93e74c72dba511 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:10.601733 14385 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:10.629438 14385 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:10.630339 14385 tablet_bootstrap.cc:492] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:10.630992 14385 ts_tablet_manager.cc:1403] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.029s	user 0.022s	sys 0.005s
I20260709 12:56:10.633981 14385 raft_consensus.cc:359] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.634800 14385 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:10.635434 14385 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:10.637917 14385 ts_tablet_manager.cc:1434] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:56:10.639456 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-3f7e1f6a8119482c9f8056bb615edfa8 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:10.639837 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-3f7e1f6a8119482c9f8056bb615edfa8 on tablet 3f7e1f6a8119482c9f8056bb615edfa8 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:10.816525 14245 raft_consensus.cc:1217] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:10.873934 14245 raft_consensus.cc:1217] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:10.907087 14245 raft_consensus.cc:1217] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:11.058974 14245 raft_consensus.cc:1217] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:11.345170 14375 raft_consensus.cc:1064] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:11.349514 14375 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:11.367888 13685 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.368423 14245 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.370225 14309 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:11.372306 14309 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:11.379441 13613 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.389170 14309 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:56:11.425364 14359 raft_consensus.cc:1064] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:11.427054 14404 raft_consensus.cc:1064] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:11.428623 14359 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:11.429775 14404 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:11.438547 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 67098.4us
I20260709 12:56:11.452896 13615 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.450686 14375 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.495903 13440 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.496518 14245 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.499398 14363 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:11.499970 13540 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.500881 13685 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.497795 13615 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.504164 14363 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:11.511466 14397 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:11.523218 14244 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.534405 14309 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:11.540400 14246 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.546541 13617 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.549940 14401 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:56:11.589795 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 53073.5us
I20260709 12:56:11.594229 13693 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.602488 14363 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.605374 13613 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.638757 14391 raft_consensus.cc:1064] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:11.642463 14391 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:11.672474 14400 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:11.682845 14246 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.684605 14391 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:11.696194 13693 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.706127 13540 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:11.708387 14398 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:11.713160 14417 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:11.712682 14403 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.719955 13617 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.722298 13540 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.760398 14245 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.780289 14244 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.782945 13537 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:11.797789 14243 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:11.800257 14375 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:11.807744 13613 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:11.800648 13692 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.824522 14375 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:11.827551 14398 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.829480 14246 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.849959 13441 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.851715 13540 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.854501 13441 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.857261 14309 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.872673 14243 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.876425 13613 raft_consensus.cc:2955] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.884536 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f125a6a76f974409807014b9ddaa1b8c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:11.886477 13685 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.902222 13685 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:11.911451 13540 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:11.916741 14243 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:11.911752 13438 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.920862 14359 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:11.913061 13615 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:11.925208 13615 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:11.926144 14359 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:11.927711 14243 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:11.928704 14372 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:11.929883 14309 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:11.935206 13441 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.936434 14397 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.937489 13611 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.942961 14246 raft_consensus.cc:2955] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.958175 14245 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.954417 14372 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.973284 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bc9a7a0b2fe343e48ea3ff1bae778069 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:11.976476 13611 raft_consensus.cc:2955] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:11.989393 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3f7e1f6a8119482c9f8056bb615edfa8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:11.988570 13439 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:12.017519 13617 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:12.024431 14246 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:12.026392 13692 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:12.027753 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9a7a0b2fe343e48ea3ff1bae778069 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:12.027333 13438 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:12.028228 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet f125a6a76f974409807014b9ddaa1b8c with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:12.028681 14417 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:12.031060 14391 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:12.040752 14428 tablet_replica.cc:333] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:12.043076 14428 raft_consensus.cc:2243] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:12.044387 14428 raft_consensus.cc:2272] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:12.047602 14427 tablet_replica.cc:333] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:12.048770 14427 raft_consensus.cc:2243] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:12.052621 14398 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:12.053576 14427 raft_consensus.cc:2272] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:12.056969 14428 ts_tablet_manager.cc:1916] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:12.057456 14427 ts_tablet_manager.cc:1916] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:12.065160 14246 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:12.066058 13685 raft_consensus.cc:2955] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:12.076807 14427 ts_tablet_manager.cc:1929] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:12.077581 14427 log.cc:1199] T bc9a7a0b2fe343e48ea3ff1bae778069 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/bc9a7a0b2fe343e48ea3ff1bae778069
I20260709 12:56:12.079731 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet bc9a7a0b2fe343e48ea3ff1bae778069 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:12.087067 14428 ts_tablet_manager.cc:1929] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:12.087528 14428 log.cc:1199] T f125a6a76f974409807014b9ddaa1b8c P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/f125a6a76f974409807014b9ddaa1b8c
I20260709 12:56:12.091080 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f7e1f6a8119482c9f8056bb615edfa8 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:12.092778 14428 tablet_replica.cc:333] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:12.093006 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet f125a6a76f974409807014b9ddaa1b8c (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:12.093547 14428 raft_consensus.cc:2243] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:12.094270 14428 raft_consensus.cc:2272] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:12.099776 14428 ts_tablet_manager.cc:1916] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:12.105822 13439 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:12.109889 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a315f47e47184081bd93e74c72dba511 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:12.118374 14428 ts_tablet_manager.cc:1929] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:12.118937 14428 log.cc:1199] T 3f7e1f6a8119482c9f8056bb615edfa8 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/3f7e1f6a8119482c9f8056bb615edfa8
I20260709 12:56:12.120576 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 3f7e1f6a8119482c9f8056bb615edfa8 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:12.124486 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet a315f47e47184081bd93e74c72dba511 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:12.125428 14427 tablet_replica.cc:333] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:12.126447 14427 raft_consensus.cc:2243] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:12.127542 14427 raft_consensus.cc:2272] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:12.131171 14427 ts_tablet_manager.cc:1916] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:12.146279 14427 ts_tablet_manager.cc:1929] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:12.146617 14427 log.cc:1199] T a315f47e47184081bd93e74c72dba511 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/a315f47e47184081bd93e74c72dba511
I20260709 12:56:12.148255 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet a315f47e47184081bd93e74c72dba511 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:13.197566 13537 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.203553 13617 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:13.204197 13692 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:13.204420 13508 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85eb0ac09bf7466899637997aae42111. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:13.204926 14430 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.205891 14401 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.214011 14430 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.216446 13692 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.216817 13617 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.233485 13438 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.238610 13537 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:56:13.253140 13508 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0863151795784affb539e2b7a55fc2b8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:13.253870 13611 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:13.254706 13685 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:13.255435 14430 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.257961 14372 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.268712 14372 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.273358 13617 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.281325 13685 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.294514 13537 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.305755 13439 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:56:13.307148 13508 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7d1ab18f48a4689857964b7200b72c9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:13.310791 13617 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:13.312752 14372 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.331835 13686 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:13.339170 14372 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.364787 14440 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.369339 13617 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.379729 13692 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.388954 13441 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.394897 13611 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:56:13.411113 13584 consensus_peers.cc:597] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2181ddb48f3d44d583ad7ecf793222fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:13.412134 13686 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:13.413877 13537 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:13.414585 14398 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.415328 14429 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:13.423112 14398 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.424824 13686 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.427696 13537 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:13.440371 13439 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:13.668692 14450 ts_tablet_manager.cc:933] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:13.672662 14450 tablet_copy_client.cc:323] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:13.675158 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 85eb0ac09bf7466899637997aae42111 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:13.675709 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 85eb0ac09bf7466899637997aae42111 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-85eb0ac09bf7466899637997aae42111
I20260709 12:56:13.683212 13554 tablet_copy_source_session.cc:215] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:13.686869 14450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85eb0ac09bf7466899637997aae42111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:13.701676 14450 tablet_copy_client.cc:806] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:13.702355 14450 tablet_copy_client.cc:670] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:13.706173 14450 tablet_copy_client.cc:538] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:13.713886 14450 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:13.733618 14450 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:13.734336 14450 tablet_bootstrap.cc:492] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:13.734859 14450 ts_tablet_manager.cc:1403] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.002s
I20260709 12:56:13.737238 14450 raft_consensus.cc:359] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.737991 14450 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:13.738657 14450 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.744390 14450 ts_tablet_manager.cc:1434] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.005s
I20260709 12:56:13.749251 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-85eb0ac09bf7466899637997aae42111 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:13.749832 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-85eb0ac09bf7466899637997aae42111 on tablet 85eb0ac09bf7466899637997aae42111 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:13.763306 14450 ts_tablet_manager.cc:933] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:13.765413 14450 tablet_copy_client.cc:323] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:13.767228 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet e7d1ab18f48a4689857964b7200b72c9 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:13.767808 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet e7d1ab18f48a4689857964b7200b72c9 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-e7d1ab18f48a4689857964b7200b72c9
I20260709 12:56:13.775404 13554 tablet_copy_source_session.cc:215] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:13.778647 14450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7d1ab18f48a4689857964b7200b72c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:13.782461 14455 ts_tablet_manager.cc:933] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:13.788122 14455 tablet_copy_client.cc:323] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:13.790001 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 0863151795784affb539e2b7a55fc2b8 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:13.790549 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 0863151795784affb539e2b7a55fc2b8 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-0863151795784affb539e2b7a55fc2b8
I20260709 12:56:13.797200 14450 tablet_copy_client.cc:806] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:13.797798 13554 tablet_copy_source_session.cc:215] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:13.797915 14450 tablet_copy_client.cc:670] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:13.801800 14455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0863151795784affb539e2b7a55fc2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:13.802543 14450 tablet_copy_client.cc:538] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:13.811659 14450 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:13.822741 14455 tablet_copy_client.cc:806] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:13.823395 14455 tablet_copy_client.cc:670] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:13.844000 14455 tablet_copy_client.cc:538] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:13.844894 14450 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:13.845824 14450 tablet_bootstrap.cc:492] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:13.846513 14450 ts_tablet_manager.cc:1403] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.014s
I20260709 12:56:13.849454 14450 raft_consensus.cc:359] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.850288 14450 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:13.850962 14450 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.855825 14450 ts_tablet_manager.cc:1434] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 12:56:13.858150 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-e7d1ab18f48a4689857964b7200b72c9 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:13.858595 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-e7d1ab18f48a4689857964b7200b72c9 on tablet e7d1ab18f48a4689857964b7200b72c9 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:13.867596 14455 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:13.905858 14455 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:13.906826 14455 tablet_bootstrap.cc:492] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:13.907869 14455 ts_tablet_manager.cc:1403] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.040s	user 0.027s	sys 0.006s
I20260709 12:56:13.911106 14455 raft_consensus.cc:359] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.911985 14455 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:13.912649 14455 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:13.915041 14455 ts_tablet_manager.cc:1434] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:56:13.916821 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-0863151795784affb539e2b7a55fc2b8 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:13.917259 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-0863151795784affb539e2b7a55fc2b8 on tablet 0863151795784affb539e2b7a55fc2b8 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:13.948653 14455 ts_tablet_manager.cc:933] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:13.950515 14455 tablet_copy_client.cc:323] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:13.952208 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 2181ddb48f3d44d583ad7ecf793222fa from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:13.952772 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 2181ddb48f3d44d583ad7ecf793222fa from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-2181ddb48f3d44d583ad7ecf793222fa
I20260709 12:56:13.963603 13629 tablet_copy_source_session.cc:215] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:13.966744 14455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2181ddb48f3d44d583ad7ecf793222fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:13.983990 14455 tablet_copy_client.cc:806] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:13.984606 14455 tablet_copy_client.cc:670] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:13.989126 14455 tablet_copy_client.cc:538] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:13.998867 14455 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:14.018908 14455 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:14.020089 14455 tablet_bootstrap.cc:492] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:14.020880 14455 ts_tablet_manager.cc:1403] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.003s
I20260709 12:56:14.024587 14455 raft_consensus.cc:359] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:14.025436 14455 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:14.026086 14455 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:14.031153 14455 ts_tablet_manager.cc:1434] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:56:14.033977 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-2181ddb48f3d44d583ad7ecf793222fa received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:14.034437 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-2181ddb48f3d44d583ad7ecf793222fa on tablet 2181ddb48f3d44d583ad7ecf793222fa with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:14.210330 14245 raft_consensus.cc:1217] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:14.215142 14245 raft_consensus.cc:1217] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:14.335096 14245 raft_consensus.cc:1217] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:14.381253 14245 raft_consensus.cc:1217] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:14.700773 14440 raft_consensus.cc:1064] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:14.703338 14440 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:14.711283 14245 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.712278 13611 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.714480 13686 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.714412 14419 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:14.716310 14419 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:14.718241 14430 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:14.738512 14401 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.745054 14245 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.742743 13686 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.757483 13617 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.767496 13441 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.797402 13537 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:14.805099 14245 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:14.813241 14430 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:14.819937 13617 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:14.819988 14401 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.821344 14430 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:14.821753 14245 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.826695 13617 raft_consensus.cc:2955] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.840268 14440 raft_consensus.cc:1064] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:14.841094 13441 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.842991 14440 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:14.851333 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0863151795784affb539e2b7a55fc2b8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:14.856348 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 0863151795784affb539e2b7a55fc2b8 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:14.876255 14419 raft_consensus.cc:1064] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:14.878517 14471 tablet_replica.cc:333] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:14.880146 14471 raft_consensus.cc:2243] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:14.879544 14419 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:14.886435 14471 raft_consensus.cc:2272] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:14.890411 14471 ts_tablet_manager.cc:1916] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:14.891186 13617 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.892596 14245 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.892547 13692 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.894095 14470 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:14.898427 14470 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:14.903225 14401 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:14.909427 14471 ts_tablet_manager.cc:1929] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:14.910065 14471 log.cc:1199] T 0863151795784affb539e2b7a55fc2b8 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/0863151795784affb539e2b7a55fc2b8
I20260709 12:56:14.912091 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 0863151795784affb539e2b7a55fc2b8 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:14.919317 13686 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.921104 14430 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:14.931062 13617 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.932371 14398 raft_consensus.cc:1064] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:14.939699 14477 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:56:14.935890 14398 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:14.938040 14480 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.954026 14245 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:14.977227 13611 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:14.979887 13440 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:14.993510 13537 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:14.968951 13692 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.020099 13685 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:15.021871 14470 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.022970 14485 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:15.029912 13693 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.044760 13617 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.056052 13685 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:15.056221 13540 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:15.062278 14477 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:15.071595 14470 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.075641 14398 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:15.083984 14245 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:15.086625 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 85eb0ac09bf7466899637997aae42111 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:15.092365 13692 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.116446 14398 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:15.100967 13440 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.129421 14461 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.143291 13441 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.151590 14243 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:15.162464 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 85eb0ac09bf7466899637997aae42111 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:15.163204 14440 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:15.163856 14401 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:15.154254 13692 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.184350 14244 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.192447 14245 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.197073 13441 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.143229 13537 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.217593 14243 raft_consensus.cc:2955] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.221520 14246 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.244874 14489 tablet_replica.cc:333] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:15.246483 14489 raft_consensus.cc:2243] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:15.248205 14489 raft_consensus.cc:2272] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:15.253118 14489 ts_tablet_manager.cc:1916] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:15.269915 14489 ts_tablet_manager.cc:1929] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:15.270370 14489 log.cc:1199] T 85eb0ac09bf7466899637997aae42111 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/85eb0ac09bf7466899637997aae42111
I20260709 12:56:15.273437 13619 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:15.279594 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 85eb0ac09bf7466899637997aae42111 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:15.282362 14246 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:15.282646 13692 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:15.283890 14485 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:15.284857 14461 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:15.292070 14485 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.295033 13692 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.300282 14246 raft_consensus.cc:2955] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.303567 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2181ddb48f3d44d583ad7ecf793222fa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:15.314373 13441 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:15.326452 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet 2181ddb48f3d44d583ad7ecf793222fa with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:15.338336 14491 tablet_replica.cc:333] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:15.339510 14491 raft_consensus.cc:2243] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:15.340324 14491 raft_consensus.cc:2272] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:15.343571 14491 ts_tablet_manager.cc:1916] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:15.374387 14491 ts_tablet_manager.cc:1929] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:15.374852 14491 log.cc:1199] T 2181ddb48f3d44d583ad7ecf793222fa P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/2181ddb48f3d44d583ad7ecf793222fa
I20260709 12:56:15.376677 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet 2181ddb48f3d44d583ad7ecf793222fa (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:15.714308 13537 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:15.720994 14246 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:15.722131 13693 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:15.722523 14479 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:15.724763 14430 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:15.729534 14479 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.731009 14246 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.734786 13693 raft_consensus.cc:2955] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.746402 13441 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:15.752485 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e7d1ab18f48a4689857964b7200b72c9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:15.760229 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet e7d1ab18f48a4689857964b7200b72c9 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:15.761186 14489 tablet_replica.cc:333] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:15.762243 14489 raft_consensus.cc:2243] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:15.762857 14489 raft_consensus.cc:2272] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:15.766070 14489 ts_tablet_manager.cc:1916] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:15.782692 14489 ts_tablet_manager.cc:1929] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:15.783064 14489 log.cc:1199] T e7d1ab18f48a4689857964b7200b72c9 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/e7d1ab18f48a4689857964b7200b72c9
I20260709 12:56:15.784513 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet e7d1ab18f48a4689857964b7200b72c9 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:16.833220 13619 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:56:16.839614 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba2c2e5a7dac4eb4b3bacc18402ffa47. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:16.839668 13685 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:16.841459 14461 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:16.841826 13540 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:16.843124 14493 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:16.849593 14461 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.851377 13692 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.856683 13537 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.870378 13438 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.876600 13692 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:16.905925 13619 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:16.907474 13537 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:16.908993 14359 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:16.910679 14444 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:56:16.912055 13658 consensus_peers.cc:597] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcd2fb69161943208c2aef45acb3c33d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:16.919626 14503 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.921465 13619 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.941270 13537 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.952916 13540 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:16.951339 13441 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:56:16.971451 13508 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9eb8a2c5d7846099d30c0839aefc5a9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:16.973999 13685 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:16.975389 14477 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:16.978272 13619 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:16.980172 14477 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:16.995157 14492 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.996845 13619 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:16.999382 13685 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:17.010154 13537 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.015221 13438 catalog_manager.cc:5697] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:56:17.022517 13508 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87a8c869a6b840e591423340bc32af00. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:17.025890 13685 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:17.027531 14492 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:17.028395 13619 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:17.030102 14492 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:17.041283 14492 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:17.042876 13693 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:17.048866 13617 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:17.072171 13438 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:17.376147 14516 ts_tablet_manager.cc:933] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:17.379345 14516 tablet_copy_client.cc:323] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:56:17.380932 13703 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet bcd2fb69161943208c2aef45acb3c33d from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:56:17.381410 13703 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet bcd2fb69161943208c2aef45acb3c33d from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-bcd2fb69161943208c2aef45acb3c33d
I20260709 12:56:17.387159 13703 tablet_copy_source_session.cc:215] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:17.393708 14516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd2fb69161943208c2aef45acb3c33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:17.407177 14516 tablet_copy_client.cc:806] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:17.407850 14516 tablet_copy_client.cc:670] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:17.411660 14516 tablet_copy_client.cc:538] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:17.420626 14516 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:17.435789 14519 ts_tablet_manager.cc:933] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:17.438788 14516 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:17.439779 14516 tablet_bootstrap.cc:492] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:17.440514 14516 ts_tablet_manager.cc:1403] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 12:56:17.443434 14516 raft_consensus.cc:359] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.446836 14516 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:17.447551 14516 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.454250 14519 tablet_copy_client.cc:323] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:17.456562 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:17.457077 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-ba2c2e5a7dac4eb4b3bacc18402ffa47
I20260709 12:56:17.457410 14516 ts_tablet_manager.cc:1434] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.003s
I20260709 12:56:17.459053 13703 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-bcd2fb69161943208c2aef45acb3c33d received from {username='slave'} at 127.0.0.1:40872
I20260709 12:56:17.459441 13703 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-bcd2fb69161943208c2aef45acb3c33d on tablet bcd2fb69161943208c2aef45acb3c33d with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:17.464558 13629 tablet_copy_source_session.cc:215] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:17.468189 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:17.481051 14519 tablet_copy_client.cc:806] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:17.481686 14519 tablet_copy_client.cc:670] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:17.486984 14519 tablet_copy_client.cc:538] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:17.498251 14519 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:17.512233 14516 ts_tablet_manager.cc:933] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:17.528841 14516 tablet_copy_client.cc:323] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:17.530907 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet d9eb8a2c5d7846099d30c0839aefc5a9 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:17.531446 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet d9eb8a2c5d7846099d30c0839aefc5a9 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-d9eb8a2c5d7846099d30c0839aefc5a9
I20260709 12:56:17.561414 13554 tablet_copy_source_session.cc:215] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:17.572788 14516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9eb8a2c5d7846099d30c0839aefc5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:17.579473 14519 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:17.580489 14519 tablet_bootstrap.cc:492] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:17.581424 14519 ts_tablet_manager.cc:1403] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.083s	user 0.035s	sys 0.045s
I20260709 12:56:17.584905 14519 raft_consensus.cc:359] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.586007 14519 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:17.586726 14519 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.591109 14519 ts_tablet_manager.cc:1434] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:56:17.594815 14516 tablet_copy_client.cc:806] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:17.595679 14516 tablet_copy_client.cc:670] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:17.596141 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-ba2c2e5a7dac4eb4b3bacc18402ffa47 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:17.596596 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-ba2c2e5a7dac4eb4b3bacc18402ffa47 on tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:17.600545 14516 tablet_copy_client.cc:538] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:17.612465 14516 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:17.717327 14519 ts_tablet_manager.cc:933] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:17.729748 14519 tablet_copy_client.cc:323] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:17.732937 13554 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 87a8c869a6b840e591423340bc32af00 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:17.733848 13554 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 87a8c869a6b840e591423340bc32af00 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-87a8c869a6b840e591423340bc32af00
I20260709 12:56:17.752552 13554 tablet_copy_source_session.cc:215] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:17.756707 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87a8c869a6b840e591423340bc32af00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:17.759724 14516 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:17.760704 14516 tablet_bootstrap.cc:492] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:17.761543 14516 ts_tablet_manager.cc:1403] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.149s	user 0.032s	sys 0.014s
I20260709 12:56:17.765755 14516 raft_consensus.cc:359] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.767298 14516 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:17.768184 14516 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.773226 14516 ts_tablet_manager.cc:1434] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.001s
I20260709 12:56:17.779816 14519 tablet_copy_client.cc:806] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:17.780539 14519 tablet_copy_client.cc:670] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:17.782614 13554 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-d9eb8a2c5d7846099d30c0839aefc5a9 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:17.783172 13554 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-d9eb8a2c5d7846099d30c0839aefc5a9 on tablet d9eb8a2c5d7846099d30c0839aefc5a9 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:17.786139 14519 tablet_copy_client.cc:538] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:17.799798 14519 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:17.848340 14519 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:17.849686 14519 tablet_bootstrap.cc:492] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:17.850732 14519 ts_tablet_manager.cc:1403] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.027s
I20260709 12:56:17.854822 14519 raft_consensus.cc:359] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.856050 14519 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:17.862134 14519 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:17.889565 14519 ts_tablet_manager.cc:1434] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:56:17.909612 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-87a8c869a6b840e591423340bc32af00 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:17.910140 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-87a8c869a6b840e591423340bc32af00 on tablet 87a8c869a6b840e591423340bc32af00 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:17.924346 14243 raft_consensus.cc:1217] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:17.976163 14243 raft_consensus.cc:1217] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:18.012951 14246 raft_consensus.cc:1217] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:18.276376 14246 raft_consensus.cc:1217] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:18.429347 14530 raft_consensus.cc:1064] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:18.432266 14530 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:18.436971 14509 raft_consensus.cc:1064] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:18.442744 14509 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:18.454375 14246 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.456365 14530 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.459216 13692 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.464658 14530 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.469534 13537 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.471163 14530 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:18.477298 13617 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.479015 14535 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:18.483808 14246 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.484781 13537 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.498217 14359 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.499070 14509 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.528072 14359 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.531304 14246 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.542701 13540 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.544821 14477 raft_consensus.cc:1064] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:18.547624 14477 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:18.563165 13441 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.568064 13619 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.569980 14461 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.573423 13540 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.595992 13685 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.635514 14246 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.654076 14246 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.655953 13693 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.655754 14524 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.657510 13611 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.659629 14524 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:18.668205 14524 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.691955 14477 raft_consensus.cc:1064] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:18.697903 14477 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:18.719786 14246 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.727329 14494 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.731040 14555 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.734819 13611 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.740602 13619 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.744949 13439 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:18.745985 14492 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:18.758124 13685 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:18.765739 13692 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:18.781049 14244 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:18.779184 14528 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:56:18.794701 13615 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:18.795162 14509 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:18.798360 14535 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:18.812521 14534 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.822871 13617 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.837595 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bcd2fb69161943208c2aef45acb3c33d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:18.838336 14246 raft_consensus.cc:2955] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.857527 13440 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:18.863639 13611 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:18.853957 13438 catalog_manager.cc:5697] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:56:18.871156 14556 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:56:18.874806 14244 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.877214 13537 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:18.878480 14246 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:18.880179 14549 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:18.881201 14550 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:18.893024 14461 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.894853 14524 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.900516 13685 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.904227 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:18.906872 13693 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.900739 13537 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.911105 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet bcd2fb69161943208c2aef45acb3c33d with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:18.914213 14563 tablet_replica.cc:333] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:18.915534 14563 raft_consensus.cc:2243] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:18.923908 14563 raft_consensus.cc:2272] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:18.925170 14243 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.927568 14563 ts_tablet_manager.cc:1916] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:18.932554 14246 raft_consensus.cc:2955] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.956867 14563 ts_tablet_manager.cc:1929] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:18.960429 14563 log.cc:1199] T bcd2fb69161943208c2aef45acb3c33d P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/bcd2fb69161943208c2aef45acb3c33d
I20260709 12:56:18.958770 13611 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.962507 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet bcd2fb69161943208c2aef45acb3c33d (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:18.970512 13439 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:18.975831 13540 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:18.973991 13441 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:18.987250 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:18.994946 14246 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:19.003772 13692 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:19.004267 14555 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:19.012889 14564 tablet_replica.cc:333] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:19.013314 14477 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.014320 14564 raft_consensus.cc:2243] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:19.016691 14564 raft_consensus.cc:2272] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:19.021620 14555 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:19.026230 14246 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.033872 13693 raft_consensus.cc:2955] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.041028 13540 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:19.054615 14564 ts_tablet_manager.cc:1916] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:19.056416 13441 catalog_manager.cc:5697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.068914 14243 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:19.071092 14555 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:19.074659 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet d9eb8a2c5d7846099d30c0839aefc5a9 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:19.082180 14564 ts_tablet_manager.cc:1929] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:19.082779 14564 log.cc:1199] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/ba2c2e5a7dac4eb4b3bacc18402ffa47
I20260709 12:56:19.084779 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet ba2c2e5a7dac4eb4b3bacc18402ffa47 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:19.086671 14565 tablet_replica.cc:333] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:19.088078 14565 raft_consensus.cc:2243] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:19.088800 14565 raft_consensus.cc:2272] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:19.092423 14565 ts_tablet_manager.cc:1916] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:19.093101 14492 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.100870 13684 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:19.109329 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d9eb8a2c5d7846099d30c0839aefc5a9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:19.109216 14565 ts_tablet_manager.cc:1929] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:19.111068 14565 log.cc:1199] T d9eb8a2c5d7846099d30c0839aefc5a9 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/d9eb8a2c5d7846099d30c0839aefc5a9
I20260709 12:56:19.112807 14243 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.116752 14494 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:19.125008 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet d9eb8a2c5d7846099d30c0839aefc5a9 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:19.129714 13692 raft_consensus.cc:2955] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.148566 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 87a8c869a6b840e591423340bc32af00 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:19.157934 13440 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:19.170017 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 87a8c869a6b840e591423340bc32af00 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:19.171453 14565 tablet_replica.cc:333] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:19.172672 14565 raft_consensus.cc:2243] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:19.173175 14565 raft_consensus.cc:2272] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:19.176782 14565 ts_tablet_manager.cc:1916] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:19.190804 14565 ts_tablet_manager.cc:1929] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:19.191105 14565 log.cc:1199] T 87a8c869a6b840e591423340bc32af00 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/87a8c869a6b840e591423340bc32af00
I20260709 12:56:19.192252 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 87a8c869a6b840e591423340bc32af00 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:20.255286 13611 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:20.261826 13540 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:20.262260 13584 consensus_peers.cc:597] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22f14b8d35f741a28ad886b19614c4c8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:20.263017 13693 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:20.264192 14461 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:20.264878 14550 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:20.275976 14550 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.277966 13693 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.299105 13441 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.313429 13537 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.315276 13540 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:20.333082 13693 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:20.334697 14492 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:20.337803 13611 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:20.340598 13508 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91a3f49b0e164d27994cfe440af6f514. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:20.344630 14555 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:20.354205 14492 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.365227 13693 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.366422 13611 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.383872 13440 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.395454 13693 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:56:20.406404 13658 consensus_peers.cc:597] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12822fb99cf24dbcac8dbb9d77e56af8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:20.407665 13611 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:20.407886 13540 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:20.409010 14582 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:20.409960 14566 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:20.419500 14582 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.420859 13540 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.422629 13619 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.440289 13441 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.453017 13540 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:20.465638 13619 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:20.467530 13693 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:20.467553 14555 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:20.471701 14586 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:56:20.472429 13508 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 32b8156356cf4b709a2903d6861a117c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:20.481015 14586 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.483333 13619 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.484498 13692 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:20.505314 13438 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:20.844272 14590 ts_tablet_manager.cc:933] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:20.846750 14590 tablet_copy_client.cc:323] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:20.848825 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 22f14b8d35f741a28ad886b19614c4c8 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:20.849368 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 22f14b8d35f741a28ad886b19614c4c8 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-22f14b8d35f741a28ad886b19614c4c8
I20260709 12:56:20.857275 13629 tablet_copy_source_session.cc:215] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:20.860955 14590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f14b8d35f741a28ad886b19614c4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:20.883265 14590 tablet_copy_client.cc:806] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:20.884002 14590 tablet_copy_client.cc:670] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:20.888582 14590 tablet_copy_client.cc:538] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:20.893426 14592 ts_tablet_manager.cc:933] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:20.899217 14590 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:20.925853 14592 tablet_copy_client.cc:323] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:56:20.927889 13703 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet 12822fb99cf24dbcac8dbb9d77e56af8 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:56:20.928522 13703 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet 12822fb99cf24dbcac8dbb9d77e56af8 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-12822fb99cf24dbcac8dbb9d77e56af8
I20260709 12:56:20.932713 14590 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:20.933748 14590 tablet_bootstrap.cc:492] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:20.934451 14590 ts_tablet_manager.cc:1403] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.019s
I20260709 12:56:20.999938 13703 tablet_copy_source_session.cc:215] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:21.001804 14590 raft_consensus.cc:359] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.002761 14590 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
W20260709 12:56:21.004438 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 67425.8us
I20260709 12:56:21.011438 14592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12822fb99cf24dbcac8dbb9d77e56af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:21.037482 14596 ts_tablet_manager.cc:933] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:21.045674 14595 ts_tablet_manager.cc:933] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:21.006815 14590 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.052489 14595 tablet_copy_client.cc:323] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:21.054399 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 32b8156356cf4b709a2903d6861a117c from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:21.054989 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 32b8156356cf4b709a2903d6861a117c from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-32b8156356cf4b709a2903d6861a117c
I20260709 12:56:21.056437 14590 ts_tablet_manager.cc:1434] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.122s	user 0.014s	sys 0.007s
I20260709 12:56:21.058480 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-22f14b8d35f741a28ad886b19614c4c8 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:21.058890 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-22f14b8d35f741a28ad886b19614c4c8 on tablet 22f14b8d35f741a28ad886b19614c4c8 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:21.072521 13553 tablet_copy_source_session.cc:215] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:21.078372 14595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b8156356cf4b709a2903d6861a117c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:21.093515 14596 tablet_copy_client.cc:323] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:21.097888 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 91a3f49b0e164d27994cfe440af6f514 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:21.099220 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 91a3f49b0e164d27994cfe440af6f514 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-91a3f49b0e164d27994cfe440af6f514
I20260709 12:56:21.123142 13553 tablet_copy_source_session.cc:215] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:21.126078 14592 tablet_copy_client.cc:806] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:21.128657 14592 tablet_copy_client.cc:670] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:21.131475 14596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a3f49b0e164d27994cfe440af6f514. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:21.128129 14595 tablet_copy_client.cc:806] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:21.142936 14592 tablet_copy_client.cc:538] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:21.143379 14595 tablet_copy_client.cc:670] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:21.154522 14595 tablet_copy_client.cc:538] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:21.160919 14596 tablet_copy_client.cc:806] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:21.161617 14596 tablet_copy_client.cc:670] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:21.162988 14592 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:21.166013 14595 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:21.167856 14596 tablet_copy_client.cc:538] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:21.184702 14596 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:21.211436 14592 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:21.212478 14592 tablet_bootstrap.cc:492] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:21.213188 14592 ts_tablet_manager.cc:1403] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.019s
I20260709 12:56:21.216192 14592 raft_consensus.cc:359] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.217135 14592 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:21.217846 14592 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.225282 14592 ts_tablet_manager.cc:1434] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 12:56:21.227222 13703 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-12822fb99cf24dbcac8dbb9d77e56af8 received from {username='slave'} at 127.0.0.1:40872
I20260709 12:56:21.227665 13703 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-12822fb99cf24dbcac8dbb9d77e56af8 on tablet 12822fb99cf24dbcac8dbb9d77e56af8 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:21.230011 14596 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:21.231453 14596 tablet_bootstrap.cc:492] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:21.233158 14596 ts_tablet_manager.cc:1403] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.001s
I20260709 12:56:21.238770 14596 raft_consensus.cc:359] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.239709 14596 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:21.240679 14596 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.243103 14596 ts_tablet_manager.cc:1434] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:56:21.245872 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-91a3f49b0e164d27994cfe440af6f514 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:21.246361 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-91a3f49b0e164d27994cfe440af6f514 on tablet 91a3f49b0e164d27994cfe440af6f514 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:21.255542 14595 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:21.256728 14595 tablet_bootstrap.cc:492] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:21.257472 14595 ts_tablet_manager.cc:1403] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.092s	user 0.034s	sys 0.005s
I20260709 12:56:21.261708 14595 raft_consensus.cc:359] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.262583 14595 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:21.263120 14595 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:21.264880 14595 ts_tablet_manager.cc:1434] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:56:21.267321 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-32b8156356cf4b709a2903d6861a117c received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:21.267705 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-32b8156356cf4b709a2903d6861a117c on tablet 32b8156356cf4b709a2903d6861a117c with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:21.295233 14243 raft_consensus.cc:1217] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:21.425396 14246 raft_consensus.cc:1217] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:21.451634 14246 raft_consensus.cc:1217] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:21.566318 14246 raft_consensus.cc:1217] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:21.751904 14359 raft_consensus.cc:1064] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:21.754293 14359 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:21.762089 14246 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:21.763715 14359 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:21.764106 13540 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:21.764559 13617 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:21.765628 14607 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:56:21.771032 14607 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:21.798393 14607 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.808362 13617 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.820441 13540 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.823464 13441 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:21.854095 14246 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.883764 13693 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:21.888186 14246 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:21.888186 13617 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:21.889629 14607 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:21.890714 14359 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:21.900657 14246 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.901818 13611 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.913913 14607 raft_consensus.cc:2955] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.926788 14550 raft_consensus.cc:1064] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:21.930982 13441 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:21.931751 14550 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:21.944881 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet 12822fb99cf24dbcac8dbb9d77e56af8 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:21.947847 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12822fb99cf24dbcac8dbb9d77e56af8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:21.954463 14617 tablet_replica.cc:333] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:21.957403 14617 raft_consensus.cc:2243] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:21.958336 14617 raft_consensus.cc:2272] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:21.961890 14617 ts_tablet_manager.cc:1916] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:21.990047 14617 ts_tablet_manager.cc:1929] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:21.990576 14246 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:21.990571 13693 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:21.996708 14586 raft_consensus.cc:1064] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:21.994119 13540 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:22.023734 14616 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:22.024766 14550 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:22.025456 14586 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:22.029577 14615 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:22.056308 13692 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:22.060369 14243 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:22.060581 13611 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:22.058475 14555 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:56:22.022723 14617 log.cc:1199] T 12822fb99cf24dbcac8dbb9d77e56af8 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/12822fb99cf24dbcac8dbb9d77e56af8
I20260709 12:56:22.068272 14586 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:22.073380 14616 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.076423 13685 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.111503 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet 12822fb99cf24dbcac8dbb9d77e56af8 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:22.114858 14246 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.119321 13441 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.146092 14555 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:22.149592 14623 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.151260 13693 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.160845 14586 raft_consensus.cc:1064] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:22.163303 14586 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:22.166357 13540 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.169395 13617 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.188928 14244 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.189369 13441 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:22.210677 13611 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:22.211555 14246 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:22.214623 14586 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:22.216887 14492 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:22.217450 13692 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:22.218992 14524 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:22.231515 13611 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:22.232630 14586 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.234495 13615 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.251269 13692 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.254462 13693 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:22.254639 14244 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:22.256359 14615 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:22.257371 14616 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:22.257421 14244 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.265723 13693 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.267587 13441 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.268074 14246 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.264106 13540 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:22.264336 14626 raft_consensus.cc:2955] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.283612 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 22f14b8d35f741a28ad886b19614c4c8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:22.289484 14244 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:22.291033 14555 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:22.295583 13611 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:22.297328 13440 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.298223 14555 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:22.300251 14524 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.309152 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f14b8d35f741a28ad886b19614c4c8 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:22.310176 14617 tablet_replica.cc:333] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:22.311270 14617 raft_consensus.cc:2243] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:22.311908 14617 raft_consensus.cc:2272] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:22.314500 14243 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.316583 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 91a3f49b0e164d27994cfe440af6f514 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:22.326606 14617 ts_tablet_manager.cc:1916] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:22.326884 13540 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:22.334900 14246 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:22.338371 14555 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:22.337126 13615 raft_consensus.cc:2955] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.342598 13611 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:22.344149 14555 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:22.337409 13437 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:22.345292 14623 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.356647 14244 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.366498 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 32b8156356cf4b709a2903d6861a117c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:22.358597 13615 raft_consensus.cc:2955] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:22.360458 14617 ts_tablet_manager.cc:1929] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:22.365130 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 91a3f49b0e164d27994cfe440af6f514 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:22.373221 14617 log.cc:1199] T 22f14b8d35f741a28ad886b19614c4c8 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/22f14b8d35f741a28ad886b19614c4c8
I20260709 12:56:22.386597 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet 22f14b8d35f741a28ad886b19614c4c8 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:22.393657 13437 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:22.406827 14632 tablet_replica.cc:333] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:22.417579 14632 raft_consensus.cc:2243] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:22.419572 14632 raft_consensus.cc:2272] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:22.421581 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 32b8156356cf4b709a2903d6861a117c with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:22.425248 14632 ts_tablet_manager.cc:1916] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:22.441392 14632 ts_tablet_manager.cc:1929] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:22.441721 14632 log.cc:1199] T 91a3f49b0e164d27994cfe440af6f514 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/91a3f49b0e164d27994cfe440af6f514
I20260709 12:56:22.443176 14632 tablet_replica.cc:333] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:22.443101 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 91a3f49b0e164d27994cfe440af6f514 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:22.444286 14632 raft_consensus.cc:2243] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:22.444923 14632 raft_consensus.cc:2272] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:22.447394 14632 ts_tablet_manager.cc:1916] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:22.458478 14632 ts_tablet_manager.cc:1929] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:22.458781 14632 log.cc:1199] T 32b8156356cf4b709a2903d6861a117c P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/32b8156356cf4b709a2903d6861a117c
I20260709 12:56:22.460011 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 32b8156356cf4b709a2903d6861a117c (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:23.497277 13693 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:23.504650 13617 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:23.504871 13540 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:56:23.504742 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d739d3dda4e4370b33be67fca1c66d5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:23.506193 14635 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:23.507148 14607 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:23.515369 14607 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.519045 13617 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.521270 13540 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.530422 13611 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:23.533782 13441 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:56:23.569977 13584 consensus_peers.cc:597] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa97d994ab11478b9f735ced13bc6d48. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:23.570854 13692 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:23.572218 14626 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:23.582442 13537 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:23.590010 14626 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:23.606524 13692 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.606907 14626 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.629093 13537 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:23.653134 13540 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.696761 13441 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:56:23.722108 13508 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c2d2db0b5674796bbd0426953d99538. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:23.727484 13685 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:23.742946 13617 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:23.744882 14633 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:23.747640 14638 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:23.757455 13693 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.758519 13617 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.755798 14653 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.776068 13437 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.787050 13537 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:56:23.794212 13508 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9aa892a8dfec433d8aa859d442e3e309. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:23.795059 13693 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:23.796257 14524 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:23.796810 13617 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:23.798035 14524 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:23.803329 14638 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.804817 13693 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.818933 13615 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:23.822085 13440 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:24.014605 14659 ts_tablet_manager.cc:933] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:24.018968 14659 tablet_copy_client.cc:323] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:56:24.020663 13703 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet 9d739d3dda4e4370b33be67fca1c66d5 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:56:24.021189 13703 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet 9d739d3dda4e4370b33be67fca1c66d5 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-9d739d3dda4e4370b33be67fca1c66d5
I20260709 12:56:24.028816 13703 tablet_copy_source_session.cc:215] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:24.032091 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d739d3dda4e4370b33be67fca1c66d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:24.048099 14659 tablet_copy_client.cc:806] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:24.050735 14659 tablet_copy_client.cc:670] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:24.054518 14659 tablet_copy_client.cc:538] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:24.063831 14659 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:24.085765 14659 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:24.086747 14659 tablet_bootstrap.cc:492] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:24.087458 14659 ts_tablet_manager.cc:1403] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.024s	user 0.025s	sys 0.000s
I20260709 12:56:24.090490 14659 raft_consensus.cc:359] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.091408 14659 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:24.092079 14659 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.095111 14659 ts_tablet_manager.cc:1434] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:56:24.098446 13703 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-9d739d3dda4e4370b33be67fca1c66d5 received from {username='slave'} at 127.0.0.1:40872
I20260709 12:56:24.098893 13703 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-9d739d3dda4e4370b33be67fca1c66d5 on tablet 9d739d3dda4e4370b33be67fca1c66d5 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:24.160305 14659 ts_tablet_manager.cc:933] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:24.162456 14659 tablet_copy_client.cc:323] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:24.164114 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet fa97d994ab11478b9f735ced13bc6d48 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:24.164697 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet fa97d994ab11478b9f735ced13bc6d48 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-fa97d994ab11478b9f735ced13bc6d48
I20260709 12:56:24.173542 13629 tablet_copy_source_session.cc:215] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:24.176542 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa97d994ab11478b9f735ced13bc6d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:24.193344 14659 tablet_copy_client.cc:806] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:24.194115 14659 tablet_copy_client.cc:670] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:24.197760 14659 tablet_copy_client.cc:538] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:24.209367 14659 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:24.237437 14659 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:24.238510 14659 tablet_bootstrap.cc:492] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:24.239274 14659 ts_tablet_manager.cc:1403] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.014s
I20260709 12:56:24.242553 14659 raft_consensus.cc:359] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.243496 14659 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:24.244506 14659 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.247233 14659 ts_tablet_manager.cc:1434] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:56:24.257121 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-fa97d994ab11478b9f735ced13bc6d48 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:24.258807 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-fa97d994ab11478b9f735ced13bc6d48 on tablet fa97d994ab11478b9f735ced13bc6d48 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:24.312147 14659 ts_tablet_manager.cc:933] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:24.313926 14659 tablet_copy_client.cc:323] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:24.315646 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 1c2d2db0b5674796bbd0426953d99538 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:24.316226 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 1c2d2db0b5674796bbd0426953d99538 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-1c2d2db0b5674796bbd0426953d99538
I20260709 12:56:24.324337 13553 tablet_copy_source_session.cc:215] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:24.328552 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2d2db0b5674796bbd0426953d99538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:24.345098 14666 ts_tablet_manager.cc:933] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:24.350658 14666 tablet_copy_client.cc:323] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:24.352419 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 9aa892a8dfec433d8aa859d442e3e309 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:24.352982 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 9aa892a8dfec433d8aa859d442e3e309 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-9aa892a8dfec433d8aa859d442e3e309
I20260709 12:56:24.353180 14659 tablet_copy_client.cc:806] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:24.354055 14659 tablet_copy_client.cc:670] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:24.359957 14659 tablet_copy_client.cc:538] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:24.367254 13553 tablet_copy_source_session.cc:215] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:24.369704 14659 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:24.370577 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa892a8dfec433d8aa859d442e3e309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:24.390600 14666 tablet_copy_client.cc:806] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:24.391148 14666 tablet_copy_client.cc:670] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:24.391283 14659 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:24.392170 14659 tablet_bootstrap.cc:492] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:24.393102 14659 ts_tablet_manager.cc:1403] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.001s
I20260709 12:56:24.395552 14666 tablet_copy_client.cc:538] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:24.395705 14659 raft_consensus.cc:359] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.397567 14659 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:24.398185 14659 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.400279 14659 ts_tablet_manager.cc:1434] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:56:24.405427 14666 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:24.409126 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-1c2d2db0b5674796bbd0426953d99538 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:24.409611 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-1c2d2db0b5674796bbd0426953d99538 on tablet 1c2d2db0b5674796bbd0426953d99538 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:24.426496 14243 raft_consensus.cc:1217] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:56:24.436285 14666 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:24.437337 14666 tablet_bootstrap.cc:492] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:24.438956 14666 ts_tablet_manager.cc:1403] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.008s
I20260709 12:56:24.441666 14666 raft_consensus.cc:359] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.442484 14666 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:24.443130 14666 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:24.449231 14666 ts_tablet_manager.cc:1434] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:56:24.451318 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-9aa892a8dfec433d8aa859d442e3e309 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:24.451761 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-9aa892a8dfec433d8aa859d442e3e309 on tablet 9aa892a8dfec433d8aa859d442e3e309 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:24.694952 14244 raft_consensus.cc:1217] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:56:24.806191 14244 raft_consensus.cc:1217] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:24.827222 14244 raft_consensus.cc:1217] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:24.834743 14359 raft_consensus.cc:1064] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:24.837005 14359 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:24.843751 13617 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:56:24.843751 14244 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:56:24.845278 14635 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:24.845542 13540 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:56:24.847028 14635 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:24.847895 14669 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:24.859552 14635 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:24.864223 14244 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:24.865513 13540 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:24.880618 13617 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:24.882813 13441 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:24.909351 13692 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:24.920356 14243 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:24.922231 14359 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:24.924714 13540 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:24.928045 14635 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:24.936589 14669 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:24.944568 14243 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:24.948541 13540 raft_consensus.cc:2955] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:24.952292 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9d739d3dda4e4370b33be67fca1c66d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:24.967710 13440 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:24.980321 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d739d3dda4e4370b33be67fca1c66d5 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:24.996421 14682 tablet_replica.cc:333] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:24.997779 14682 raft_consensus.cc:2243] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:24.998450 14682 raft_consensus.cc:2272] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:25.001928 14682 ts_tablet_manager.cc:1916] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:25.015064 14682 ts_tablet_manager.cc:1929] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:56:25.015492 14682 log.cc:1199] T 9d739d3dda4e4370b33be67fca1c66d5 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/9d739d3dda4e4370b33be67fca1c66d5
I20260709 12:56:25.017134 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 9d739d3dda4e4370b33be67fca1c66d5 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:25.268992 14642 raft_consensus.cc:1064] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:25.271553 14642 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:25.279495 13692 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:56:25.279973 14243 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:56:25.281418 14673 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:25.286489 14684 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:25.288867 13537 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:56:25.294648 14673 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:25.297683 14638 raft_consensus.cc:1064] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:25.302776 14638 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:25.323848 14673 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.333976 14244 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.340210 13540 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.353186 13441 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.361159 13692 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.371635 13615 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:25.386539 14244 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:25.389084 14673 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:25.392378 14246 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:25.394173 14651 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:25.396664 14684 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.400291 13692 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:25.408373 13692 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:25.408995 14638 raft_consensus.cc:1064] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:25.413074 14638 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:25.413494 14246 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.416527 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fa97d994ab11478b9f735ced13bc6d48 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:25.421924 14673 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:25.428968 13693 raft_consensus.cc:2955] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.439980 14243 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:25.441923 14691 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:25.453791 13692 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:25.461511 14691 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:25.467088 13611 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:25.468209 13615 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:25.473922 14692 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:25.477082 13437 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.481325 14689 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:25.484543 14689 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:25.507431 14651 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.510936 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet fa97d994ab11478b9f735ced13bc6d48 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:25.512854 13615 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.510308 14671 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.533907 13619 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.535265 14244 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.540185 13684 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.551079 14246 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.552009 13437 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.568403 13693 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.574867 13441 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.584846 13536 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:25.591958 14243 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:25.596496 14651 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:25.599881 14700 tablet_replica.cc:333] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:25.600903 14700 raft_consensus.cc:2243] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:25.601603 14700 raft_consensus.cc:2272] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:25.598644 13615 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:25.610101 14651 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:25.617756 14700 ts_tablet_manager.cc:1916] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:25.628129 14243 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.625907 14651 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.630894 13536 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:25.635424 13611 raft_consensus.cc:2955] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.645462 14246 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:25.649825 13693 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:25.652509 14638 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:25.654048 14691 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:25.667394 13440 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.673854 14692 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.676131 14700 ts_tablet_manager.cc:1929] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:56:25.676654 14700 log.cc:1199] T fa97d994ab11478b9f735ced13bc6d48 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/fa97d994ab11478b9f735ced13bc6d48
I20260709 12:56:25.678061 13692 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.680154 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet fa97d994ab11478b9f735ced13bc6d48 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:25.686565 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c2d2db0b5674796bbd0426953d99538 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:25.692420 14246 raft_consensus.cc:2955] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:25.694391 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa892a8dfec433d8aa859d442e3e309 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:25.698100 14702 tablet_replica.cc:333] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:25.701001 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9aa892a8dfec433d8aa859d442e3e309 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:25.702625 13437 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:25.723456 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d2db0b5674796bbd0426953d99538 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:25.724813 14702 raft_consensus.cc:2243] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:25.725910 14702 raft_consensus.cc:2272] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:25.728304 14703 tablet_replica.cc:333] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:25.729434 14703 raft_consensus.cc:2243] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:25.730237 14702 ts_tablet_manager.cc:1916] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:25.730283 14703 raft_consensus.cc:2272] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:25.745107 14703 ts_tablet_manager.cc:1916] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:25.751446 14702 ts_tablet_manager.cc:1929] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:25.752106 14702 log.cc:1199] T 9aa892a8dfec433d8aa859d442e3e309 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/9aa892a8dfec433d8aa859d442e3e309
I20260709 12:56:25.753814 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 9aa892a8dfec433d8aa859d442e3e309 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:25.759770 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:25.763504 14703 ts_tablet_manager.cc:1929] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:25.764021 14703 log.cc:1199] T 1c2d2db0b5674796bbd0426953d99538 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/1c2d2db0b5674796bbd0426953d99538
I20260709 12:56:25.765533 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 1c2d2db0b5674796bbd0426953d99538 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:25.814505 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:25.815131 13537 raft_consensus.cc:606] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:25.817577 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f54dde33fa845098e64673b576d780c"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:25.818023 13537 raft_consensus.cc:606] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:25.820675 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5406eb61fd534169b3c1645f80144779"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:25.821168 13537 raft_consensus.cc:606] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:25.823418 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:25.824086 13537 raft_consensus.cc:606] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:25.826396 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:25.826869 13537 raft_consensus.cc:606] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:25.827879 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:56:25.829547 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:25.875701 14690 raft_consensus.cc:993] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:25.876276 14651 raft_consensus.cc:1081] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:25.877700 13693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:25.878214 13693 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:25.878491 13693 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:25.883615 13693 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:25.884999 13693 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:25.885850 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:25.886020 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:25.886489 13537 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:25.886572 13611 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:25.886857 13537 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:25.887600 13537 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:25.888715 13537 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:25.892719 13611 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:25.894013 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:25.894726 14359 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:25.894888 13537 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:25.895185 14359 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:25.895989 14359 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:25.907817 13438 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:26.018332 14690 raft_consensus.cc:993] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:26.018893 14690 raft_consensus.cc:1081] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:26.020757 13615 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f54dde33fa845098e64673b576d780c"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:56:26.021265 13615 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:26.021641 13615 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.026156 13615 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:26.028101 13615 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:26.029024 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:26.029312 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:26.029657 13692 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.029863 13537 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:26.031710 13537 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:26.033386 13537 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:26.035646 13537 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.038244 14651 raft_consensus.cc:993] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:26.039700 14651 raft_consensus.cc:1081] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:26.041683 13693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5406eb61fd534169b3c1645f80144779"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:26.041886 13692 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:26.042219 13693 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:26.042656 13693 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.042791 13537 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:26.043685 13582 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:26.044775 14673 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:26.045282 14673 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:26.046262 14690 raft_consensus.cc:993] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:26.046121 14673 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:26.046818 14651 raft_consensus.cc:1081] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:26.048569 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59164
I20260709 12:56:26.049216 14243 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:26.049589 14243 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.050302 13693 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:26.053359 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:26.054152 13537 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:26.054517 13537 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:26.055187 13537 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:26.057385 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:26.058059 13537 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.058070 13611 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.059044 13438 catalog_manager.cc:5697] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:26.057195 13693 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:26.066661 14243 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:26.067731 13537 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:26.069126 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:26.069375 13611 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:26.070464 14359 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:26.071233 14243 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:26.072108 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:26.072722 13537 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:26.073004 13537 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:26.071766 14359 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:26.074005 13537 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:26.074203 14359 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:26.075212 13537 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.070757 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:26.075795 13615 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.101507 13537 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:26.102602 14691 raft_consensus.cc:993] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:26.102684 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:26.103245 14690 raft_consensus.cc:1081] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:26.103343 13615 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:26.103827 14662 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:26.105484 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59164
I20260709 12:56:26.106094 14243 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:26.106503 14243 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.113377 13438 catalog_manager.cc:5697] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:26.123219 14662 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:26.124374 14662 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:26.125418 14243 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:26.127864 14243 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:26.128242 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:26.128803 13537 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:26.129076 13537 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:26.129884 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:26.129879 13537 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:26.130708 13615 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.131511 13537 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:26.136034 13438 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:26.137008 13615 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:26.137741 13537 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:26.138213 14212 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:26.139130 14662 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:26.139593 14662 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:26.140314 14662 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:26.151227 13438 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:26.395057 13537 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:26.396270 14669 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.408893 13617 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:26.410420 14669 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.514783 13617 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:26.516055 14662 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:26.530941 13536 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:26.540166 14709 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:26.549996 13692 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:26.552476 14673 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.566452 13537 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:26.568507 14673 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.576704 13536 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:26.577500 13615 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:26.578461 14669 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.581022 14669 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.746920 13536 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:26.748718 14662 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:26.760591 13615 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:26.762557 14719 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:26.830543 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:26.847220 13617 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:26.857836 13692 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:26.858837 13584 consensus_peers.cc:597] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e634cc18febe4680bb70a3673cabbae4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:26.859647 14722 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.861328 13536 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:26.864665 14722 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.881340 14722 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.883469 13536 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.883471 13692 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.884497 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:26.906670 13441 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.916193 13536 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:26.941004 13617 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:26.941545 13508 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d1c4c42ef7ad4d8fb669a6a5fe183f6f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:26.944664 13693 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:26.945310 14691 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.947357 14651 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:26.970700 14651 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.972939 13693 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.981880 13615 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.992566 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c5bef52669b468c8765408698f5884d"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:26.991202 13440 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:26.993288 13536 raft_consensus.cc:606] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:26.996744 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:26.997371 13536 raft_consensus.cc:606] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:27.002178 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:27.002801 13536 raft_consensus.cc:606] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:27.007192 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85eb0ac09bf7466899637997aae42111"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:27.007858 13536 raft_consensus.cc:606] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:27.010561 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:27.011559 13536 raft_consensus.cc:606] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:27.012708 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:56:27.014928 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:27.027866 13615 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.034657 13693 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:27.035720 13536 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:27.036083 14673 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:27.038296 14673 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:56:27.039170 13584 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d73171142594388972ed22eae44f95f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:27.049274 14673 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:27.050895 13693 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:27.069527 13537 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:27.069638 13440 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:27.078627 13536 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.095099 13693 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:27.096485 13508 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be389b2236484464bc3e9f558d7b4230. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:27.097095 14690 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:27.099563 14651 raft_consensus.cc:993] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:27.100049 14651 raft_consensus.cc:1081] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:27.101385 13615 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:27.101429 14246 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85eb0ac09bf7466899637997aae42111"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59164
I20260709 12:56:27.102087 14246 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:27.104481 14690 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:27.102407 14246 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.118100 14651 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:27.128238 13693 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:27.130934 13617 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:27.118269 14246 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:27.138759 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:27.138985 14246 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:27.139349 13537 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:27.139470 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:27.140043 13692 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.139714 13537 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:27.145289 13537 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:27.146508 13537 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.148910 13692 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:27.150173 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:27.151032 14719 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:27.151494 14719 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:27.152314 14719 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:27.162133 14727 raft_consensus.cc:993] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:27.162631 14727 raft_consensus.cc:1081] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:27.167017 13615 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:56:27.166277 13441 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:27.167567 13615 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:27.169764 14691 raft_consensus.cc:993] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:27.170053 13615 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.170248 14691 raft_consensus.cc:1081] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:27.172705 13693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:27.173259 13693 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:27.173601 13693 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.173854 13437 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:27.177508 13615 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.180171 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:27.180351 13615 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:27.180742 13536 raft_consensus.cc:3055] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:27.181052 13536 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:27.181839 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:27.181731 13536 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.182368 13692 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.182821 13536 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.189736 13692 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:27.191058 13582 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:27.191457 13536 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:27.191792 14684 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:27.192333 14684 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:27.193190 14684 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.199468 13537 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:27.201259 13693 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.205964 13693 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:27.206931 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:27.207510 13537 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:27.207794 13537 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:27.207832 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:27.208560 13615 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.208642 13537 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.209925 13537 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.217659 13537 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:27.218732 13658 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:27.219471 14669 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:27.219972 14669 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:27.220053 13441 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:27.220868 14669 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.278831 13440 catalog_manager.cc:5697] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:27.286561 14690 raft_consensus.cc:993] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:27.287081 14690 raft_consensus.cc:1081] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:27.290361 13617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c5bef52669b468c8765408698f5884d"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:56:27.290917 13617 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:27.291271 13617 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.297142 13615 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:27.302268 13617 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.304935 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:27.305405 13617 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:27.305559 13537 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:27.306042 13537 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:27.306811 13537 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.307612 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:27.308015 13537 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.308434 13693 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:27.315026 13693 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:27.320726 13582 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:27.321281 13537 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:27.321506 14684 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:27.321995 14684 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:27.322836 14684 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.340776 13440 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:27.345973 14749 ts_tablet_manager.cc:933] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:27.363394 14749 tablet_copy_client.cc:323] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:27.365038 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:27.365612 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-d1c4c42ef7ad4d8fb669a6a5fe183f6f
I20260709 12:56:27.373123 13553 tablet_copy_source_session.cc:215] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:27.376778 14749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:27.392454 14749 tablet_copy_client.cc:806] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:27.393038 14749 tablet_copy_client.cc:670] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:27.396577 14749 tablet_copy_client.cc:538] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:27.404090 14749 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:27.413708 14752 ts_tablet_manager.cc:933] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:27.416400 14752 tablet_copy_client.cc:323] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:27.418085 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet e634cc18febe4680bb70a3673cabbae4 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:27.418628 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet e634cc18febe4680bb70a3673cabbae4 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-e634cc18febe4680bb70a3673cabbae4
I20260709 12:56:27.426296 13629 tablet_copy_source_session.cc:215] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:27.429390 14749 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:27.429612 14752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e634cc18febe4680bb70a3673cabbae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:27.430333 14749 tablet_bootstrap.cc:492] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:27.431030 14749 ts_tablet_manager.cc:1403] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.007s
I20260709 12:56:27.433825 14749 raft_consensus.cc:359] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.434617 14749 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:27.435252 14749 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.440827 14749 ts_tablet_manager.cc:1434] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:56:27.442435 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-d1c4c42ef7ad4d8fb669a6a5fe183f6f received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:27.442859 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-d1c4c42ef7ad4d8fb669a6a5fe183f6f on tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:27.450359 14752 tablet_copy_client.cc:806] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:27.450961 14752 tablet_copy_client.cc:670] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:27.454457 14752 tablet_copy_client.cc:538] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:27.463522 14752 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:27.483480 14749 ts_tablet_manager.cc:933] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:27.484784 14752 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:27.485893 14752 tablet_bootstrap.cc:492] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:27.486855 14752 ts_tablet_manager.cc:1403] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.009s
I20260709 12:56:27.490005 14651 raft_consensus.cc:993] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:27.490451 14651 raft_consensus.cc:1081] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:27.491238 14752 raft_consensus.cc:359] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.492146 14752 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:27.492858 14752 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.504060 13693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:27.504631 13693 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:27.505048 13693 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:27.507031 14752 ts_tablet_manager.cc:1434] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.004s
I20260709 12:56:27.508178 14749 tablet_copy_client.cc:323] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:27.511806 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 1d73171142594388972ed22eae44f95f from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:27.512523 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 1d73171142594388972ed22eae44f95f from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-1d73171142594388972ed22eae44f95f
I20260709 12:56:27.515951 14756 ts_tablet_manager.cc:933] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:27.530447 13629 tablet_copy_source_session.cc:215] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:27.532076 13628 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-e634cc18febe4680bb70a3673cabbae4 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:27.533254 13628 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-e634cc18febe4680bb70a3673cabbae4 on tablet e634cc18febe4680bb70a3673cabbae4 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:27.532960 14756 tablet_copy_client.cc:323] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:27.536487 14749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d73171142594388972ed22eae44f95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:27.536826 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet be389b2236484464bc3e9f558d7b4230 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:27.537350 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet be389b2236484464bc3e9f558d7b4230 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-be389b2236484464bc3e9f558d7b4230
I20260709 12:56:27.546653 13553 tablet_copy_source_session.cc:215] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:27.550788 14756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be389b2236484464bc3e9f558d7b4230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:27.521862 13693 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.566433 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:27.567178 13537 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:27.567449 13537 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:27.568158 13537 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.572930 14749 tablet_copy_client.cc:806] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:27.573577 14749 tablet_copy_client.cc:670] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:27.569304 13537 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:27.584882 14749 tablet_copy_client.cc:538] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:27.592967 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:27.593814 13617 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:27.617760 14749 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:27.619163 14756 tablet_copy_client.cc:806] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:27.619750 14756 tablet_copy_client.cc:670] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:27.622521 13617 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:27.627668 14756 tablet_copy_client.cc:538] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:27.629628 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:27.630831 14359 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:27.631577 13693 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:27.633294 14359 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:27.634161 14359 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:27.636822 13537 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:27.641728 14756 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:27.674311 13693 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:27.676774 14673 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:27.690955 13440 catalog_manager.cc:5697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 2 to 3, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 3 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:27.712630 14749 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:27.713955 14749 tablet_bootstrap.cc:492] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:27.714708 14749 ts_tablet_manager.cc:1403] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.097s	user 0.020s	sys 0.011s
I20260709 12:56:27.721511 14756 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:27.723033 14756 tablet_bootstrap.cc:492] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:27.723805 14756 ts_tablet_manager.cc:1403] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.082s	user 0.035s	sys 0.037s
I20260709 12:56:27.718060 14749 raft_consensus.cc:359] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.724931 14749 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:27.726815 14756 raft_consensus.cc:359] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.727756 14756 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:27.725524 14749 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.728842 14756 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:27.730360 14749 ts_tablet_manager.cc:1434] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:56:27.732225 14756 ts_tablet_manager.cc:1434] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:56:27.742060 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-be389b2236484464bc3e9f558d7b4230 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:27.742362 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-1d73171142594388972ed22eae44f95f received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:27.742547 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-be389b2236484464bc3e9f558d7b4230 on tablet be389b2236484464bc3e9f558d7b4230 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:27.742828 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-1d73171142594388972ed22eae44f95f on tablet 1d73171142594388972ed22eae44f95f with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:27.748018 13537 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:27.749800 14673 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:27.753768 14243 raft_consensus.cc:1217] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:27.771162 13540 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:27.772714 13693 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:27.778241 14709 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:27.780144 14709 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:27.794356 13540 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:27.801050 14673 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:27.870792 13543 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:56:27.874217 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46131.1us
I20260709 12:56:27.980960 14669 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
W20260709 12:56:28.073134 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 195739us
W20260709 12:56:28.081362 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 119266us
I20260709 12:56:28.100149 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:28.126380 14243 raft_consensus.cc:1217] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:28.173017 13613 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:28.180365 14243 raft_consensus.cc:1217] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:28.218935 14245 raft_consensus.cc:1217] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:56:28.219877 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46465.1us
I20260709 12:56:28.232870 13692 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:28.237512 14778 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:28.276227 13611 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:28.291024 14784 raft_consensus.cc:1064] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:28.296177 14777 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:28.302273 14784 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:28.327314 14635 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:28.342824 14243 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.345144 14783 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.374241 13611 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.386762 14784 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.393220 13692 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.421895 14787 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.453576 13543 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:28.482065 14784 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.487593 14781 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:56:28.498529 13617 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.499604 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:28.522842 14651 raft_consensus.cc:1064] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:28.525130 14651 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:28.528332 13692 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.539351 13438 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.555871 14246 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.571163 14243 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.574973 13693 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.585595 14788 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.586658 14691 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:28.585924 13615 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.593863 14788 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.613456 14784 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.615865 13686 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.631556 14243 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.637439 13617 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.646826 13537 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:28.648811 14722 raft_consensus.cc:1064] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:28.651908 14722 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:28.658929 14790 raft_consensus.cc:1064] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:28.676083 14790 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:28.683676 14243 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.686391 13440 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.688454 13686 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.689808 14245 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:28.694101 14691 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:28.697448 13686 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:28.702420 14691 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:28.687868 14684 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:56:28.714746 14777 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.738406 14784 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.755959 14243 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.763890 13539 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.765141 13537 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:28.760016 13693 raft_consensus.cc:2955] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.780025 14684 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.785537 14246 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:28.788688 13686 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:28.792922 14691 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:28.797398 14789 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:28.810138 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:28.811357 13539 raft_consensus.cc:606] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:28.810302 14786 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.812762 14789 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.816998 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:56:28.817185 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 123989us
I20260709 12:56:28.825644 14246 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.835057 14812 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.836751 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:28.839051 13539 raft_consensus.cc:606] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:28.838383 13537 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.823422 13437 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.843253 14246 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.852620 13538 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:28.853374 13538 raft_consensus.cc:606] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:28.855046 13539 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.875439 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:28.876453 14786 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:28.880450 14819 tablet_replica.cc:333] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:28.886456 13693 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:28.890569 14245 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.899755 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet be389b2236484464bc3e9f558d7b4230 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:28.910076 14819 raft_consensus.cc:2243] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:28.911113 14819 raft_consensus.cc:2272] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:28.910894 13438 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:28.915692 14819 ts_tablet_manager.cc:1916] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:28.920719 13441 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.928053 13686 raft_consensus.cc:2955] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.935802 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet be389b2236484464bc3e9f558d7b4230 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:28.936909 14777 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.939836 14812 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:28.944908 13539 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.963346 13693 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.976989 14783 raft_consensus.cc:993] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:28.977531 14783 raft_consensus.cc:1081] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:28.982018 13692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:28.982661 13692 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:28.983026 13692 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:28.988976 14245 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:28.994825 13692 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.012434 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:29.013145 13617 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.016157 13440 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:29.009821 13692 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:29.003471 13684 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.022845 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:29.025406 13539 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:29.020877 13615 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:29.028297 13617 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:29.028928 13539 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:29.031132 14819 ts_tablet_manager.cc:1929] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:29.041330 14819 log.cc:1199] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/d1c4c42ef7ad4d8fb669a6a5fe183f6f
I20260709 12:56:29.043601 14245 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:29.038956 13539 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.042230 13537 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:29.046126 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:29.046907 13539 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.047276 14669 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:29.047816 14669 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:29.048880 14669 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.052747 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet d1c4c42ef7ad4d8fb669a6a5fe183f6f (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:29.052871 14722 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:29.053992 14790 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:29.055269 14819 tablet_replica.cc:333] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:29.065243 14819 raft_consensus.cc:2243] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:29.064456 14722 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.070170 14819 raft_consensus.cc:2272] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:29.071084 13684 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:29.071749 13684 raft_consensus.cc:606] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:29.061586 13539 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:29.075091 13537 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.077880 14243 raft_consensus.cc:2955] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.081506 13538 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:29.082166 13538 raft_consensus.cc:606] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:29.083889 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:56:29.084532 14819 ts_tablet_manager.cc:1916] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:29.085462 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:29.088887 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1d73171142594388972ed22eae44f95f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:29.099378 13438 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:29.109933 13437 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:29.117781 14819 ts_tablet_manager.cc:1929] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:29.118285 14819 log.cc:1199] T be389b2236484464bc3e9f558d7b4230 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/be389b2236484464bc3e9f558d7b4230
I20260709 12:56:29.120410 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet be389b2236484464bc3e9f558d7b4230 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:29.127024 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d73171142594388972ed22eae44f95f with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:29.130131 14824 tablet_replica.cc:333] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:29.131377 14824 raft_consensus.cc:2243] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:29.132190 14824 raft_consensus.cc:2272] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:29.136202 14824 ts_tablet_manager.cc:1916] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:29.157698 14824 ts_tablet_manager.cc:1929] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:29.158099 14824 log.cc:1199] T 1d73171142594388972ed22eae44f95f P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/1d73171142594388972ed22eae44f95f
I20260709 12:56:29.160241 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 1d73171142594388972ed22eae44f95f (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:29.212255 14784 raft_consensus.cc:993] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:29.213217 14691 raft_consensus.cc:1081] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:29.214848 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59164
I20260709 12:56:29.215328 14243 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:29.215606 14243 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.221380 14243 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:29.223269 14243 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:29.225744 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:29.226431 13537 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:29.226711 13537 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:29.228960 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:29.229506 13617 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.227496 13537 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:29.235114 13537 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.240077 13617 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:29.242370 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:29.243296 14767 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:29.243782 14767 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:29.245266 14767 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:29.258229 13537 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:29.262132 13438 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:29.309278 14669 raft_consensus.cc:993] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:29.309795 14669 raft_consensus.cc:1081] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:29.312063 13617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56784
I20260709 12:56:29.312608 13617 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:29.312886 14789 raft_consensus.cc:993] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:29.313020 13617 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.313498 14784 raft_consensus.cc:1081] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:29.315249 13684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:29.315800 13684 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:29.316183 13684 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.318324 13617 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.320664 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:29.320989 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:29.321353 13537 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.321588 13692 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:29.321923 13692 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:29.322626 13692 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.323088 13684 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.323704 13692 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.325649 13684 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:29.326150 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:29.326691 13615 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.320683 13617 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:29.326150 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:29.329031 13538 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:29.329342 13538 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:29.330212 13538 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.331796 13538 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.333025 13692 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:29.353474 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:29.345152 13615 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:29.344044 13537 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:29.354501 14722 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:29.354969 14722 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:29.355908 14722 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.356491 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:29.357405 13538 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:29.357700 14359 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:29.359205 14359 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:29.360028 14359 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.376036 13438 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:29.377563 13437 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 2, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:29.383020 14784 raft_consensus.cc:993] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:29.383589 14691 raft_consensus.cc:1081] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:29.385562 13617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:56:29.386090 13617 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:29.386413 13617 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.396347 13617 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.398190 13617 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:29.398778 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:29.399407 13538 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:29.399713 13538 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:29.399739 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:29.400481 13692 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.400825 13538 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.402094 13538 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:29.407344 13692 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:29.410188 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:29.411193 13538 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:29.411528 14777 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:29.412103 14777 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:29.412993 14777 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:29.428700 13437 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:29.449678 13538 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.451112 14359 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:29.480206 13617 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:29.491328 14245 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:29.494480 14722 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:29.497066 13692 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:29.498857 14722 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:29.507611 13617 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.516171 14790 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.518471 14778 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:29.517908 14243 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.525426 13693 raft_consensus.cc:2955] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.529834 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e634cc18febe4680bb70a3673cabbae4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:29.552855 13440 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 1 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:29.565788 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet e634cc18febe4680bb70a3673cabbae4 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:29.568840 14834 tablet_replica.cc:333] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:29.569671 14834 raft_consensus.cc:2243] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:29.570302 14834 raft_consensus.cc:2272] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:29.573601 14834 ts_tablet_manager.cc:1916] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:29.587153 14834 ts_tablet_manager.cc:1929] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:29.587566 14834 log.cc:1199] T e634cc18febe4680bb70a3673cabbae4 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/e634cc18febe4680bb70a3673cabbae4
I20260709 12:56:29.589301 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet e634cc18febe4680bb70a3673cabbae4 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:29.792279 13693 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.793493 14790 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:29.805915 13537 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.809144 14790 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:29.815173 13537 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:29.816970 14767 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:29.834187 13617 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:29.838070 14838 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:29.838584 13539 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.840093 14669 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:29.877964 13615 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.879485 14359 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:29.927500 13537 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.928785 14810 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:29.938221 13693 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:29.939857 14810 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:30.086781 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:30.139653 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:30.181818 13692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:30.182358 13692 raft_consensus.cc:606] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:30.184995 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:30.185530 13537 raft_consensus.cc:606] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:30.188272 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:30.188849 13537 raft_consensus.cc:606] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:30.205890 13615 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:56582
I20260709 12:56:30.206362 14669 raft_consensus.cc:993] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.206476 13615 raft_consensus.cc:606] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:30.206945 14778 raft_consensus.cc:1081] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.208959 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59176
I20260709 12:56:30.209899 14243 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:30.209892 13692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:30.210345 14243 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.210778 13692 raft_consensus.cc:606] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:30.213685 13537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:30.214250 13537 raft_consensus.cc:606] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:30.215366 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:56:30.216910 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:30.217085 14243 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.219471 14243 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:30.219722 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:30.220340 13617 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.222942 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:30.223470 13693 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:30.223742 13693 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:30.224521 13693 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.225646 13693 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.232836 13617 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.232971 13693 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.234025 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:30.234831 14841 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:30.235340 14841 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:30.236300 14841 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.244758 13437 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:30.415647 14789 raft_consensus.cc:993] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:30.416183 14843 raft_consensus.cc:1081] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:30.417577 13693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:30.418026 13693 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:30.418289 13693 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.422690 13693 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:30.424248 13693 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:30.425135 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:30.425529 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:30.425736 13539 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:30.426119 13539 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:30.426141 13617 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.426856 13539 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:30.427742 13539 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.432193 13617 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:30.433501 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:30.433729 13539 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:30.434358 14669 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:30.434970 14669 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:30.435766 14669 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:30.443153 14790 raft_consensus.cc:993] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.443670 14810 raft_consensus.cc:1081] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.445112 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59162
I20260709 12:56:30.445497 14243 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:30.445793 14243 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.450177 14243 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.452898 14243 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:30.453886 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:30.453948 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:30.454581 13615 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:30.454635 13539 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.455034 13615 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:30.455814 13615 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.457087 13615 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.459303 13437 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:30.461971 13539 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.463150 14211 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:30.464190 14854 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:30.464654 14854 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:30.465466 14854 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.466749 13615 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.480973 14843 raft_consensus.cc:993] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.481439 14843 raft_consensus.cc:1081] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.483907 13437 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:30.486670 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59164
I20260709 12:56:30.489830 14243 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:30.490240 14243 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.498404 14243 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.501108 14243 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:30.501267 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:30.501873 13539 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:30.502169 13539 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:30.502094 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:30.502754 13693 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.508679 13539 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.510043 13539 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.510663 13693 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.511718 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:30.512465 14854 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:30.513010 14854 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:30.516243 13539 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.513916 14854 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.531284 14857 raft_consensus.cc:993] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.531759 14857 raft_consensus.cc:1081] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.534138 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59164
I20260709 12:56:30.534754 14243 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:30.535136 14243 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.549258 14243 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.552109 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:30.552716 13539 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:30.552984 13539 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:30.553781 13539 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.554880 13539 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.558565 14243 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:30.560232 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:30.560810 13615 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.562143 13437 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:30.565654 13539 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.565838 13615 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.566831 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:30.568118 14854 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:30.569005 14854 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:30.569980 14854 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.584815 13437 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:30.641873 13539 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:30.648757 13692 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:56:30.650352 13508 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a689534693d4469befa622d6b4917e3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:30.650599 14862 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:56:30.658465 13613 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:30.660219 14863 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:30.662168 14862 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.663872 13686 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.668664 14778 raft_consensus.cc:993] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.669209 14669 raft_consensus.cc:1081] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:30.670926 14245 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59176
I20260709 12:56:30.671430 14245 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:30.671751 14245 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.681819 13437 catalog_manager.cc:5697] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.684195 13615 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:56:30.683490 13489 auto_rebalancer.cc:629] Failed to schedule move for tablet c665edf3d7834e18a67df3f75b30856a: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:56:30.688210 14245 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.688231 13613 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:30.690233 13617 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:30.692317 14245 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:30.693572 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:30.693665 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:30.694212 13692 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:30.693974 14854 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:30.694329 13613 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.694593 13692 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:30.695639 13692 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.697295 13692 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:30.719344 13613 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.730219 13692 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:30.730477 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:30.731513 14838 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:30.731997 14838 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:30.732869 14838 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:30.753353 13437 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:30.755270 13692 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:30.763816 13539 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:56:30.765413 13584 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72d95a7092b0445eb00a28a1fdb8d239. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:30.766234 14790 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:30.768949 14838 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:30.776238 13684 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:56:30.781158 14790 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:30.807807 14858 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.814502 13684 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.824409 13538 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.827230 13539 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:30.836474 13440 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:56:30.842275 13508 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 835fe463242d40e5b82ddb5898323fc5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:30.846639 13619 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:30.842366 13684 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:30.849396 14862 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:30.850237 14789 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:30.859618 14789 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.861689 13684 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.880678 13440 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.883728 13617 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:30.897229 13619 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:30.900238 14838 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:30.921336 13539 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:30.923554 14854 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:30.941586 13692 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:30.943890 14854 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:30.951750 13537 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:30.954015 14841 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:31.010815 13617 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:31.013594 14669 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:31.040212 13539 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:31.051337 14886 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:31.122936 13692 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:31.126295 14838 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:31.141803 13617 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:31.143523 14854 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:31.161540 13619 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:31.173226 14841 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:31.177354 13537 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:31.178920 14838 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:31.217813 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:31.234803 14894 ts_tablet_manager.cc:933] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:31.240340 14894 tablet_copy_client.cc:323] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:31.242380 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 1a689534693d4469befa622d6b4917e3 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:31.242936 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 1a689534693d4469befa622d6b4917e3 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-1a689534693d4469befa622d6b4917e3
I20260709 12:56:31.257959 13553 tablet_copy_source_session.cc:215] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:31.261471 14894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a689534693d4469befa622d6b4917e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:31.335907 14894 tablet_copy_client.cc:806] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:31.336609 14894 tablet_copy_client.cc:670] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:31.353484 14894 tablet_copy_client.cc:538] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:31.389113 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:31.396833 14894 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:31.404835 14898 ts_tablet_manager.cc:933] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:31.411506 14899 ts_tablet_manager.cc:933] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:31.415274 14898 tablet_copy_client.cc:323] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:31.417052 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 835fe463242d40e5b82ddb5898323fc5 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:31.417644 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 835fe463242d40e5b82ddb5898323fc5 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-835fe463242d40e5b82ddb5898323fc5
I20260709 12:56:31.424667 14899 tablet_copy_client.cc:323] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:56:31.426509 13553 tablet_copy_source_session.cc:215] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:31.426956 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 72d95a7092b0445eb00a28a1fdb8d239 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:56:31.427523 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 72d95a7092b0445eb00a28a1fdb8d239 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-72d95a7092b0445eb00a28a1fdb8d239
I20260709 12:56:31.432853 14894 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:31.433828 14894 tablet_bootstrap.cc:492] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:31.434499 14894 ts_tablet_manager.cc:1403] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.012s
I20260709 12:56:31.434900 14898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835fe463242d40e5b82ddb5898323fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:31.437703 14894 raft_consensus.cc:359] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:31.438625 14894 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:31.439268 14894 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:31.441859 14894 ts_tablet_manager.cc:1434] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:56:31.446141 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-1a689534693d4469befa622d6b4917e3 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:31.446976 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-1a689534693d4469befa622d6b4917e3 on tablet 1a689534693d4469befa622d6b4917e3 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:31.450021 13629 tablet_copy_source_session.cc:215] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:31.453471 14899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d95a7092b0445eb00a28a1fdb8d239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:31.462733 14898 tablet_copy_client.cc:806] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:31.463382 14898 tablet_copy_client.cc:670] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:31.468191 14898 tablet_copy_client.cc:538] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:31.489969 14898 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:31.494025 14899 tablet_copy_client.cc:806] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:31.494666 14899 tablet_copy_client.cc:670] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:31.497160 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:31.497779 13539 raft_consensus.cc:606] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:31.500867 13617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
new_leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:56582
I20260709 12:56:31.501389 13617 raft_consensus.cc:606] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Received request to transfer leadership to TS ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:31.502629 14899 tablet_copy_client.cc:538] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:31.504526 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:31.505086 13539 raft_consensus.cc:606] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:31.506534 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:56:31.508580 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:31.511615 14899 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:31.530860 14898 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:31.531860 14898 tablet_bootstrap.cc:492] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:31.532804 14898 ts_tablet_manager.cc:1403] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.043s	user 0.030s	sys 0.007s
I20260709 12:56:31.538235 14898 raft_consensus.cc:359] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:31.539328 14898 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:31.540781 14898 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:31.543104 14863 raft_consensus.cc:993] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:31.543715 14863 raft_consensus.cc:1081] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:31.544574 14898 ts_tablet_manager.cc:1434] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 12:56:31.547360 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-835fe463242d40e5b82ddb5898323fc5 received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:31.547972 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-835fe463242d40e5b82ddb5898323fc5 on tablet 835fe463242d40e5b82ddb5898323fc5 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:31.551147 14245 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59164
I20260709 12:56:31.551844 14245 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:31.552233 14245 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:31.569500 14245 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:31.573539 14245 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:31.574244 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:31.575027 13539 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:31.575438 13539 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:31.576414 13539 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:31.578087 13539 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:31.580886 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:31.581492 13684 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:31.589521 13684 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:31.591806 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:31.593454 14841 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:31.594161 14841 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:31.595237 14841 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:31.601891 13539 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:31.613888 14899 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:31.614990 14899 tablet_bootstrap.cc:492] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:31.615629 14899 ts_tablet_manager.cc:1403] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.104s	user 0.042s	sys 0.047s
I20260709 12:56:31.618680 14899 raft_consensus.cc:359] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:31.619534 14899 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:31.620190 14899 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:31.622694 14899 ts_tablet_manager.cc:1434] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:56:31.631664 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-72d95a7092b0445eb00a28a1fdb8d239 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:56:31.632202 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-72d95a7092b0445eb00a28a1fdb8d239 on tablet 72d95a7092b0445eb00a28a1fdb8d239 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:31.643671 13437 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:31.676126 14789 raft_consensus.cc:993] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:31.676654 14789 raft_consensus.cc:1081] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:31.678289 13684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:31.678797 13684 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:31.679142 13684 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:31.684959 13684 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:31.686695 13684 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:31.687676 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:31.688297 13539 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:31.688483 13539 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:31.689056 13539 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:31.689845 13539 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:31.692732 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:31.693281 13617 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:31.698475 13539 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:31.700647 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:31.700837 13617 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:31.701486 14669 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:31.702013 14669 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:31.702731 14669 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:31.713505 13437 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:31.800239 14245 raft_consensus.cc:1217] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:31.822445 14245 raft_consensus.cc:1217] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:32.013196 14245 raft_consensus.cc:1217] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:56:32.028009 14896 raft_consensus.cc:993] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: : Instructing follower ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:56:32.028548 14871 raft_consensus.cc:1081] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Signalling peer ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:56:32.033257 13539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:49540
I20260709 12:56:32.034379 13539 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:32.034747 13539 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:32.044575 13539 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:32.047164 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:32.047744 13617 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:32.048058 13617 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:32.048813 13617 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:32.049851 13617 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:32.052282 13539 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:32.054850 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:32.055402 13686 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:32.062448 13686 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:32.063867 13506 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:32.064694 14857 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:32.065780 14857 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:32.066730 14857 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:32.069137 13617 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:32.082561 13438 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 2, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:32.094805 13539 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:32.096226 14841 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:32.100829 13538 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:32.102365 14887 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:32.140000 13686 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:32.141536 14916 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:56:32.206085 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 50200.8us
I20260709 12:56:32.207837 13617 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:32.212648 14669 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:32.220577 14789 raft_consensus.cc:1064] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:32.229147 14789 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.242348 14245 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:32.247509 14857 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:56:32.250808 13613 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:32.300663 14928 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:56:32.310540 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 67009.4us
W20260709 12:56:32.330924 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 89358us
I20260709 12:56:32.332764 13692 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:32.340993 14863 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.344729 14925 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:32.345744 14243 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.365370 13438 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.379128 13692 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.387395 13617 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.405925 14929 raft_consensus.cc:1064] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:32.409636 14929 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.422461 14243 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:32.422919 13692 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:32.423827 13613 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:32.424806 14925 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:32.425937 14863 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:32.434072 14929 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:32.439052 14927 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.443531 13613 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.460949 13437 catalog_manager.cc:5697] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.467628 14243 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.480556 13692 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.486972 13537 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.489765 14910 raft_consensus.cc:1064] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:32.495676 14910 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.509509 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:32.513139 14243 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:32.514786 14863 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:32.521526 14243 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:32.522809 14871 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:32.523422 13692 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:32.524878 14871 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:32.527871 13617 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:32.527385 14925 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.528887 14246 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.533358 13537 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:32.534838 14871 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:32.540820 14863 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:32.544309 13686 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:32.546168 14863 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:32.545918 13617 raft_consensus.cc:2955] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.545960 13613 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:32.554860 14927 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:32.577838 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1a689534693d4469befa622d6b4917e3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:32.594449 14871 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.662452 14243 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.712738 13440 catalog_manager.cc:5697] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index 3 to 4, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:32.720602 13537 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.727339 13437 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.735314 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:32.735246 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a689534693d4469befa622d6b4917e3 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:32.763975 14945 tablet_replica.cc:333] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:32.770789 13693 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.791086 13537 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.798547 14243 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:32.795042 13617 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.800247 14926 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:32.802779 14945 raft_consensus.cc:2243] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:32.803537 14945 raft_consensus.cc:2272] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:32.803694 13686 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:32.807030 14945 ts_tablet_manager.cc:1916] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:32.808252 14926 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.810514 14928 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:32.813236 14246 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.815663 14245 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:32.817855 13539 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:32.823418 14938 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:32.824376 14910 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:32.827644 13686 raft_consensus.cc:2955] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.844063 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 835fe463242d40e5b82ddb5898323fc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:32.843446 14245 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.845852 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:32.846408 13539 raft_consensus.cc:606] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:32.851245 13693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:32.851867 13693 raft_consensus.cc:606] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:32.836977 13437 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.840154 14910 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.857302 13693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:32.857944 13693 raft_consensus.cc:606] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:32.859256 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:56:32.860877 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:32.866948 13539 raft_consensus.cc:2955] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:32.868209 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 72d95a7092b0445eb00a28a1fdb8d239 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:32.880137 13438 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:32.902680 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 835fe463242d40e5b82ddb5898323fc5 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:32.902977 14945 ts_tablet_manager.cc:1929] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:32.903553 14945 log.cc:1199] T 1a689534693d4469befa622d6b4917e3 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/1a689534693d4469befa622d6b4917e3
I20260709 12:56:32.906313 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 1a689534693d4469befa622d6b4917e3 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:32.907799 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 72d95a7092b0445eb00a28a1fdb8d239 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:32.909283 14945 tablet_replica.cc:333] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:32.910641 14945 raft_consensus.cc:2243] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:32.911580 14945 raft_consensus.cc:2272] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:32.917587 14945 ts_tablet_manager.cc:1916] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:32.921463 14946 tablet_replica.cc:333] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:32.922868 14946 raft_consensus.cc:2243] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:32.924069 14946 raft_consensus.cc:2272] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:32.928493 14925 raft_consensus.cc:993] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:32.929283 14946 ts_tablet_manager.cc:1916] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:32.929425 14926 raft_consensus.cc:1081] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:32.931166 13686 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:32.931664 13686 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:32.932031 13686 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:32.942962 14945 ts_tablet_manager.cc:1929] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:56:32.942451 13686 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.943526 14945 log.cc:1199] T 72d95a7092b0445eb00a28a1fdb8d239 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/72d95a7092b0445eb00a28a1fdb8d239
I20260709 12:56:32.945451 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 72d95a7092b0445eb00a28a1fdb8d239 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:32.946651 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:32.946794 13686 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:32.947275 13537 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:32.947589 13537 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:32.947752 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:32.948344 14245 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:32.948398 13537 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.949759 13537 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:32.956617 14245 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:32.958112 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:32.959297 14921 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:32.959859 14921 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:32.960743 14921 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:32.963649 14917 raft_consensus.cc:993] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:32.964218 14917 raft_consensus.cc:1081] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:32.965982 13617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56784
I20260709 12:56:32.966535 13617 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:32.966905 13617 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:32.967553 14946 ts_tablet_manager.cc:1929] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:32.968276 14946 log.cc:1199] T 835fe463242d40e5b82ddb5898323fc5 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/835fe463242d40e5b82ddb5898323fc5
I20260709 12:56:32.970762 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 835fe463242d40e5b82ddb5898323fc5 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:32.977283 13617 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:32.979738 13617 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:32.980953 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:32.981570 13543 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:32.985021 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:32.986397 13537 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:32.983897 13440 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:32.987109 13686 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:32.988592 13543 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:32.988646 13686 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:32.989850 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:32.990620 14871 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:32.991153 14871 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:32.990224 13686 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:32.991993 14871 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:32.993553 13686 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:33.005060 13686 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:33.012583 13440 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:33.098104 14669 raft_consensus.cc:993] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:33.098723 14669 raft_consensus.cc:1081] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:33.100279 14243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59176
I20260709 12:56:33.100836 14243 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:33.101123 14243 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:33.105918 14243 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:33.108476 14243 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:33.108476 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:33.109289 13543 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:33.109208 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:33.110041 13686 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:33.110349 13686 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:33.111042 13686 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:33.112315 13686 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:33.115980 13543 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:33.117067 14211 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:33.118058 13686 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:33.117942 14854 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:33.119334 14854 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:33.120306 14854 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:33.128914 13440 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:33.468281 13543 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:33.469441 14921 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:33.478428 14245 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:33.480628 14953 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:33.494606 13543 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:33.496850 14871 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:33.514357 13686 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:33.515799 14854 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:33.519955 13686 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:33.523439 14871 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:33.535974 13543 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:33.540172 14916 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:33.862128 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:33.935348 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:33.991123 13693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:33.991624 13693 raft_consensus.cc:606] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Received request to transfer leadership to TS ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:33.994199 13613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
new_leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:56582
I20260709 12:56:33.994668 13613 raft_consensus.cc:606] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Received request to transfer leadership to TS ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:33.996990 13543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:33.997447 13543 raft_consensus.cc:606] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:33.998772 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:56:34.000108 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:34.009855 13693 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.018103 14871 raft_consensus.cc:993] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: : Instructing follower ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:56:34.018573 14871 raft_consensus.cc:1081] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Signalling peer ad2fdb1517c542439654bd3f7c72dad0 to start an election
W20260709 12:56:34.018802 13658 consensus_peers.cc:597] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b9bdd77e564429c9c0434b358f48302. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:34.020730 13543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:49540
I20260709 12:56:34.021394 13543 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:34.021749 13537 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:56:34.021752 13543 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:34.022560 13613 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:56:34.024865 14921 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:34.029665 14669 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:34.039085 14953 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.042838 13539 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.059257 13543 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:34.064433 13543 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:34.065621 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:34.066334 13617 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:34.066648 13617 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:34.076859 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:34.077697 13686 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:34.067364 13617 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:34.079084 13617 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:34.089720 13617 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:34.105000 13613 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.102984 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:34.107760 13489 auto_rebalancer.cc:629] Failed to schedule move for tablet 12c636bac0584e2e92ff66a089d880ee: Remote error: Service unavailable: leader transfer in progress
I20260709 12:56:34.106953 14925 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:34.110564 14925 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:34.112543 14925 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:34.113782 13686 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:34.129966 13440 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.138082 13537 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.146026 13693 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:56:34.146817 13508 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ee08c4df55f41c7a18951168c44fb5a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:34.147210 13613 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:56:34.147866 14925 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:34.150190 14926 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:34.159001 14968 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.163502 13693 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.166707 14926 raft_consensus.cc:993] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:34.167212 14966 raft_consensus.cc:1081] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:34.172312 13686 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:34.172829 13686 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:34.169548 13613 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.173151 13686 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:34.185930 13693 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.191565 14965 raft_consensus.cc:993] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: : Instructing follower ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:56:34.192042 14953 raft_consensus.cc:1081] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Signalling peer ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:56:34.194005 13539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:49526
I20260709 12:56:34.194543 13539 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:34.194857 13539 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:34.196163 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82077fd60c1346d7b057a65a778ee2f5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:56:34.201640 13537 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:34.199713 13441 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 3 to 4, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 4 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:34.206341 14967 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:34.206327 13441 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 3, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:34.209789 13539 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:34.203015 13686 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:34.216804 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:34.217134 13686 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:34.217427 13617 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:34.218999 13613 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:56:34.223671 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:34.226639 13693 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:34.227025 13693 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:34.225368 13539 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:34.228422 13693 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:34.230881 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:34.231535 13539 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:34.231894 13539 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:34.236261 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:34.237249 14245 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:34.232445 13693 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:34.234174 14953 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:34.235363 13539 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:34.247280 13539 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:34.252945 13539 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:34.254199 14967 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.255661 13613 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.256865 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:34.258020 14965 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:34.258652 14965 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:34.259836 14965 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:34.274652 13440 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.286769 13693 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:34.288093 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:34.290072 14926 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:34.290772 14926 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:34.291867 14926 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:34.293510 13438 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:34.280056 13617 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:34.275852 14245 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:34.331005 13438 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:34.331604 13539 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:56:34.550776 13613 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:56:34.552242 14926 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:34.571131 13693 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:56:34.585114 14968 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:56:34.637046 14985 ts_tablet_manager.cc:933] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:34.658763 14988 ts_tablet_manager.cc:933] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:34.669863 14988 tablet_copy_client.cc:323] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:56:34.679466 13703 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet 82077fd60c1346d7b057a65a778ee2f5 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:56:34.680104 13703 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet 82077fd60c1346d7b057a65a778ee2f5 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-82077fd60c1346d7b057a65a778ee2f5
I20260709 12:56:34.689536 14985 tablet_copy_client.cc:323] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:56:34.691747 13703 tablet_copy_source_session.cc:215] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:34.692718 13702 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet 4b9bdd77e564429c9c0434b358f48302 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:56:34.693331 13702 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet 4b9bdd77e564429c9c0434b358f48302 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-4b9bdd77e564429c9c0434b358f48302
I20260709 12:56:34.695093 14988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82077fd60c1346d7b057a65a778ee2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:34.701613 13702 tablet_copy_source_session.cc:215] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:34.705199 14985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9bdd77e564429c9c0434b358f48302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:34.711977 14988 tablet_copy_client.cc:806] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:34.712616 14988 tablet_copy_client.cc:670] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:34.720130 14988 tablet_copy_client.cc:538] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:34.726155 14985 tablet_copy_client.cc:806] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:34.726922 14985 tablet_copy_client.cc:670] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:34.729513 13537 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:34.733631 14988 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:34.733557 14965 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:34.735324 14985 tablet_copy_client.cc:538] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:34.757370 14991 ts_tablet_manager.cc:933] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:34.759469 14985 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:34.769613 14991 tablet_copy_client.cc:323] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:56:34.771392 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 2ee08c4df55f41c7a18951168c44fb5a from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:56:34.771999 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 2ee08c4df55f41c7a18951168c44fb5a from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-2ee08c4df55f41c7a18951168c44fb5a
I20260709 12:56:34.776641 14988 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:34.777740 14988 tablet_bootstrap.cc:492] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:34.778529 14988 ts_tablet_manager.cc:1403] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.045s	user 0.030s	sys 0.000s
I20260709 12:56:34.781679 13553 tablet_copy_source_session.cc:215] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:56:34.781682 14988 raft_consensus.cc:359] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.797803 14988 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:34.798002 13617 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:34.792789 13692 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:34.798537 14988 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.801028 14988 ts_tablet_manager.cc:1434] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 12:56:34.801504 14926 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:34.805445 14926 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:34.814213 14991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee08c4df55f41c7a18951168c44fb5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:56:34.829375 14985 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:34.830394 14985 tablet_bootstrap.cc:492] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:34.831077 14985 ts_tablet_manager.cc:1403] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.072s	user 0.033s	sys 0.023s
I20260709 12:56:34.832113 14991 tablet_copy_client.cc:806] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:56:34.832687 14991 tablet_copy_client.cc:670] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:56:34.840991 14991 tablet_copy_client.cc:538] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:56:34.845415 14985 raft_consensus.cc:359] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.846403 14985 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:34.847054 14985 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.849687 14985 ts_tablet_manager.cc:1434] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:56:34.853811 14991 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:56:34.857952 13702 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-82077fd60c1346d7b057a65a778ee2f5 received from {username='slave'} at 127.0.0.1:40872
I20260709 12:56:34.858469 13702 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-82077fd60c1346d7b057a65a778ee2f5 on tablet 82077fd60c1346d7b057a65a778ee2f5 with peer 0f34cda059124f839711dbef61bb2c28
W20260709 12:56:34.879010 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 64497.4us
I20260709 12:56:34.887217 14245 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:34.890131 13702 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-4b9bdd77e564429c9c0434b358f48302 received from {username='slave'} at 127.0.0.1:40872
I20260709 12:56:34.890558 13702 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-4b9bdd77e564429c9c0434b358f48302 on tablet 4b9bdd77e564429c9c0434b358f48302 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:34.894383 14991 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:56:34.895579 14991 tablet_bootstrap.cc:492] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:56:34.896436 14991 ts_tablet_manager.cc:1403] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.043s	user 0.030s	sys 0.009s
I20260709 12:56:34.896862 14965 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:34.899773 14991 raft_consensus.cc:359] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:34.900727 14991 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:56:34.901523 14991 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:56:34.941135 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49109.6us
I20260709 12:56:34.951629 14991 ts_tablet_manager.cc:1434] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.001s
I20260709 12:56:34.953651 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-2ee08c4df55f41c7a18951168c44fb5a received from {username='slave'} at 127.0.0.1:45566
I20260709 12:56:34.954252 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-2ee08c4df55f41c7a18951168c44fb5a on tablet 2ee08c4df55f41c7a18951168c44fb5a with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:35.000939 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:35.055629 14245 raft_consensus.cc:1217] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:56:35.078989 14243 raft_consensus.cc:1217] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:56:35.119491 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:35.176492 13693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:35.177131 13693 raft_consensus.cc:606] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:35.180033 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87a8c869a6b840e591423340bc32af00"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:35.180620 13539 raft_consensus.cc:606] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:35.183426 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:35.183882 13539 raft_consensus.cc:606] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:35.185999 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:56:35.196980 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:35.286393 14243 raft_consensus.cc:1217] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:56:35.357172 14966 raft_consensus.cc:993] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:35.357703 14968 raft_consensus.cc:1081] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:35.359097 13617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:56:35.359544 13617 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:35.359824 13617 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:35.365760 13617 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.367321 13617 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:35.368155 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:35.368461 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:35.369616 13539 raft_consensus.cc:3055] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:35.369709 14243 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:35.370069 13539 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:35.371076 13539 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.372416 13539 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:35.375801 14243 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:35.376873 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:35.377580 15002 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:35.378021 15002 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:35.378778 13539 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:35.378742 15002 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.390430 13441 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:35.419322 14965 raft_consensus.cc:993] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:35.419821 14965 raft_consensus.cc:1081] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:56:35.421252 13613 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56784
I20260709 12:56:35.421811 13613 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:35.422096 13613 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:35.426813 13613 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:35.428318 13613 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:35.429013 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:35.429409 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:35.429654 13539 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:35.429996 13692 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:35.430291 13692 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:35.430918 13692 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:35.431850 13692 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:35.440074 13692 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:35.441002 14966 raft_consensus.cc:993] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:35.441236 13582 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:35.441475 14966 raft_consensus.cc:1081] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:35.440639 13539 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:35.442061 15000 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:35.442514 15000 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:35.443336 15000 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:35.443889 13692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87a8c869a6b840e591423340bc32af00"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:56:35.444478 13692 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:35.444804 13692 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:35.451537 13692 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.454062 13692 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:35.455039 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:35.455637 14243 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:35.458936 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:35.459583 13537 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:35.459872 13537 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:35.462616 13537 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.463809 13441 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:35.464920 13537 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:35.481122 14243 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:35.483405 13537 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:35.485183 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:35.485991 14997 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:35.486446 14997 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:35.487264 14997 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.527406 14967 raft_consensus.cc:1064] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:35.529999 14967 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.548985 14246 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:35.554281 13613 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:35.555966 15011 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:35.569919 14997 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:35.576534 13539 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:56:35.577809 13441 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:35.578806 14967 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:35.619441 15013 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.626173 14246 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.632488 13537 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.641098 13613 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.644131 13440 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 2 to 3, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:35.646166 15014 raft_consensus.cc:1064] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:35.648825 15014 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.665220 13693 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.688367 14243 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:35.689599 13617 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:35.690871 14246 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:35.692395 14967 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:35.692762 14997 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:35.693992 15011 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:35.696798 13537 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:56:35.706786 15011 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:35.708737 15011 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.705199 13537 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:35.715102 14245 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.720465 14997 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:35.744899 13441 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New cstate: current_term: 1 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.746831 15013 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.748970 13538 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.755344 13607 tablet_service.cc:1558] Processing DeleteTablet for tablet 82077fd60c1346d7b057a65a778ee2f5 with delete_type TABLET_DATA_TOMBSTONED (TS d35f063cd3f942b0bf559b7c6e9b299f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56772
I20260709 12:56:35.759896 13537 raft_consensus.cc:2955] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.771322 14243 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:56:35.825026 15015 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.011s
I20260709 12:56:35.825176 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82077fd60c1346d7b057a65a778ee2f5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:35.839843 13615 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.848026 15023 tablet_replica.cc:333] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:56:35.849773 15023 raft_consensus.cc:2243] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:35.861717 13441 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.880406 15023 raft_consensus.cc:2272] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:35.892870 13543 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:35.894387 15000 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:35.896518 14925 raft_consensus.cc:1064] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:56:35.901364 14925 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.902132 13693 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:35.912384 15023 ts_tablet_manager.cc:1916] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:35.931562 13615 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:35.935001 14997 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:35.937837 14246 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:35.944535 14997 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:35.948268 15023 ts_tablet_manager.cc:1929] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:56:35.948819 15023 log.cc:1199] T 82077fd60c1346d7b057a65a778ee2f5 P d35f063cd3f942b0bf559b7c6e9b299f: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/wals/82077fd60c1346d7b057a65a778ee2f5
I20260709 12:56:35.950953 13428 catalog_manager.cc:5028] TS d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): tablet 82077fd60c1346d7b057a65a778ee2f5 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:35.952435 14243 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:35.953956 15000 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:35.960151 15014 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.968743 13692 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:35.971889 15000 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:35.973624 14243 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:35.989615 14245 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:35.991266 14968 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:36.030360 13693 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:36.030901 13617 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:56:36.032034 14968 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:36.032898 14966 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:36.057098 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4b9bdd77e564429c9c0434b358f48302 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:36.074884 13693 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.074632 14966 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.095847 14244 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:36.098054 15011 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:36.113431 13538 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:36.115136 15011 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:36.166936 13613 raft_consensus.cc:2955] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.184931 13441 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 4 to 5, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:36.187709 13441 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.190609 13537 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:56:36.193156 15033 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:56:36.198117 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:36.203212 14242 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.210955 15000 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
W20260709 12:56:36.203871 15030 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:56:36.254659 13613 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.291065 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b9bdd77e564429c9c0434b358f48302 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49524
I20260709 12:56:36.317210 15045 tablet_replica.cc:333] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:56:36.327944 15045 raft_consensus.cc:2243] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:36.330623 15045 raft_consensus.cc:2272] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:36.339895 15045 ts_tablet_manager.cc:1916] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:36.359573 15045 ts_tablet_manager.cc:1929] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:56:36.360141 15045 log.cc:1199] T 4b9bdd77e564429c9c0434b358f48302 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/4b9bdd77e564429c9c0434b358f48302
I20260709 12:56:36.362476 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet 4b9bdd77e564429c9c0434b358f48302 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:36.365553 13537 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:36.373566 14246 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:36.374019 13617 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:36.376019 15024 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:36.376735 14966 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:36.383770 14966 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.386137 13617 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.390918 14246 raft_consensus.cc:2955] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.405015 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2ee08c4df55f41c7a18951168c44fb5a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:56:36.406704 13440 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 4 to 5, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:56:36.420107 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee08c4df55f41c7a18951168c44fb5a with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45304
I20260709 12:56:36.421943 15046 tablet_replica.cc:333] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:56:36.434393 15046 raft_consensus.cc:2243] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:56:36.435184 15046 raft_consensus.cc:2272] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:56:36.438766 15046 ts_tablet_manager.cc:1916] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:56:36.453943 15046 ts_tablet_manager.cc:1929] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:56:36.454309 15046 log.cc:1199] T 2ee08c4df55f41c7a18951168c44fb5a P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/2ee08c4df55f41c7a18951168c44fb5a
I20260709 12:56:36.455616 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 2ee08c4df55f41c7a18951168c44fb5a (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:56:36.494627 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:36.534803 13693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:36.535356 13693 raft_consensus.cc:606] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Received request to transfer leadership to TS ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:36.537771 13617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a315f47e47184081bd93e74c72dba511"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:56582
I20260709 12:56:36.538309 13617 raft_consensus.cc:606] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:36.540967 13617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:56582
I20260709 12:56:36.541502 13617 raft_consensus.cc:606] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:36.543776 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:56:36.545549 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:36.780591 15013 raft_consensus.cc:993] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: : Instructing follower ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:56:36.781131 15011 raft_consensus.cc:1081] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Signalling peer ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:56:36.782604 13537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:49526
I20260709 12:56:36.783093 13537 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:36.783442 13537 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.790200 13537 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:36.791930 13537 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:36.793006 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:36.793090 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:36.793625 13692 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:36.793929 13692 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:36.793682 13617 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.794760 13692 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:36.795734 13692 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.798983 13617 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:36.800153 13692 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:36.800177 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:36.801263 14968 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:36.801714 14968 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:36.802415 14968 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:36.810660 15000 raft_consensus.cc:993] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:36.811192 15000 raft_consensus.cc:1081] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:56:36.812757 13692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45328
I20260709 12:56:36.813282 13692 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:36.813797 13692 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.813220 13441 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 2, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:36.820590 13692 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:36.826154 13692 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:36.826812 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:36.827147 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:36.827445 13537 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.827759 13617 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:36.828114 13617 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:36.828744 13617 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:36.829562 13617 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.835124 13537 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:36.835189 13617 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:36.836318 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:36.837045 15011 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:36.837531 15011 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:36.838435 15011 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:36.849548 13441 catalog_manager.cc:5697] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:36.891939 15031 raft_consensus.cc:993] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:36.892496 15000 raft_consensus.cc:1081] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:56:36.893980 14246 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a315f47e47184081bd93e74c72dba511"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59162
I20260709 12:56:36.894474 14246 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:56:36.894825 14246 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.899093 14246 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:36.900604 14246 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:36.901630 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:36.901511 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:36.902279 13617 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:36.902704 13617 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:36.902622 13692 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.903503 13617 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:36.904752 13617 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:36.910058 13692 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:36.911219 13617 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:36.911176 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:36.912142 14854 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:36.912609 14854 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:36.913403 14854 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:36.921998 13441 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:38.643301 15060 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:38.644300 15060 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:38.647109 15060 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:38.491675 15059 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:38.779831 15059 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:38.783102 15059 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:39.389724 13714 debug-util.cc:398] Leaking SignalData structure 0x7b0800040520 after lost signal to thread 12412
I20260709 12:56:39.428596 15058 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.429980 15058 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.434162 15058 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:39.434970 15058 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.435959 15058 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.438977 15058 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.439733 15058 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.444185 15058 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.448523 15059 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:39.449446 15060 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:39.592342 15059 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.579320 15061 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.597060 15061 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.592470 15060 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.596194 15059 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.602312 15060 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:39.602483 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.248s	user 0.000s	sys 0.066s
I20260709 12:56:39.603446 15061 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:39.603713 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.250s	user 0.002s	sys 0.066s
W20260709 12:56:39.606293 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 2.3138e+06us
W20260709 12:56:39.608148 13657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.259s	user 0.028s	sys 0.052s
I20260709 12:56:39.610184 15058 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.635373 15064 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:39.636651 15064 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:39.610406 13657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.261s	user 0.030s	sys 0.052s
I20260709 12:56:39.612977 13693 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:56:39.660357 13507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.023s	sys 0.008s
W20260709 12:56:39.662773 13507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.024s	sys 0.008s
I20260709 12:56:39.666986 15062 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.668071 15062 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.671276 15062 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:39.682832 13656 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.363s	user 0.017s	sys 0.029s
W20260709 12:56:39.683594 13656 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.364s	user 0.018s	sys 0.029s
I20260709 12:56:39.700287 15067 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:39.701301 15067 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.708989 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:39.712757 15064 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.713562 15013 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:39.703982 15062 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:39.721320 15062 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.739440 15064 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:39.740324 15064 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.740608 15062 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:39.745618 15064 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:39.748528 15063 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:39.750638 15058 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:39.750854 13538 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:39.751262 15063 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.751399 15058 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.752342 15025 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:39.753202 15025 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.714169 15013 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.753995 15063 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.754038 15058 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.750887 15061 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:39.754395 15065 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:39.903306 15067 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:39.760704 15064 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:39.905550 15064 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.758301 15062 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:39.751410 15073 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:39.909430 15072 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.915481 15077 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:39.916536 15077 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.910326 15061 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.917001 15065 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.921754 15062 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.924881 15076 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:39.925642 15064 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.925812 15067 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:39.925899 15076 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.930197 15073 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.926591 15067 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:39.934105 15061 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.908989 15058 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.929589 15076 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.935837 15067 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:39.944972 15073 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:39.945308 15070 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.970616 15070 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.973212 15070 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:39.945799 15058 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.021114 15058 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:39.946447 15072 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.025664 15072 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.050750 15063 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.052938 15063 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.058303 15063 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.063674 15064 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.189877 15064 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.064826 15075 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.192557 15075 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.066449 15071 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.196719 15071 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.074046 15084 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.200446 15084 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.203419 15084 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:39.946945 15025 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.204419 15025 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.205113 15025 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:39.948886 15074 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.208432 15074 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.210945 15080 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.211370 15074 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.950591 15062 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:39.951395 15013 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.211699 15080 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.214493 15079 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.961346 15077 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.214671 15080 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:39.943504 15068 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.215703 15068 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.167186 15058 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.168174 15059 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.168251 15066 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.169425 15078 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.172340 13692 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:40.224634 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.174978 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.175899 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.180898 15072 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.181885 15067 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.187043 15076 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:39.946609 15065 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.229950 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.207710 15025 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.195717 15075 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.230800 15078 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.233515 15067 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.233517 15076 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.238221 15067 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.238438 15076 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.239018 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.199851 15071 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.224289 15079 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.242878 15078 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.246351 15068 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.251739 13685 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:40.233405 15059 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.253700 15077 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.258344 15063 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.260413 15077 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.261344 15059 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.261384 15063 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.262303 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.255616 15084 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.250512 15058 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.253077 15075 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.240690 15066 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.263581 15077 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.263717 15063 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.269538 15064 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.271349 15078 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.256703 15088 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.283753 15085 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.287883 14854 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.278158 15067 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.265693 15072 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.278625 15065 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.285925 15073 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.302073 15065 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.305150 15065 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.288494 14854 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.286811 15025 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.284550 15077 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.284510 15075 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.324576 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.325594 14854 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.326121 15075 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.325876 15025 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.326233 14854 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.300678 15070 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.289618 15084 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.297302 15078 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.312191 15076 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.319947 15074 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.318221 15060 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.323848 15071 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:40.330775 13714 debug-util.cc:398] Leaking SignalData structure 0x7b080017bd40 after lost signal to thread 13475
I20260709 12:56:40.342636 15069 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.333292 15061 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.342124 15087 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.339960 15067 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.340046 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.343266 15069 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.330433 15083 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.333418 15088 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.356755 15086 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.355546 14854 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.328629 15073 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.331458 15079 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:40.359705 13659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.015s	sys 0.014s
W20260709 12:56:40.364065 13659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.016s	sys 0.014s
I20260709 12:56:40.361006 15061 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.351189 15070 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.366818 15062 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.343423 15071 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.374089 15073 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.374747 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.374874 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.375463 13691 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:40.377260 15084 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.377413 15065 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:40.377538 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.378279 13542 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:56:40.378054 15065 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.357829 15067 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.385725 15090 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.386866 15090 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.388999 15011 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:40.390570 15093 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.391012 15073 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.327286 15085 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.347183 15077 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.366940 15076 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.365285 15068 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.366662 15086 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.404557 15072 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.405413 15090 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.406435 15086 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.367601 15060 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.407636 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.408480 13542 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:40.412894 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.394058 15079 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.414023 14854 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:40.402249 13714 debug-util.cc:398] Leaking SignalData structure 0x7b080023a1c0 after lost signal to thread 13568
I20260709 12:56:40.391736 15084 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.354717 15087 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.391664 15073 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.416677 14854 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.430327 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.430547 15091 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.431062 14246 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:40.427531 13689 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:40.392110 15074 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.398483 15062 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.414163 15059 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.438395 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.391302 15093 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.431133 15091 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.414634 15089 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.425015 15061 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.444414 15066 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.433462 15065 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.415689 15068 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.440688 14854 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.440308 15092 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:56:40.443081 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56016.9us
I20260709 12:56:40.398278 15083 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.437530 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.456513 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.441143 15090 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.498875 13543 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:40.499711 15090 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.525437 15062 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.456503 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.525609 15025 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.456511 15088 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.530656 15087 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:40.528806 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 93751.4us
I20260709 12:56:40.458197 15081 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.458119 15059 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.521358 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.460117 15079 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.535610 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.502370 15063 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.539463 13691 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:40.458128 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.543475 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.543745 15070 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.544144 14854 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.544219 14244 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:40.544761 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.544890 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.547933 15071 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.548399 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.546105 14245 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:40.546654 13693 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:40.487066 15084 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.548017 15076 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.553148 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:40.558785 14854 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.561296 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:40.563203 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.564546 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.565336 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:40.563755 14243 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:40.566341 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.564199 14854 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.576146 15095 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:40.577013 15095 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.577600 15095 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:40.565995 13714 debug-util.cc:398] Leaking SignalData structure 0x7b080020db20 after lost signal to thread 13643
I20260709 12:56:40.569339 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.588204 15065 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:40.597127 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.563331 15076 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:40.560365 15081 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.589514 15079 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.606323 15084 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.569877 15093 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.604131 15065 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.634831 15065 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:40.635138 15090 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.603634 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.608333 15073 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.608214 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.636946 13693 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:40.566361 15063 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.639227 15063 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.609966 15059 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.611089 15074 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.612505 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:40.640132 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 220) took 1786 ms. Trace:
I20260709 12:56:40.536760 15089 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.640591 13540 rpcz_store.cc:276] 0709 12:56:38.853222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:40.360178 (+1506956us) service_pool.cc:224] Handling call
0709 12:56:40.360553 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:40.384783 (+ 24230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:40.384802 (+    19us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:40.384829 (+    27us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:40.384845 (+    16us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:40.639804 (+254959us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b54000db954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:40.639841 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:40.639964 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55283072}
I20260709 12:56:40.615865 14246 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:40.526739 15092 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.570206 15071 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.623443 13615 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:40.625756 15091 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.588101 15088 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.626346 15069 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.621964 15075 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.631363 14854 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.613211 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.641791 15077 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.645447 15059 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:56:40.645444 15084 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.648744 15071 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:40.649559 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:40.650183 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.650439 13658 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:40.652061 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.661855 13541 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:40.645949 15074 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:40.659260 15083 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.664163 15087 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:56:40.665202 13714 debug-util.cc:398] Leaking SignalData structure 0x7b080032cb20 after lost signal to thread 13717
I20260709 12:56:40.665658 15087 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.668957 15087 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:40.664038 15075 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.668512 15084 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.646293 15085 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.661947 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.673883 15089 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.661697 15088 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.675048 15092 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:40.668318 13714 debug-util.cc:398] Leaking SignalData structure 0x7b08002b1880 after lost signal to thread 13718
I20260709 12:56:40.682834 15096 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.664619 15074 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.661820 15082 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:40.682852 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.684478 14246 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:40.667074 15059 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:40.670909 15075 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.684254 13538 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:40.686111 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.686553 15071 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:40.686795 13534 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:40.686985 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:40.684350 15074 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:40.671149 15100 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.687849 15084 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.688014 15079 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:40.688485 15079 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.688908 15079 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:40.689520 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:40.689942 15088 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.671098 15098 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:40.687027 15059 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.697679 15089 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:40.675834 15081 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.688258 15100 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.769475 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.762256 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.745205 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.742852 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.746847 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.745764 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.671355 14240 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:40.782680 15068 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.782950 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.740590 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:40.741917 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.700388 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.741405 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:40.787547 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 47036.3us
I20260709 12:56:40.746323 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.740761 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.790657 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.745081 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.791365 13612 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:40.744446 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.793094 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.758893 15102 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:40.793937 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.738809 15069 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:40.797379 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:40.784644 15059 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:40.740000 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.768988 15060 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.799288 15025 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.774616 15076 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.804628 15025 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.805899 15078 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.806180 15060 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:40.806594 15076 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:40.807067 15025 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:40.806617 15060 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.811053 15058 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.794409 15102 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.793017 15085 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:40.823231 15085 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.826813 15085 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:40.826956 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.823191 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:40.808090 15060 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:40.757819 15065 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.797519 15069 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.827701 13616 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:40.831410 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.835657 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:40.837459 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:40.814806 15095 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.793606 15089 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.837139 13614 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:56:40.836400 13714 debug-util.cc:398] Leaking SignalData structure 0x7b0800327ee0 after lost signal to thread 14270
I20260709 12:56:40.842089 15077 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:40.849320 15077 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.849416 15099 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:40.849563 15065 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:40.849828 15077 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:40.848719 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.837260 15069 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:40.842170 15078 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:40.854490 15102 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.850008 15099 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.855993 15074 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.860006 13684 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:40.862100 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:40.704495 15096 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.862774 15104 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:40.858270 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.859650 15078 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:40.864341 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:40.852181 13714 debug-util.cc:398] Leaking SignalData structure 0x7b08003624c0 after lost signal to thread 14271
I20260709 12:56:40.865857 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.828137 15098 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.698997 15082 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:40.870404 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.888978 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.958993 13686 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:40.993037 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.988994 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:41.050380 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.111836 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:41.010675 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.116644 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.117300 15089 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.117920 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:41.118995 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.120281 15077 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:41.121351 15096 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.907734 15095 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.122512 15104 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:40.979084 15103 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:41.124707 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.124534 15103 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:41.124794 15104 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:40.887557 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:41.129347 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.131471 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.126590 14211 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:41.130445 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:41.135129 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.135145 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.136521 13536 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:56:41.136600 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.136930 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.137374 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.137632 13535 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:41.034917 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.138458 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:41.055264 14212 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:41.140852 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.141409 13542 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:41.130940 15100 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:41.142208 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:41.140600 15059 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:41.004763 15083 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.138s
I20260709 12:56:41.145253 13542 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:41.145394 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:41.146073 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 148161us
I20260709 12:56:41.146384 14211 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:40.888362 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.126910 15078 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:41.149963 15100 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:41.061277 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:41.152068 15077 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.169030 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.169332 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:41.169629 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.170262 13535 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:41.170290 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.171118 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:41.171386 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.172205 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.152602 15098 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.019585 15106 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:41.174361 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:41.163345 15100 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:41.174583 15106 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:41.175333 15100 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:41.175606 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.176999 15106 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.178601 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:40.957273 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:40.865540 15060 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:41.165151 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:41.132903 15099 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.055747 15079 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:41.139691 15095 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:41.148195 15065 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:41.139528 13614 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:41.171406 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:41.173302 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:41.173121 15105 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:41.173812 15082 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.181788 13542 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:41.184649 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:40.909329 15076 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:41.186524 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:41.182665 15058 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:41.159092 15086 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:41.182112 15074 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.153083 15025 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:41.148372 15087 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:41.203428 13534 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:41.181162 15085 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:41.218665 13687 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:41.222697 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.197754 15105 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:41.214109 15025 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:41.228116 15060 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.228792 15087 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.206460 13535 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:41.208416 15081 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:41.223536 13611 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:41.201997 15065 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:41.400703 13690 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:41.220942 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:41.401367 15103 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:41.230611 15097 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:41.234329 15025 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:41.401808 15076 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.233078 15105 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:41.213850 15095 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:41.213964 15058 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:41.473729 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 289017us
I20260709 12:56:41.229617 15086 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:41.477695 13538 raft_consensus.cc:2393] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:41.475627 15103 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:41.497669 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.498536 13613 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
W20260709 12:56:41.489405 15069 consensus_meta.cc:287] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.617s	user 0.016s	sys 0.000s
I20260709 12:56:41.503540 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.504108 15065 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.510847 15086 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.504226 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.505142 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:41.466811 15081 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.272531 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.273367 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.273882 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.274117 15081 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.492684 15103 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:42.274454 14238 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:41.490113 15097 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.274704 15103 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:42.275098 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.275468 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.276528 15097 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:42.276681 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.530893 15076 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:41.619858 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.565307 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 362) took 1401 ms. Trace:
I20260709 12:56:41.620091 13508 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:42.567795 15108 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:56:42.567993 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 948045us
I20260709 12:56:42.568922 13508 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:42.569882 15108 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:41.831221 15110 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:42.572353 15110 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.574585 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 996) took 1869 ms. Trace:
I20260709 12:56:42.575029 13690 rpcz_store.cc:276] 0709 12:56:40.704727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:40.704840 (+   113us) service_pool.cc:224] Handling call
0709 12:56:40.705169 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:41.463051 (+757882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.574305 (+1111254us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54002ec7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:42.574517 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205863680}
W20260709 12:56:42.160770 13509 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.600s (SENT) [suppressed 2 similar messages]
I20260709 12:56:42.172578 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.173561 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.577867 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.577945 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.578799 13613 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:42.579185 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.175057 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 954) took 1028 ms. Trace:
I20260709 12:56:42.580603 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.180205 13536 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:42.581908 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.582573 13612 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:42.199258 13584 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39902 dropped due to backpressure: service queue is full with 50 items
I20260709 12:56:42.201974 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:42.206148 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.058s	sys 0.094s
W20260709 12:56:42.584270 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.395s	user 0.059s	sys 0.094s
I20260709 12:56:42.584539 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 173) took 1088 ms. Trace:
W20260709 12:56:42.584651 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 1.39593e+06us
I20260709 12:56:42.584940 13611 rpcz_store.cc:276] 0709 12:56:41.495679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.495961 (+   282us) service_pool.cc:224] Handling call
0709 12:56:42.584453 (+1088492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.207216 15118 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:42.213973 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.220961 13619 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:42.589155 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1134) took 1071 ms. Trace:
I20260709 12:56:42.230181 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1061) took 1321 ms. Trace:
W20260709 12:56:42.237394 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55142.8us
I20260709 12:56:42.233884 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:42.151994 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 234349us
I20260709 12:56:42.243440 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:42.669116 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:42.669797 13688 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:42.671084 15081 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:42.671576 15081 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:42.671988 15081 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:42.165584 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2354) took 1296 ms. Trace:
W20260709 12:56:41.557663 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 366519us
I20260709 12:56:41.576709 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:42.182339 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:42.773099 14241 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 367) took 1379 ms (client timeout 1743 ms). Trace:
I20260709 12:56:42.266057 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.278337 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.775146 14245 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
W20260709 12:56:42.775580 14245 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 632) took 1239 ms (client timeout 1639 ms). Trace:
I20260709 12:56:42.775753 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:42.775892 14245 rpcz_store.cc:269] 0709 12:56:41.536330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.265721 (+729391us) service_pool.cc:224] Handling call
0709 12:56:42.775535 (+509814us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.187950 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.268602 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 366) took 1036 ms. Trace:
I20260709 12:56:41.578810 15025 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.776857 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:41.502463 15059 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.537470 15058 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.531836 15069 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:42.783466 13543 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:42.784586 15069 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:42.784708 15136 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:42.278044 15076 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.785192 15136 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:42.786767 15136 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:42.787248 15076 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.543548 15078 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:41.536058 15079 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:42.789561 15078 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.554188 15085 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:41.544745 15083 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:41.545264 15095 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:41.545926 15100 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.796228 14242 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:42.797199 15100 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:41.565804 15105 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:42.286094 13616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.015s	sys 0.014s
W20260709 12:56:42.799070 14242 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 620) took 1891 ms (client timeout 1941 ms). Trace:
W20260709 12:56:42.799499 13616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.291s	user 0.015s	sys 0.014s
I20260709 12:56:42.800228 14243 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:42.800372 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 178) took 1293 ms. Trace:
I20260709 12:56:42.287444 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.801571 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 365) took 1579 ms. Trace:
I20260709 12:56:42.801249 15136 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.800707 13616 rpcz_store.cc:276] 0709 12:56:41.506906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.507081 (+   175us) service_pool.cc:224] Handling call
0709 12:56:41.507463 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:41.507906 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:41.507962 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:41.508609 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 12:56:42.800004 (+1291395us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:42.800045 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.800116 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:42.800155 (+    39us) raft_consensus.cc:1685] finished
0709 12:56:42.800171 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:42.800298 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":1290923,"threads_started":1,"wal-append.queue_time_us":67224,"wal-append.run_cpu_time_us":1839,"wal-append.run_wall_time_us":1011658}
I20260709 12:56:42.802168 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.801954 14243 rpcz_store.cc:276] 0709 12:56:41.221899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.222052 (+   153us) service_pool.cc:224] Handling call
0709 12:56:42.800030 (+1577978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:42.800630 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.801384 (+   754us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54005b18d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:42.801500 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3600569088}
I20260709 12:56:42.803110 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.803278 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.803407 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.804104 14239 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:42.804230 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.805333 13616 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:42.806336 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.807426 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.580087 13542 rpcz_store.cc:276] 0709 12:56:41.146658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.146973 (+   315us) service_pool.cc:224] Handling call
0709 12:56:41.147243 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:41.202924 (+ 55681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.174817 (+971893us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b54000b7cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:42.174997 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79052672}
I20260709 12:56:42.806084 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 277) took 1622 ms. Trace:
I20260709 12:56:42.807757 15105 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:42.808534 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.808636 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.808200 14239 rpcz_store.cc:276] 0709 12:56:41.183746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.194764 (+ 11018us) service_pool.cc:224] Handling call
0709 12:56:41.221247 (+ 26483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:42.805170 (+1583923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.805868 (+   698us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54005b2554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:42.806030 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3612991744}
I20260709 12:56:42.809330 13542 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:42.809842 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.809929 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.810284 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.810499 13616 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:42.810894 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 630) took 1275 ms. Trace:
I20260709 12:56:42.811251 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.811379 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.803861 14243 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:42.579903 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.812503 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.812795 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.812963 13614 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:42.590817 13615 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:42.242977 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.583552 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.813670 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.813819 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.814664 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.814821 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.814934 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.815266 13612 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:42.815419 13616 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:42.815549 13614 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:42.584111 13584 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39902 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1150)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1144)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1164)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1168)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2185)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2184)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1161)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1154)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1163)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1155)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2183)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2186)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2190)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1169)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1153)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1152)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1141)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1159)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1143)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1167)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1165)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1166)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2182)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2179)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2180)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1136)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1137)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1138)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1139)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1140)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1146)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 180)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1147)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1148)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1149)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1156)
I20260709 12:56:42.816294 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.811803 13542 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:42.591930 13610 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:42.825325 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.826341 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.826857 13542 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:42.592434 15118 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.575899 15110 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:42.565883 14240 rpcz_store.cc:276] 0709 12:56:41.164351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.181638 (+ 17287us) service_pool.cc:224] Handling call
0709 12:56:41.619658 (+438020us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:56:42.565181 (+945523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5484928}
I20260709 12:56:42.832860 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.833707 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 633) took 1297 ms. Trace:
I20260709 12:56:42.834049 14240 rpcz_store.cc:276] 0709 12:56:41.536665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.832579 (+1295914us) service_pool.cc:224] Handling call
0709 12:56:42.833645 (+  1066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.835047 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:42.624713 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 71941us
I20260709 12:56:42.816493 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.631322 15120 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:42.634685 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 1.22889e+06us
I20260709 12:56:42.827682 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.840195 15120 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.842545 15120 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:42.843024 13509 connection.cc:256] RPC call timeout handler was delayed by -1.4621727399999145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
I20260709 12:56:42.644816 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.646442 13535 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:42.844493 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.844954 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.845875 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.846169 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.846527 13541 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:42.847322 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.650679 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.848035 13537 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:42.848811 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.849306 13537 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:42.849594 15069 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:42.850076 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.850111 15069 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:42.851117 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.648092 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.651803 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.655038 15128 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:42.666397 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:56:42.669745 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 426335us
I20260709 12:56:42.673484 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:42.185969 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1007) took 1400 ms. Trace:
I20260709 12:56:42.675694 13689 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:42.676775 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:42.681921 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.695698 13687 rpcz_store.cc:276] 0709 12:56:40.868800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.141800 (+273000us) service_pool.cc:224] Handling call
0709 12:56:41.142075 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:41.221497 (+ 79422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.165291 (+943794us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b540001c054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:42.165516 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175620352}
I20260709 12:56:42.755267 14238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 634) took 1216 ms. Trace:
I20260709 12:56:42.765626 13688 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:42.755931 15103 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.763065 15126 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:42.768962 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:42.766534 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 93670.5us
W20260709 12:56:42.773495 14241 rpcz_store.cc:269] 0709 12:56:41.393438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.182014 (+788576us) service_pool.cc:224] Handling call
0709 12:56:42.773032 (+591018us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.774267 13691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:42.776857 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.777753 15081 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:42.774325 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 1.19766e+06us
I20260709 12:56:42.778332 14244 rpcz_store.cc:276] 0709 12:56:41.232506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.267797 (+1035291us) service_pool.cc:224] Handling call
0709 12:56:42.268190 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:42.268328 (+   138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:42.268342 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:42.268390 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:42.268407 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.268439 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:42.268523 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:56:42.777778 14246 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 637) took 1239 ms (client timeout 1533 ms). Trace:
I20260709 12:56:42.780141 13617 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:42.782719 15025 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:42.791740 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:42.803512 15136 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:42.179054 13538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1056) took 1417 ms (client timeout 1654 ms). Trace:
I20260709 12:56:42.583510 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 952) took 1451 ms. Trace:
I20260709 12:56:42.589512 13619 rpcz_store.cc:276] 0709 12:56:41.520457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.220061 (+699604us) service_pool.cc:224] Handling call
0709 12:56:42.220476 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:42.588922 (+368446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.589103 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.816442 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.813385 13613 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:42.818217 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.813769 13618 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:42.785167 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 957) took 1615 ms. Trace:
I20260709 12:56:42.818941 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 636) took 1281 ms. Trace:
I20260709 12:56:42.811259 14239 rpcz_store.cc:276] 0709 12:56:41.535474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.809707 (+1274233us) service_pool.cc:224] Handling call
0709 12:56:42.810821 (+  1114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.590080 13540 rpcz_store.cc:276] 0709 12:56:40.908516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:40.908816 (+   300us) service_pool.cc:224] Handling call
0709 12:56:41.107631 (+198815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:41.139700 (+ 32069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:41.139730 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:41.165890 (+ 26160us) log.cc:844] Serialized 26 byte log entry
0709 12:56:41.206533 (+ 40643us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:41.206570 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:41.206797 (+   227us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:42.229915 (+1023118us) raft_consensus.cc:1685] finished
0709 12:56:42.229936 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:42.230089 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":709,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":40204,"threads_started":1,"wal-append.queue_time_us":313396,"wal-append.run_cpu_time_us":2053,"wal-append.run_wall_time_us":1053282}
I20260709 12:56:42.823380 15131 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:42.823381 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:42.799518 14242 rpcz_store.cc:269] 0709 12:56:40.908130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:40.959727 (+ 51597us) service_pool.cc:224] Handling call
0709 12:56:42.798854 (+1839127us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54005b18d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:42.799015 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3742355328}
I20260709 12:56:42.847404 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.819392 13659 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:42.851897 13537 raft_consensus.cc:2410] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:56:42.852635 13614 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:42.901571 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:42.854635 13509 connection.cc:256] RPC call timeout handler was delayed by -1.3928727399998024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
I20260709 12:56:42.904455 15083 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:42.853732 13539 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:42.858685 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:42.905386 13689 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:42.905121 15083 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.858945 13692 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:42.907284 15083 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:42.854260 15128 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:42.907789 15105 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:42.860682 14237 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:42.908092 15128 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:42.908455 15105 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.909381 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 626) took 1442 ms. Trace:
I20260709 12:56:42.909624 15128 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:42.909876 14237 rpcz_store.cc:276] 0709 12:56:41.467365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.504277 (+ 36912us) service_pool.cc:224] Handling call
0709 12:56:41.504604 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:42.908500 (+1403896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.909093 (+   593us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54007238d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:42.909322 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3118203136}
W20260709 12:56:42.904641 13509 connection.cc:256] RPC call timeout handler was delayed by -1.3541727399997399s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 12:56:42.912635 13509 connection.cc:256] RPC call timeout handler was delayed by -1.3370727399997122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 12:56:42.911504 15128 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:42.862005 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 93085.2us
I20260709 12:56:42.859251 14238 rpcz_store.cc:276] 0709 12:56:41.536966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.275233 (+738267us) service_pool.cc:224] Handling call
0709 12:56:42.692469 (+417236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.861775 15103 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:42.862794 15110 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:42.927634 15110 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:42.927769 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.928099 15110 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:42.928465 14238 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:42.929606 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.930694 14238 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:42.852782 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.932282 13534 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:42.863497 13691 raft_consensus.cc:606] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:42.862615 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.868547 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.865315 15081 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:42.869535 13693 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:42.938313 14244 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:42.870065 13536 rpcz_store.cc:276] 0709 12:56:41.131794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.138189 (+  6395us) service_pool.cc:224] Handling call
0709 12:56:42.180011 (+1041822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:42.581540 (+401529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.583239 (+  1699us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002846d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:42.583432 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395332864}
I20260709 12:56:42.873070 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.941063 15105 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:42.941618 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.943229 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.944291 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.945427 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.873559 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:42.891829 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 68525.5us
I20260709 12:56:42.893877 15095 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:42.892642 13541 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:42.950922 15095 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:42.900154 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.892295 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.951341 15095 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:42.889497 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:42.852427 15069 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:42.856184 13685 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:42.937319 14211 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:42.872280 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:42.954139 14211 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:42.873924 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:42.893712 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 74379.9us
I20260709 12:56:42.950500 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:42.956911 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:42.874274 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:42.958288 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 73062.7us
I20260709 12:56:42.959151 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:42.888156 14243 rpcz_store.cc:276] 0709 12:56:41.537666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.803037 (+1265371us) service_pool.cc:224] Handling call
0709 12:56:42.818890 (+ 15853us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.959586 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:42.961063 13540 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:42.863300 14245 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
W20260709 12:56:42.961633 14245 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 628) took 1430 ms (client timeout 1854 ms). Trace:
W20260709 12:56:42.961984 14245 rpcz_store.cc:269] 0709 12:56:41.531499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.776610 (+1245111us) service_pool.cc:224] Handling call
0709 12:56:42.961578 (+184968us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:42.857735 13686 rpcz_store.cc:276] 0709 12:56:40.785256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:40.785396 (+   140us) service_pool.cc:224] Handling call
0709 12:56:40.795563 (+ 10167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:41.127362 (+331799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.185613 (+1058251us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400159d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:42.185851 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255308544}
W20260709 12:56:42.925961 13509 connection.cc:256] RPC call timeout handler was delayed by -1.2497727399995711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
I20260709 12:56:42.885900 13543 rpcz_store.cc:276] 0709 12:56:41.169403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:41.170355 (+   952us) service_pool.cc:224] Handling call
0709 12:56:41.170641 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:42.783846 (+1613205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:42.784915 (+  1069us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540007e4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:42.785106 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3708795136}
I20260709 12:56:42.955806 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:42.952313 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:56:42.868688 14246 rpcz_store.cc:269] 0709 12:56:41.538003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.187554 (+649551us) service_pool.cc:224] Handling call
0709 12:56:42.777723 (+590169us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:56:42.961627 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59082.7us
I20260709 12:56:42.972246 14242 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:42.966756 15059 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:56:42.972786 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:42.861646 15126 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:42.854497 13690 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:42.931675 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:42.966428 15083 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:42.911631 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.966934 15078 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 12:56:42.869959 13538 rpcz_store.cc:269] 0709 12:56:40.761267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:40.761534 (+   267us) service_pool.cc:224] Handling call
0709 12:56:42.178802 (+1417268us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54002846d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:42.178969 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361832704}
I20260709 12:56:42.975466 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:42.971244 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:42.977270 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:42.973572 15097 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:42.979385 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:42.939199 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.997066 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.998359 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.999027 14244 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:42.966617 15079 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:43.003786 13542 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:42.871905 15118 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:42.980480 13689 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:42.892277 15117 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:42.974416 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:42.891122 15131 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:42.892277 13688 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:42.983371 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.984936 13658 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:42.988476 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:43.016654 13543 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:42.987605 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.969264 15105 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:42.898937 13535 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:42.889909 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.937287 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 631) took 1401 ms. Trace:
I20260709 12:56:42.898499 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.974876 15124 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:42.990389 15083 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:42.992676 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.993959 14243 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:42.978132 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:43.035526 13537 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:56:43.002036 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:42.983572 15081 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:43.001430 14238 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.016883 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.030233 14246 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
W20260709 12:56:43.033280 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:43.039827 14244 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:43.045030 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:43.045787 14244 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:43.046635 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:43.047850 15146 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:43.004136 15079 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:43.048445 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.048671 15079 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.050967 15123 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.051474 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.006227 15076 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:56:43.040468 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 60994.1us
I20260709 12:56:43.058921 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 629) took 1523 ms. Trace:
I20260709 12:56:43.060474 14244 rpcz_store.cc:276] 0709 12:56:41.535072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:43.047525 (+1512453us) service_pool.cc:224] Handling call
0709 12:56:43.047804 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:43.048056 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:43.048086 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:43.048141 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:43.048179 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:43.048250 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:43.058784 (+ 10534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:43.008685 15131 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:43.064069 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:43.062880 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.034489 15083 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.066401 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:43.032774 15124 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:43.070892 15124 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.081261 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:43.037149 15105 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:56:43.081982 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 125168us
I20260709 12:56:43.083308 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:43.085117 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 118297us
I20260709 12:56:43.053010 15120 raft_consensus.cc:2764] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:56:43.060084 15122 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:43.039603 15097 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:43.029978 14241 rpcz_store.cc:276] 0709 12:56:41.535856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.862324 (+1326468us) service_pool.cc:224] Handling call
0709 12:56:42.937214 (+ 74890us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:43.088455 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:43.076143 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 638) took 1454 ms. Trace:
I20260709 12:56:43.065218 14242 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:43.043229 15110 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.092597 14239 rpcz_store.cc:276] 0709 12:56:41.614896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.888605 (+1273709us) service_pool.cc:224] Handling call
0709 12:56:43.069226 (+180621us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:43.060684 15076 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:43.086263 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:43.099364 15076 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.100507 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:43.088568 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:43.085179 15105 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.090046 15146 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.090595 15136 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.109825 15144 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:43.135035 14245 raft_consensus.cc:2393] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:43.098443 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:43.093663 15058 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:56:43.134681 15144 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:56:43.165047 15141 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:56:43.177306 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:43.094868 15131 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:43.182073 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 85361.9us
I20260709 12:56:43.171131 15130 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:43.168496 15153 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:43.171326 15144 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.193735 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:43.174547 15082 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:56:43.202764 13690 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:56:43.205753 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 117630us
I20260709 12:56:43.208455 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:43.228251 13584 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:43.193534 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 278) took 1011 ms. Trace:
I20260709 12:56:43.194051 15153 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:43.229390 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:43.190907 15126 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:43.196724 15130 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:43.206002 15118 raft_consensus.cc:2764] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:56:43.170706 15097 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.233561 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:43.209307 15131 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:43.204191 15152 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:43.236603 13584 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:43.241879 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:43.231824 15128 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:43.233620 15153 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:43.290102 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 221533us
W20260709 12:56:43.288028 13640 debug-util.cc:398] Leaking SignalData structure 0x7b08001411a0 after lost signal to thread 12412
I20260709 12:56:43.232057 15155 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:43.234990 15146 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.240512 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 368) took 1057 ms. Trace:
I20260709 12:56:43.230365 15154 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.281268 15108 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:43.295065 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.288702 15110 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:43.247790 15142 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.288913 15128 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:56:43.309952 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:43.316936 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 334653us
I20260709 12:56:43.298418 15155 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:43.231146 14242 rpcz_store.cc:276] 0709 12:56:42.179678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:43.131031 (+951353us) service_pool.cc:224] Handling call
0709 12:56:43.164752 (+ 33721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:43.172637 (+  7885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:43.174042 (+  1405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:43.176975 (+  2933us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:43.178917 (+  1942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:43.182318 (+  3401us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:43.190735 (+  8417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:43.316581 15025 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.305608 15154 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.305366 15108 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:43.320317 15128 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.320461 15146 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.331009 15126 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:56:43.331142 13640 debug-util.cc:398] Leaking SignalData structure 0x7b080018f6e0 after lost signal to thread 13475
W20260709 12:56:43.351586 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56644.3us
I20260709 12:56:43.356995 13685 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:43.359037 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:43.359885 15151 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:56:43.358146 15126 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.361496 15126 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.353113 14243 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:43.362880 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 279) took 1182 ms. Trace:
W20260709 12:56:43.351677 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 144629us
I20260709 12:56:43.363333 14243 rpcz_store.cc:276] 0709 12:56:42.179980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.960626 (+780646us) service_pool.cc:224] Handling call
0709 12:56:43.362808 (+402182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:56:43.356073 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 293011us
I20260709 12:56:43.317785 15142 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.351574 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.368299 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.364552 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:43.354298 13543 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:43.366674 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.430967 13685 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.369694 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.372355 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.372814 14243 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:43.351040 15154 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.433507 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.297674 15130 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:43.431241 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 79731.9us
I20260709 12:56:43.373929 13540 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:43.432235 13684 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.373391 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.434975 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:43.423669 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59994.6us
I20260709 12:56:43.434592 13686 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.300256 14240 rpcz_store.cc:276] 0709 12:56:42.176469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:43.110312 (+933843us) service_pool.cc:224] Handling call
0709 12:56:43.172220 (+ 61908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:43.185716 (+ 13496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:43.187762 (+  2046us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:43.197046 (+  9284us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:43.208902 (+ 11856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:43.228085 (+ 19183us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:43.233721 (+  5636us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:43.440507 13693 raft_consensus.cc:2393] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:56:43.435377 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 70913.3us
I20260709 12:56:43.435477 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.435201 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.437172 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.438052 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.438970 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.482551 13692 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.438365 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:43.482736 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 43846.6us
I20260709 12:56:43.423113 15025 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.460736 13685 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:56:43.433254 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 201759us
I20260709 12:56:43.483080 13691 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:56:43.481838 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:43.482669 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 45553.5us
I20260709 12:56:43.482666 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:43.424263 15142 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:56:43.484685 15159 log.cc:927] Time spent T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:56:43.437808 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.439471 15085 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:43.516485 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:56:43.489790 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 485280us
I20260709 12:56:43.519253 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:43.520807 13508 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.459576 14242 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:43.525441 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:43.526835 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.468099 15156 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:43.528314 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:43.524175 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:43.489859 13640 debug-util.cc:398] Leaking SignalData structure 0x7b080031a7c0 after lost signal to thread 13568
I20260709 12:56:43.532025 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:43.532907 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:43.488263 15078 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:56:43.538133 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:43.539206 15147 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.539700 15147 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:43.540112 15147 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.542443 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:43.543562 15146 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:43.544147 15146 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:43.544519 15059 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.544943 15059 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.545262 15146 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.548350 15147 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.545321 15059 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.550429 15147 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:43.550922 15146 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.486322 15025 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.576243 15148 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.577055 15148 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:43.577579 15148 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.578783 15160 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:43.579777 15160 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:56:43.580003 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:43.520393 15085 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:43.435047 15155 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.558082 15078 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.490716 15157 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:56:43.563359 15079 consensus_meta.cc:287] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.006s
I20260709 12:56:43.515357 15163 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:43.535306 15120 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:43.582036 15025 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.596519 15161 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:43.467221 15150 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.124s
I20260709 12:56:43.587303 15155 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.591199 15078 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.620378 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.596549 13690 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:43.617461 15059 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:56:43.637156 14238 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.567s	user 0.016s	sys 0.005s
I20260709 12:56:43.637550 14238 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:43.638109 15059 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:43.638260 14238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 282) took 1457 ms. Trace:
I20260709 12:56:43.638638 14238 rpcz_store.cc:276] 0709 12:56:42.180797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.931384 (+750587us) service_pool.cc:224] Handling call
0709 12:56:43.638196 (+706812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:43.597266 15120 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:43.605643 13640 debug-util.cc:398] Leaking SignalData structure 0x7b0800357c20 after lost signal to thread 13643
W20260709 12:56:43.629402 15122 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.001s
W20260709 12:56:43.632926 15105 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.500s	user 0.007s	sys 0.004s
I20260709 12:56:43.467806 15108 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.596987 15079 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.658034 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.682654 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.616880 15161 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:56:43.614292 15095 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
W20260709 12:56:43.664477 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:43.686899 14241 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:43.649907 15120 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:43.683295 15122 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:43.719175 15079 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:43.721005 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 374) took 1542 ms. Trace:
I20260709 12:56:43.721472 14241 rpcz_store.cc:276] 0709 12:56:42.178493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:43.110862 (+932369us) service_pool.cc:224] Handling call
0709 12:56:43.682746 (+571884us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:43.684763 (+  2017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:43.704674 (+ 19911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:43.720916 (+ 16242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11873024}
I20260709 12:56:43.699002 15161 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:43.701628 13640 debug-util.cc:398] Leaking SignalData structure 0x7b08000f2280 after lost signal to thread 13717
I20260709 12:56:43.643234 15085 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.707056 15108 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.740865 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.747278 13688 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.686050 15105 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:43.727952 15076 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.002s
I20260709 12:56:43.776962 14237 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:43.777220 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.778546 14244 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:43.780205 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 377) took 1600 ms. Trace:
I20260709 12:56:43.780568 14244 rpcz_store.cc:276] 0709 12:56:42.179386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:43.106500 (+927114us) service_pool.cc:224] Handling call
0709 12:56:43.778367 (+671867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:43.778936 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:43.780028 (+  1092us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b540044a7d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:43.780150 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1544233216}
I20260709 12:56:43.707442 15095 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.761737 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.775249 15085 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:56:43.781621 13640 debug-util.cc:398] Leaking SignalData structure 0x7b0800101680 after lost signal to thread 14270
I20260709 12:56:43.760032 15076 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.762094 15153 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:43.795989 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.081s	sys 0.119s
I20260709 12:56:43.799531 13693 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:43.812688 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.082s	sys 0.119s
W20260709 12:56:43.813287 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 576770us
W20260709 12:56:43.821558 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:43.812142 15131 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.001s
I20260709 12:56:43.826524 13692 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:43.825727 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:43.828413 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:43.813195 15124 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.003s
I20260709 12:56:43.830417 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:43.834218 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 310741us
I20260709 12:56:43.834867 15164 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:56:43.830231 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 302005us
I20260709 12:56:43.835933 15164 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:43.836341 15164 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.797771 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.843878 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:43.842018 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:43.829455 15146 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:56:43.826556 15153 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:43.829387 15131 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.837618 15152 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:43.863163 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.864048 13687 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.864573 13509 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:56:43.866094 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:43.834878 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:43.868443 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:43.867115 15078 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.860184 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.874935 15131 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:43.879513 15105 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:43.866217 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.881971 15146 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:43.885965 15117 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:43.887818 15146 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.885180 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.842873 15124 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.889622 13690 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.889633 15146 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:43.896723 15076 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:43.886726 15131 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:43.897900 15131 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.889679 15117 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:43.900992 15117 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.902612 15117 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.892911 15078 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:43.894125 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:43.870880 15163 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.920145 15163 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.920620 15163 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.904839 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.923172 13687 raft_consensus.cc:2393] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.914063 15146 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.939745 13619 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:43.938993 15095 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:43.936656 15128 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
W20260709 12:56:43.942286 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 76128.9us
I20260709 12:56:43.947876 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:56:43.943041 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 57955.3us
I20260709 12:56:43.944752 15105 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:43.950446 15105 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:43.951016 15105 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:43.944967 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.952101 15105 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.975391 15105 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:43.975735 15146 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:43.976229 15163 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.978477 15163 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:43.980295 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:43.981094 13687 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:43.939942 15142 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.984366 13690 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:56:43.989202 15148 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.991389 15148 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:43.991642 15069 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:43.992091 15069 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:43.992522 15069 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:43.995293 15120 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:43.959228 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.997179 15124 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.008078 15128 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:44.002977 15083 consensus_meta.cc:287] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.915s	user 0.002s	sys 0.009s
W20260709 12:56:44.004181 15144 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.798s	user 0.008s	sys 0.004s
W20260709 12:56:44.004815 15154 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.001s
W20260709 12:56:44.001032 15126 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 12:56:43.999987 15155 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:43.985865 15142 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:44.055399 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:44.058292 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 353881us
W20260709 12:56:44.027696 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 506826us
W20260709 12:56:44.062850 13507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.023s	sys 0.027s
W20260709 12:56:44.068478 13507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.023s	sys 0.027s
I20260709 12:56:43.944864 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:43.953860 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:43.983266 15164 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:44.023409 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 471069us
W20260709 12:56:44.070785 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 125996us
I20260709 12:56:44.069224 14211 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:43.952919 15156 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.074652 14211 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.075634 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:44.057076 15082 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.012s	sys 0.035s
W20260709 12:56:44.087990 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 134236us
I20260709 12:56:44.087551 13508 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.089234 13617 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:44.090934 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 222560us
W20260709 12:56:44.092182 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 128589us
I20260709 12:56:44.093156 13509 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.093544 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:44.087369 15082 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.013s	sys 0.036s
I20260709 12:56:44.091670 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:44.096410 15165 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:44.096977 15165 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.097959 15165 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:43.991389 15025 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.031615 15083 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.088368 15105 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:44.100725 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:44.070060 15126 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.119984 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.123667 13508 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.123757 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:44.070071 15154 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:44.035858 15144 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.128314 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.091720 15122 consensus_queue.cc:1059] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:56:44.134521 15164 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.132478 15079 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:44.136078 15167 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:44.132217 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:44.137444 14244 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.118789 15118 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:44.131855 15145 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:56:44.136144 15079 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.120484 15155 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:44.125687 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:44.134495 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:44.156970 15160 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.156309 15079 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.157723 15162 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:44.157892 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:44.158210 15162 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:44.158654 15162 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.130314 15105 raft_consensus.cc:697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.159130 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:44.134496 15122 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.156806 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:44.140877 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:44.179530 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.141455 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:44.179842 15145 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.180286 15145 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.132720 15117 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:44.142277 15120 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:44.142642 13543 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.143712 15131 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.130111 15148 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:44.144614 15168 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.177017 15150 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:56:44.200254 15160 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:56:44.148998 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.138394 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:44.156049 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.156298 15058 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.152067 15164 consensus_queue.cc:1059] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:56:44.158499 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:44.139832 15169 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:44.140267 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:44.118844 15124 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:56:44.243594 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.245450 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.248967 13543 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.179392 13610 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.141047 15128 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:44.159689 15105 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.182232 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.252179 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.253465 13537 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.253736 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:44.254352 13536 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.255016 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:44.255666 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:44.180785 15117 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:44.176512 15122 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.238034 15131 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.185132 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:44.266935 15142 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.267382 15142 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:44.268361 15142 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.269685 15142 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.270450 13537 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.271490 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.186029 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.187275 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61671.9us
I20260709 12:56:44.272621 15097 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.189465 14209 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.153877 15069 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.193589 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.277112 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.277263 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.277753 15069 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:44.278426 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.278637 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.279057 13539 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.196532 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:44.200745 15160 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:44.212514 15148 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.203136 15058 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:44.213747 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.216105 15146 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:44.216835 15155 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.219209 15150 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.220140 15025 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:44.178903 15118 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.219673 15162 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.222694 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.160162 15166 consensus_queue.cc:1059] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:56:44.290855 15118 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.280230 15160 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.223472 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.224400 15161 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.290663 15166 raft_consensus.cc:3055] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.294739 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.225106 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.203667 15156 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:56:44.226163 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.226652 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.227584 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.229053 13686 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.229768 13689 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:44.298362 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:44.230325 13659 leader_election.cc:336] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.530s (SENT)
I20260709 12:56:44.299005 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.230098 15154 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:56:44.182158 15167 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:44.302183 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.302815 15161 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:44.302541 15167 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.303208 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.303473 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.303916 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.233641 15178 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:56:44.238615 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:56:44.240352 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.231535 14244 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:44.129942 15078 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.251142 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:44.258750 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 120422us
I20260709 12:56:44.279891 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.258870 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.183784 15145 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.259872 15149 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.258492 15117 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.260723 15164 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:44.261379 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:44.261294 15122 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.263150 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.264360 14238 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.265683 15126 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.274077 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.274919 13543 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:56:44.274979 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 85961.6us
W20260709 12:56:44.272570 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 86610us
I20260709 12:56:44.343793 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.279681 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.281178 13685 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:44.281670 15150 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:44.344578 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.346117 13685 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.281267 15058 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.286705 15025 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:44.281759 15079 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:56:44.281409 15146 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:44.353832 15079 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.354385 15079 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.353986 15146 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.355077 13534 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.355508 15146 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:44.288033 14211 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.356498 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 68587.1us
I20260709 12:56:44.336542 15164 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:44.357892 15103 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.358078 15164 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.280665 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.361344 13619 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.282085 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.361428 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:44.362480 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 80469.4us
I20260709 12:56:44.363517 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.363833 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:44.364652 13539 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.299374 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.282043 15157 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:44.300490 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.318398 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.319937 15169 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.321520 13536 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:44.318063 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:56:44.321987 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.323014 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:44.393911 13691 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.323720 15162 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.339113 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.341205 13584 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:44.296568 15166 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:44.397866 13687 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:56:44.343529 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 69536.4us
I20260709 12:56:44.344516 13537 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.346455 13689 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.398648 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.397897 13686 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.399511 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.400027 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.347419 13692 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.401068 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:44.366597 15156 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.282836 15165 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:56:44.350265 13535 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.351226 13542 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.352955 13690 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.305706 13693 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:44.389009 13538 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:44.389411 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 159094us
I20260709 12:56:44.393366 15120 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:56:44.409081 13659 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.598s (SENT)
I20260709 12:56:44.400976 15131 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:44.393210 15169 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:44.347178 15161 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:44.421715 15161 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:44.404500 15147 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:56:44.422125 15161 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.413250 15131 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:44.422833 15120 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.423038 15131 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.423301 15120 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.424068 15025 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:44.398301 15144 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.403431 15069 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:44.347268 13685 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.349246 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.281021 15148 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.368752 15083 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:44.387427 15175 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.430703 15169 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:44.410080 13659 leader_election.cc:336] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.654s (SENT)
I20260709 12:56:44.433561 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.396894 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 55748.2us
I20260709 12:56:44.434044 15169 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.434535 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:44.435884 13584 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.437412 13584 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.355399 15079 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:56:44.442857 13582 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.579s (SENT)
I20260709 12:56:44.393247 15123 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.406234 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.404619 13688 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:44.395817 15110 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.415062 15097 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.417418 15058 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.403016 15081 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.421005 15085 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:56:44.396853 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.406677 15122 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.424371 15166 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.448983 15175 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:44.430894 15157 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.424280 15120 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.405812 15126 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.395429 15095 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:56:44.406796 15153 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:44.467051 15153 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:44.467517 15153 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:44.468096 15095 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.468459 15095 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.469609 15184 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.469446 15095 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.469964 15184 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.439604 13537 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:44.455430 15168 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:56:44.445976 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 45179.7us
I20260709 12:56:44.436151 15128 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:44.451789 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:56:44.453500 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47374.6us
I20260709 12:56:44.485244 15095 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.543493 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.454218 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 105004us
I20260709 12:56:44.545125 14209 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.451421 15079 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.428601 15069 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:44.461588 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:44.429976 15103 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.452183 15142 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.548810 15069 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.550289 15142 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.548836 15103 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.552901 15103 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.553928 15103 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.554913 13541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:44.555554 13541 raft_consensus.cc:606] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:56:44.453056 15144 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.445195 13539 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:44.455451 15105 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.528826 15145 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.526292 15120 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.462796 15124 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:44.520740 15178 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:44.539860 15175 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:56:44.520262 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.540803 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.464831 15157 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.548296 15059 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.539860 15128 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:44.428486 15150 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.463712 15097 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.573104 15128 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.559494 15100 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.570885 15124 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:44.582304 15183 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:56:44.580101 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:44.587067 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.567793 15025 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:44.571883 15081 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:44.560345 15161 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.575232 15097 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:44.573493 15154 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.589144 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.591035 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.570005 15157 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.602312 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.602982 15157 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.605196 13658 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.606114 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.606033 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.598126 15025 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.579773 15178 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.610455 15105 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.611613 15178 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:44.611164 15025 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.612080 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.613066 15187 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:44.614612 15187 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.615090 15187 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.616206 15187 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.622959 14209 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.593317 15175 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:44.590195 15123 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.628830 14209 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.630295 15149 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.599766 15150 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.604238 15081 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.600672 15154 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:44.608474 15148 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:44.558874 15184 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.622020 15100 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.621819 15118 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.593180 15147 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 12:56:44.648655 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.630625 15185 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:56:44.629889 15076 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:44.604069 15164 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.644893 15165 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.655311 15076 raft_consensus.cc:697] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:56:44.657338 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 205625us
I20260709 12:56:44.642278 15175 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.638396 15153 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.654124 15149 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.666656 13691 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:44.666163 15123 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:44.672410 13610 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.672698 15123 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.665560 15076 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.674103 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:56:44.679896 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.682209 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:56:44.655409 15069 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.685041 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:44.671060 15184 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.690018 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:44.687451 15189 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:44.646139 15097 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.688791 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:44.693815 13617 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:44.651326 15154 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.692709 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:44.688800 13584 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:44.691620 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:44.695338 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:44.697736 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.698575 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.700824 13613 raft_consensus.cc:2393] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:44.688555 15124 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.688019 15144 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.689004 15078 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:56:44.682566 15178 consensus_queue.cc:1059] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:56:44.700991 15142 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.700826 15146 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:44.705909 15166 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.705374 15154 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.723291 15161 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:44.724174 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.714437 15097 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.702452 15131 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.711308 15124 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.729053 15069 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.738547 15123 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.742334 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:44.743377 13613 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:44.692268 15081 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.718648 15146 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:44.757051 13619 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:44.720155 15175 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.741235 15123 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.755714 15146 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.779541 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:44.787776 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:44.790005 13613 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.781375 15153 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:44.805378 13615 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:44.801450 15128 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.812000 13616 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:44.808992 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:44.823418 13617 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:44.825666 13686 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:44.832975 13618 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:44.833176 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:44.831945 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:44.834318 13615 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:44.835206 13611 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:44.827620 15162 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:56:44.842397 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 380885us
I20260709 12:56:44.845870 13615 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:56:44.847069 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 241961us
I20260709 12:56:44.850996 13611 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:56:44.846900 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:44.832644 15153 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:56:44.839462 15162 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.881335 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:44.893097 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.893522 13539 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:56:44.893524 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.899031 13535 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:44.900507 14239 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:44.902357 15164 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:44.902594 14241 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:44.903530 15058 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.897939 14211 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.878741 15069 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:44.906155 13615 raft_consensus.cc:3055] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.911177 13613 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:44.913043 13615 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:56:44.919801 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 291096us
I20260709 12:56:44.919853 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.878408 15122 consensus_queue.cc:1059] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:56:44.936725 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:44.912448 15069 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:44.919450 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:44.948989 15069 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:44.949128 14238 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.905479 15193 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:44.917577 15105 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:44.950743 15105 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:44.950591 15193 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.951718 15105 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.952571 15118 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:44.952931 15118 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:44.953735 15118 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.952951 15193 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:44.944438 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 245890us
I20260709 12:56:44.931620 13615 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.923138 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.948417 13535 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:44.957342 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:44.970803 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:44.971028 15103 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.959254 15118 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.962301 15146 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:44.965731 13615 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:44.967332 15100 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:44.986717 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:44.989163 13534 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:56:44.989534 13438 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 3 to 4, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 4 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:44.990443 15162 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.991669 15162 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:56:44.992018 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:44.992941 15169 consensus_meta.cc:287] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 12:56:44.992988 15165 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:56:44.994606 15117 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.000s
I20260709 12:56:44.994976 15117 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:44.991385 13438 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:45.004034 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.004613 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.005288 13610 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:45.005422 15178 consensus_queue.cc:1059] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:45.006713 15178 raft_consensus.cc:3055] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:45.007192 15178 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:45.006635 13438 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:56:45.011102 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53845.3us
I20260709 12:56:45.009320 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.012369 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.012940 13613 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:45.014423 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:45.014813 15156 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.001s
I20260709 12:56:45.012297 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.015558 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.016575 13618 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:45.012018 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:45.016108 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.017989 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:44.946935 15153 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.021567 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.022001 13617 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.023232 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.023761 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.023958 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.023799 13617 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
W20260709 12:56:45.027339 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.015236 15156 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.028251 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.032680 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.009817 15142 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.045465 13685 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.048053 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:45.048792 13686 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:45.055972 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:45.050494 13685 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.058633 13685 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:56:45.060261 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.061491 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:45.064236 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.066877 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:45.067477 13684 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:45.070561 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.031625 15122 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:45.027072 15149 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.018601 15150 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:56:45.074324 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.066766 15164 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.076610 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.077920 15147 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:45.029379 15126 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 12:56:45.078895 15197 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:56:45.029949 15167 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:45.063642 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.049898 15196 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:56:45.064881 15142 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.066421 13509 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.099959 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.054704 15194 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:56:45.029347 15165 consensus_queue.cc:1059] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:56:45.103554 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.073227 15156 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:56:45.120765 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:56:45.112272 15160 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.820s	user 0.006s	sys 0.008s
I20260709 12:56:45.113351 13542 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:45.117141 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 50786.6us
I20260709 12:56:45.145371 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:45.146648 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:45.103777 15157 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.004s
I20260709 12:56:45.008302 15178 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.118326 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:45.098459 13534 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.102299 15147 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:45.156945 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.156895 15147 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.158206 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.064929 15168 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:56:45.111155 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:45.133304 13616 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:45.133914 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:56:45.166106 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55052.3us
I20260709 12:56:45.130331 13539 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:45.112965 13538 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.100804 15146 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:45.119345 15124 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.148464 15128 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.095194 15122 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:45.160221 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.158049 15147 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.170861 15122 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:45.174986 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.145403 15194 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:45.175282 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.127940 15172 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.179859 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.115437 15148 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:56:45.181988 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.181910 13582 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:45.118386 15198 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:56:45.188164 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.184036 15124 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:56:45.149062 15153 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:56:45.194465 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.159128 13535 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.155139 15126 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:45.112026 15079 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:56:45.183096 15194 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:45.158684 15164 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:45.188431 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:45.198799 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.158325 15167 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.210417 13617 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:45.227982 15194 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.213725 13659 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.233470 13611 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:56:45.236349 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.237113 15211 consensus_queue.cc:1059] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:56:45.237498 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.178624 15142 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:56:45.193805 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:45.238940 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.215179 14243 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:45.217372 14244 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:45.192770 13542 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:45.226059 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.226965 14245 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.150858 15160 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.178896 15117 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.193395 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.233579 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.193369 14242 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.232717 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.223817 15122 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.190136 15161 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:56:45.235316 13534 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
W20260709 12:56:45.257952 15201 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:56:45.254786 15147 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:45.258556 13536 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.255730 15124 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:45.259227 15124 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:45.259539 15142 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:45.257725 15118 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.229116 15178 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.233317 15126 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:45.234362 14238 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:45.261643 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:45.266798 13540 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.280580 13615 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:45.281562 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.282941 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.250903 15120 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.251204 15128 raft_consensus.cc:3055] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:45.287971 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.289450 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:56:45.270491 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 88685.3us
I20260709 12:56:45.291957 15191 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.294298 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:56:45.299779 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:45.264635 15204 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:56:45.268254 15175 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.276247 15117 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.259944 15142 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:45.284063 15103 raft_consensus.cc:2764] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:56:45.260519 14240 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:45.259845 15069 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.267429 15131 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.292980 15148 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:45.259899 14239 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:45.304971 15193 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.324102 14244 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:56:45.312125 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 185122us
I20260709 12:56:45.290154 15185 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:56:45.293519 15144 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:56:45.286798 15128 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:45.260205 15189 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:56:45.319139 14245 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:56:45.319978 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.323572 13618 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:45.332530 13508 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:45.332996 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:45.340209 13582 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:45.340674 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 355029us
I20260709 12:56:45.324971 14243 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.334740 13537 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.312819 15198 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.317638 15191 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.319387 15164 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:45.260083 15124 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.345269 13538 raft_consensus.cc:2393] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:45.352362 13692 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:45.358906 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:45.425738 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 66944.4us
I20260709 12:56:45.360249 15215 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.431474 13693 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:45.420097 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.335525 15185 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:45.418164 15164 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.346299 15120 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.444334 15120 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.446817 15120 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:45.427158 15218 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.427237 15128 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.344869 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.416729 14242 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:45.424660 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.428702 15069 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.417204 15160 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.343704 15154 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.360100 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.422289 15198 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.416882 15131 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:45.443720 15217 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:45.337886 15193 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.449095 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.448637 15128 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.448465 15218 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.434463 15083 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:56:45.457986 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.481052 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:45.344697 15126 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.442629 15185 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:45.454978 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.434777 15100 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:56:45.467376 15187 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:45.452533 15166 consensus_meta.cc:287] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.728s	user 0.006s	sys 0.006s
I20260709 12:56:45.469261 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.492425 13538 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:45.473380 13687 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.473814 15164 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.437414 15118 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:56:45.477448 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:45.474699 15131 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:45.488150 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:45.488778 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:45.489460 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.495869 14237 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:45.499962 15218 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.499650 15166 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:45.500746 15166 consensus_queue.cc:1243] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:56:45.485255 15217 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:45.504570 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:45.506052 15123 consensus_meta.cc:287] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.001s
I20260709 12:56:45.505977 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:45.506356 15198 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.506735 13689 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:45.507078 15217 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.492278 15160 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.512569 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:45.515074 15097 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.516121 14241 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:45.519956 15167 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.502221 15131 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:45.521975 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:45.523267 13689 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:56:45.522584 15213 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:56:45.501843 15164 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:45.520936 15193 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:45.510717 15078 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.516937 15198 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:45.554354 15211 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:45.554832 15211 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:45.543051 15198 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:45.557387 15198 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:45.662684 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 48645.3us
I20260709 12:56:45.556418 15211 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.667937 15211 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.662230 15025 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.656322 15076 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.689044 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:45.671800 15124 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:45.745271 13686 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:45.745369 15142 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:45.751006 15131 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.670650 15157 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:56:45.697253 15025 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.754386 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:45.762530 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.765735 13689 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:45.769316 15103 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:45.770512 15161 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:45.702495 15154 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:45.771477 15161 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.772380 15154 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:45.773572 15154 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.701882 15076 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:45.776481 15161 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:45.778156 15076 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:45.784436 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:45.787593 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:45.788642 14243 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:45.720304 15117 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.805843 15025 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:45.793749 15161 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.808352 15120 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:45.808893 15120 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:45.809262 15194 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:45.813838 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.818670 13689 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
W20260709 12:56:45.831732 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.837242 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:45.841523 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.842290 13616 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:45.856180 13616 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.830497 15161 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:56:45.864081 15161 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:45.864508 15161 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:45.865507 15161 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.867149 15161 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:45.809983 15120 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.840760 15169 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:56:45.868502 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.871510 13618 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:45.881628 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:45.882333 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:45.882773 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.886819 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:45.887501 13610 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.887717 13613 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:56:45.868791 15120 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.888377 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.892030 13618 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:45.897938 15198 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.899567 13614 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:56:45.900214 15198 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:45.902565 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:45.904072 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.906742 14243 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:45.910305 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:45.911760 14243 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:45.913031 15169 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:45.889398 15124 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.890928 15165 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:45.915524 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.921666 15160 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:45.924124 15193 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:45.928785 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:45.903184 15153 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.930002 15185 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.932233 13693 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:45.933321 15185 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:45.935348 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:45.936576 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.925043 15165 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:45.910216 15076 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:45.938041 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:45.945114 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:45.945461 13539 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:45.947145 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:45.943979 15202 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:56:45.944604 15095 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:56:45.944893 15191 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:45.937099 15150 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:56:45.955403 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.964123 13685 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:45.964905 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:45.965852 13539 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:45.951072 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:45.939596 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:45.931102 15124 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.959470 15169 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.935396 15160 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:45.942445 15165 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:45.955075 15076 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:45.979794 15079 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:45.981281 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:45.984293 13618 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:45.950407 13535 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:45.949710 13534 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:56:45.951061 15131 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:45.945472 15153 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:45.973116 15191 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:56:46.063091 15201 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:56:46.060096 13617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.013s	sys 0.044s
W20260709 12:56:46.186523 13617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.013s	sys 0.048s
I20260709 12:56:45.972545 15069 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:56:46.188112 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:46.082718 13542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.027s	sys 0.033s
W20260709 12:56:46.188895 13542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.029s	sys 0.034s
I20260709 12:56:46.010268 14246 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:45.989749 13539 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:45.965188 15185 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:56:45.942153 15195 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
W20260709 12:56:46.124349 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 136114us
W20260709 12:56:46.099548 13538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.014s	sys 0.025s
W20260709 12:56:46.191884 13538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.014s	sys 0.026s
I20260709 12:56:45.981868 15167 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:45.986948 15187 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.154191 13536 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:56:46.140839 15213 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.013s	sys 0.029s
W20260709 12:56:46.141168 15126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.005s	sys 0.060s
I20260709 12:56:46.128850 15079 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:46.191171 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:46.123519 15211 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:46.129662 15216 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.203172 13543 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
W20260709 12:56:46.183840 15081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.125s	user 0.019s	sys 0.066s
W20260709 12:56:46.202032 15213 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.013s	sys 0.029s
I20260709 12:56:46.194061 15193 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:46.235071 15193 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:46.235669 15193 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:46.238391 13685 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:56:46.239902 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:46.240916 15220 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:56:46.241484 15220 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:46.242347 15220 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:46.233059 15081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.019s	sys 0.067s
I20260709 12:56:46.245947 15124 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:46.194267 15167 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:46.153043 15097 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.252426 15097 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.252691 15167 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:46.222462 13584 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f54dde33fa845098e64673b576d780c. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:56:46.253690 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 968616us
I20260709 12:56:46.254004 15167 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:46.222270 15079 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:46.223883 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1079) took 1968 ms. Trace:
I20260709 12:56:46.224545 15118 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:46.256978 15079 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:46.257256 15118 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:46.257714 15118 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:46.258428 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:46.259303 14240 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:46.225024 15148 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:46.260809 15148 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:46.261204 15148 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:46.262169 15148 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:46.263348 15148 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:46.129403 15194 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.260965 13584 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:46.210973 15211 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.256660 13540 rpcz_store.cc:276] 0709 12:56:44.254908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:44.256923 (+  2015us) service_pool.cc:224] Handling call
0709 12:56:44.260935 (+  4012us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:44.261106 (+   171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:45.280761 (+1019655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:46.223522 (+942761us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002ba7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:46.223816 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317587584}
I20260709 12:56:46.267809 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:46.204883 15126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.006s	sys 0.060s
I20260709 12:56:46.245762 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:46.228268 15216 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.204448 15187 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.204839 15161 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.249933 15153 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:46.150558 15160 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.278584 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:46.310858 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.277467 15100 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.268877 15189 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:56:46.336503 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 58006.5us
I20260709 12:56:46.232105 15212 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:56:46.341370 15126 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.341841 15211 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:46.287742 15161 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:46.342598 15222 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:56:46.335064 15202 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.277595 15124 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:46.345504 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:46.337266 15100 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.261011 15191 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:46.347843 13686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.020s	sys 0.049s
I20260709 12:56:46.334784 15131 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:56:46.362802 13686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.022s	sys 0.052s
I20260709 12:56:46.363626 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 303) took 1097 ms. Trace:
I20260709 12:56:46.364500 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.364293 14245 rpcz_store.cc:276] 0709 12:56:45.266254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.266407 (+   153us) service_pool.cc:224] Handling call
0709 12:56:45.316948 (+ 50541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:45.338386 (+ 21438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:46.363382 (+1024996us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b54006d15d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:46.363550 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62987008}
W20260709 12:56:46.366482 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.362848 15202 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.389760 15100 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:46.390195 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1096) took 1053 ms. Trace:
I20260709 12:56:46.350459 15169 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:46.365322 13615 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:46.401899 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.402273 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 272) took 1177 ms. Trace:
I20260709 12:56:46.402735 13615 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:46.276041 15194 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:46.395553 13611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.000s	sys 0.021s
I20260709 12:56:46.390563 15160 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:46.410660 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.411471 13615 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:56:46.400588 13616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.007s	sys 0.016s
W20260709 12:56:46.413612 13616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.007s	sys 0.016s
W20260709 12:56:46.406157 13611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.000s	sys 0.021s
W20260709 12:56:46.400969 13691 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.018s	sys 0.005s
I20260709 12:56:46.398577 13538 rpcz_store.cc:276] 0709 12:56:45.336967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.492860 (+155893us) service_pool.cc:224] Handling call
0709 12:56:45.498812 (+  5952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:45.503288 (+  4476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:45.503318 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:45.510344 (+  7026us) log.cc:844] Serialized 26 byte log entry
0709 12:56:46.192364 (+682020us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:46.192419 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:46.192507 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:46.389900 (+197393us) raft_consensus.cc:1685] finished
0709 12:56:46.389929 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:46.390101 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4123,"raft.queue_time_us":5104,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":7391,"spinlock_wait_cycles":57088,"thread_start_us":677293,"threads_started":1,"wal-append.queue_time_us":871433}
W20260709 12:56:46.409646 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.350596 15126 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:46.402786 13542 rpcz_store.cc:276] 0709 12:56:45.224862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.446333 (+221471us) service_pool.cc:224] Handling call
0709 12:56:45.446668 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:45.457809 (+ 11141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:45.457838 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:45.465504 (+  7666us) log.cc:844] Serialized 26 byte log entry
0709 12:56:45.927434 (+461930us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b4800002dd8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:56:46.189311 (+261877us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:46.189355 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:46.189423 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:46.392173 (+202750us) raft_consensus.cc:1685] finished
0709 12:56:46.392219 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:46.402195 (+  9976us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10806,"raft.queue_time_us":10994,"raft.run_cpu_time_us":17376,"raft.run_wall_time_us":464334,"spinlock_wait_cycles":8739200,"thread_start_us":704219,"threads_started":1,"wal-append.queue_time_us":808474}
I20260709 12:56:46.416198 14212 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:46.424441 15194 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:46.426515 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.436757 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 271) took 1307 ms. Trace:
I20260709 12:56:46.436604 15118 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:46.436801 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.426038 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:46.437640 13542 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:46.437244 13541 rpcz_store.cc:276] 0709 12:56:45.129900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.441012 (+311112us) service_pool.cc:224] Handling call
0709 12:56:45.441360 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:45.463821 (+ 22461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:45.464313 (+   492us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:45.476044 (+ 11731us) log.cc:844] Serialized 26 byte log entry
0709 12:56:45.846240 (+370196us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:45.858603 (+ 12363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:45.859886 (+  1283us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:46.428933 (+569047us) raft_consensus.cc:1685] finished
0709 12:56:46.429497 (+   564us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:46.436679 (+  7182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9284,"raft.queue_time_us":13940,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5288,"thread_start_us":360857,"threads_started":1,"wal-append.queue_time_us":708821}
I20260709 12:56:46.438890 15118 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:46.444213 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.444108 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:46.425444 15110 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:46.427016 15160 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.429677 13614 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:46.447283 15216 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:46.409487 15185 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:56:46.445066 15131 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:46.450387 15078 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:56:46.459148 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:46.446053 13610 consensus_meta.cc:287] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.558s	user 0.002s	sys 0.010s
I20260709 12:56:46.448223 15189 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:46.449579 15110 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:46.445551 15187 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:46.457230 15078 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:46.417553 13615 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:56:46.498009 13692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.001s	sys 0.019s
I20260709 12:56:46.461474 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:56:46.479777 15217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.004s	sys 0.020s
I20260709 12:56:46.450452 15165 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:46.500584 13689 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.012s	sys 0.008s
W20260709 12:56:46.509366 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.506369 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:46.508368 13619 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.001s	sys 0.017s
I20260709 12:56:46.465269 13610 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:56:46.562220 14244 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.002s	sys 0.022s
I20260709 12:56:46.454449 15160 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:46.561913 15124 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:56:46.572856 15202 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:46.574892 13610 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:46.471457 15076 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:46.524248 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.029s	sys 0.059s
W20260709 12:56:46.498649 13692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.002s	sys 0.019s
I20260709 12:56:46.527462 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:46.510906 15217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.005s	sys 0.021s
I20260709 12:56:46.529241 13541 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:46.521157 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 71781us
W20260709 12:56:46.476090 15154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.023s	sys 0.056s
I20260709 12:56:46.472585 15110 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:46.575898 13618 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:46.576225 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.030s	sys 0.059s
W20260709 12:56:46.580353 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 598607us
W20260709 12:56:46.562949 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 439303us
I20260709 12:56:46.577337 13611 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:46.563537 14244 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.002s	sys 0.022s
I20260709 12:56:46.577759 13584 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:56:46.416862 13691 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.018s	sys 0.005s
I20260709 12:56:46.479557 15194 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:56:46.573133 15205 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.142s	user 0.011s	sys 0.000s
I20260709 12:56:46.530195 15211 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:46.567593 13619 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.005s	sys 0.017s
I20260709 12:56:46.580000 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.642345 15193 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.576814 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:46.587806 15165 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:46.501233 15120 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:56:46.577916 15154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.023s	sys 0.056s
W20260709 12:56:46.567536 13689 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.015s	sys 0.010s
I20260709 12:56:46.640049 15202 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:46.578406 15157 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:46.645360 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:46.671221 13613 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:46.654753 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:46.579973 13618 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:46.565994 13615 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:46.673149 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:46.673592 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:46.644490 13437 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 3, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:46.632367 13611 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
W20260709 12:56:46.635672 15224 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:56:46.576406 13610 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:46.676782 13437 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 3, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:46.682667 13437 catalog_manager.cc:5697] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 2, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:46.584882 15161 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:46.647158 15079 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:46.684831 13437 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 3 to 4, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 4 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:46.576552 15078 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:46.680912 14244 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:46.663601 15118 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:46.602837 15076 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.669700 15211 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.670467 15195 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:46.673385 15095 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:46.713116 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.022s	sys 0.017s
I20260709 12:56:46.631266 15185 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
W20260709 12:56:46.724872 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.023s	sys 0.018s
I20260709 12:56:46.713629 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:46.725324 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 522413us
I20260709 12:56:46.670164 15120 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.728652 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.729262 13537 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:56:46.724013 13509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.013s	sys 0.032s
I20260709 12:56:46.669507 15194 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:46.731324 13509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.013s	sys 0.032s
I20260709 12:56:46.731268 13613 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:46.736171 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:46.736817 13537 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:46.679050 15157 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.717651 15161 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.749737 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:46.733788 15078 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:46.796183 13542 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:46.740365 15069 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.030s	sys 0.069s
I20260709 12:56:46.710767 15079 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.717228 15118 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:46.799448 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 125927us
I20260709 12:56:46.732731 15131 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:46.720727 15076 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.726420 15195 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:46.807447 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:46.801091 15069 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.031s	sys 0.069s
I20260709 12:56:46.748749 15202 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.733935 15095 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:46.794844 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.823064 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:46.843066 13613 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:46.816196 15220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.025s	sys 0.016s
I20260709 12:56:46.798875 15189 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:46.833905 13537 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:46.795014 15161 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:46.803678 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2399) took 1872 ms. Trace:
I20260709 12:56:46.813335 15124 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:46.831990 15247 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:46.848089 15220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.025s	sys 0.018s
I20260709 12:56:46.837471 15216 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:46.855749 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 51061.8us
I20260709 12:56:46.856796 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:46.859123 15236 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:56:46.860082 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:46.863215 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.842036 15260 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:46.865559 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:46.869242 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:46.863063 13613 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:46.867142 13684 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:46.867883 13537 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:46.886363 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.888177 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.888770 13611 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 1.
I20260709 12:56:46.889430 13619 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:46.890362 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:46.868247 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:46.892292 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 314604us
I20260709 12:56:46.892656 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.892968 13584 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:46.891242 13692 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:46.893420 13611 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:46.894672 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:46.896498 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:46.897116 13692 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:46.898834 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:46.899955 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:46.901041 13537 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:46.901197 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.901410 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.903153 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:46.904048 13540 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:46.905683 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:46.903259 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:46.904553 15221 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.907665 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.908846 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:46.914309 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.867156 15220 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.902348 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.921159 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.847851 15202 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.885663 15260 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:46.859520 15189 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.933367 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:46.907308 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:46.957304 13540 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:56:46.860759 15216 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.962987 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2414) took 1142 ms. Trace:
W20260709 12:56:46.929874 15250 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:56:46.955829 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:46.954221 13543 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:46.908339 15168 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:46.858268 15247 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.966202 13685 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:46.967576 14245 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:46.919792 15025 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:46.920301 15211 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:46.958264 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:46.963493 13689 rpcz_store.cc:276] 0709 12:56:45.820623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.820878 (+   255us) service_pool.cc:224] Handling call
0709 12:56:45.830833 (+  9955us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:45.831239 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:45.831279 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:45.834968 (+  3689us) log.cc:844] Serialized 26 byte log entry
0709 12:56:46.670396 (+835428us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:56:46.670453 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:46.670534 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:46.962694 (+292160us) raft_consensus.cc:1685] finished
0709 12:56:46.962724 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:46.962914 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2016,"raft.queue_time_us":79,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":3581,"thread_start_us":790876,"threads_started":1,"wal-append.queue_time_us":841853}
I20260709 12:56:46.961956 13610 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
W20260709 12:56:46.938474 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 76439.4us
I20260709 12:56:46.908226 15221 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.951746 15260 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:46.992383 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:46.970391 13690 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:46.898653 15131 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.968613 15120 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.001713 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:47.003424 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:46.906235 15242 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:47.005867 15262 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:47.002429 13534 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:56:46.933180 15220 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.019251 13538 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:46.848934 13688 rpcz_store.cc:276] 0709 12:56:44.928146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:44.945517 (+ 17371us) service_pool.cc:224] Handling call
0709 12:56:44.968641 (+ 23124us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:45.014267 (+ 45626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:45.014305 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:45.014908 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:56:45.305139 (+290231us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:45.310717 (+  5578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:45.316821 (+  6104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:46.572722 (+1255901us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b2400244bf8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b70409d 00007fa02e33610e 00007fa02e34d88c 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221
0709 12:56:46.732872 (+160150us) raft_consensus.cc:1685] finished
0709 12:56:46.734687 (+  1815us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:46.800091 (+ 65404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39665,"raft.queue_time_us":49204,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":8258,"spinlock_wait_cycles":3948288,"thread_start_us":266395,"threads_started":1,"wal-append.queue_time_us":304431}
I20260709 12:56:46.993571 13535 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:47.028223 15255 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:47.029085 15202 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:47.020202 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:47.033514 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 130102us
I20260709 12:56:46.993369 15211 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:46.981541 15168 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:46.989090 15025 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.012372 15118 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.022733 15194 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.030933 15195 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:47.016180 15266 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:47.041649 15242 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:56:47.055989 15253 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 12:56:47.056574 15249 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:56:47.056998 15168 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.073114 15168 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:47.032749 15110 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.073760 15168 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.075521 15110 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.034966 15076 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.036641 15058 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:47.076262 15058 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:47.076759 15058 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:47.077065 15168 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:47.045300 15167 consensus_meta.cc:287] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.791s	user 0.013s	sys 0.000s
I20260709 12:56:47.057034 15025 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:47.043294 15262 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:47.055085 13542 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:47.043223 15211 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:47.093526 15211 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:47.094409 15220 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.059803 15268 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.054669 15214 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:47.055068 15255 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:47.037485 15161 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.037582 15270 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.085134 15242 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:47.067160 15148 consensus_meta.cc:287] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.803s	user 0.014s	sys 0.000s
I20260709 12:56:47.088310 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 313) took 1099 ms. Trace:
I20260709 12:56:47.089707 15154 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:47.090557 15260 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.063903 15246 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:47.063903 15258 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:47.081292 15251 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:56:47.059902 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:47.100998 13619 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:56:47.102324 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:47.095180 15266 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:47.130594 15123 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:47.131213 15123 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:47.096292 15214 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:56:47.118544 14246 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 386) took 1927 ms (client timeout 1852 ms). Trace:
W20260709 12:56:47.125777 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 65955.3us
W20260709 12:56:47.132126 14246 rpcz_store.cc:269] 0709 12:56:45.190866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.191740 (+   874us) service_pool.cc:224] Handling call
0709 12:56:47.118312 (+1926572us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54006009d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:47.118461 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1506647424}
I20260709 12:56:47.132733 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:47.096676 15221 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.111231 15154 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:56:47.117349 13541 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.005s
I20260709 12:56:47.125728 13615 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:47.126030 15258 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.127523 15161 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:47.131588 15266 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.141945 13610 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:47.142262 15123 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.096323 15268 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.142961 13541 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:47.143318 15123 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:47.136360 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:56:47.136737 15223 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.870s	user 0.000s	sys 0.011s
I20260709 12:56:47.137502 15247 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.137693 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:47.154548 13614 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:47.138017 15142 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:47.156852 15221 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.156876 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2221) took 1842 ms. Trace:
I20260709 12:56:47.125381 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 310) took 1214 ms. Trace:
I20260709 12:56:47.153580 15161 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:47.136138 15214 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:47.153903 15262 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.138927 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:47.160975 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:47.158082 13617 rpcz_store.cc:276] 0709 12:56:45.314345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.315397 (+  1052us) service_pool.cc:224] Handling call
0709 12:56:45.315699 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:45.419131 (+103432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:45.419176 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:45.436838 (+ 17662us) log.cc:844] Serialized 26 byte log entry
0709 12:56:46.186986 (+750148us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:46.187041 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:46.187134 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:47.095082 (+907948us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:56:47.155197 (+ 60115us) raft_consensus.cc:1685] finished
0709 12:56:47.155216 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:47.156815 (+  1599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2911,"raft.queue_time_us":18746,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5162,"spinlock_wait_cycles":6423424,"thread_start_us":804061,"threads_started":2,"wal-append.queue_time_us":796487}
I20260709 12:56:47.140564 15246 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:47.108983 14239 rpcz_store.cc:276] 0709 12:56:45.988819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.989380 (+   561us) service_pool.cc:224] Handling call
0709 12:56:45.992267 (+  2887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:46.688568 (+696301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:46.701243 (+ 12675us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:46.795662 (+ 94419us) log.cc:844] Serialized 26 byte log entry
0709 12:56:46.850464 (+ 54802us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:46.853644 (+  3180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:46.858492 (+  4848us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:47.078708 (+220216us) raft_consensus.cc:1685] finished
0709 12:56:47.080659 (+  1951us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:47.088234 (+  7575us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13,"raft.queue_time_us":26395,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":508730,"threads_started":2,"wal-append.queue_time_us":161322}
I20260709 12:56:47.169792 15246 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:47.173031 15246 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:47.144764 15268 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:47.185304 13659 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.870s (SENT) [suppressed 42 similar messages]
I20260709 12:56:47.108155 15270 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.170887 13612 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:47.169359 15156 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:47.144856 15154 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.204995 15258 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:47.180665 15076 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.206003 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:47.205983 15076 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.210060 15085 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:47.210594 15085 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:47.157473 15221 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.211591 15154 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:47.226608 15160 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.173878 15167 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:56:47.238016 13611 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:56:47.173784 15168 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:56:47.191579 15156 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:47.251739 15214 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.218581 15076 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.240561 15124 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.257187 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:47.255378 15270 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.237155 15078 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:47.240742 15058 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.240620 15156 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:47.164996 14237 rpcz_store.cc:276] 0709 12:56:45.911216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.924787 (+ 13571us) service_pool.cc:224] Handling call
0709 12:56:46.642314 (+717527us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:46.642573 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:46.840356 (+197783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:46.843259 (+  2903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:46.848277 (+  5018us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:46.850711 (+  2434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:47.119418 (+268707us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54005a05d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:47.121206 (+  1788us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:47.124366 (+  3160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456527360}
I20260709 12:56:47.272733 15257 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.264422 15078 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:47.265213 15124 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:56:47.273298 15257 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.276129 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:47.276873 13616 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:56:47.279534 15260 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:47.276131 15058 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.163937 15255 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.274935 15078 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:47.306948 15221 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.307341 15085 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.308595 15085 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:47.316879 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:47.317744 13610 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:47.325330 15126 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.333622 13616 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:47.335107 15076 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:47.288815 15257 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.336370 15076 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.341262 15211 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.338299 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:47.356901 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:47.357718 13685 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:47.359722 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:47.360461 13685 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:47.362587 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:47.367560 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.366551 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:47.335585 15124 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.349299 15150 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:47.389343 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.363773 15148 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.364055 15195 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:47.386202 15259 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.349299 15257 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:56:47.396932 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.363588 15142 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.366983 15161 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:47.386025 15193 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:47.407507 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:47.412719 15079 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.413950 15097 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.416457 15221 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.507273 13437 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49880 (request call id 326) took 2325 ms. Trace:
W20260709 12:56:47.481819 13539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.017s	sys 0.029s
W20260709 12:56:47.558590 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.000s	sys 0.047s
W20260709 12:56:47.701907 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.000s	sys 0.054s
I20260709 12:56:47.701812 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:47.702483 13692 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:47.600329 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.008s	sys 0.045s
W20260709 12:56:47.702960 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.010s	sys 0.046s
W20260709 12:56:47.600530 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.022s	sys 0.028s
W20260709 12:56:47.704015 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.023s	sys 0.031s
W20260709 12:56:47.704416 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 804496us
W20260709 12:56:47.505856 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.705277 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:47.410836 15150 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.706853 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.707782 15221 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.403832 15214 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.508715 13438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49896 (request call id 302) took 2933 ms. Trace:
W20260709 12:56:47.592535 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.018s	sys 0.041s
I20260709 12:56:47.712493 13684 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:56:47.701035 13539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.020s	sys 0.034s
W20260709 12:56:47.807014 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.022s	sys 0.045s
I20260709 12:56:47.456699 15195 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.807997 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:47.797148 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.812258 14246 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:56:47.712850 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.509389 13441 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:56:47.754308 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.813813 13441 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:47.390220 15124 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:47.813079 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:47.824630 14241 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:56:47.824841 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 124597us
I20260709 12:56:47.507658 15078 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.826227 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.827286 15078 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.824857 13688 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:47.422129 15157 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:47.797667 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.829615 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.822474 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.830703 14245 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:47.830830 14243 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:47.822652 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.823580 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.823051 13441 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 2, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:47.824420 15124 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.821419 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:47.814872 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:47.850996 14246 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:47.836187 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.510762 15095 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:47.540752 15154 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.482635 15216 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.841600 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.863443 14243 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:56:47.865378 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 57546.4us
I20260709 12:56:47.421809 15148 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.845337 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.848356 14237 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:47.852332 13441 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 3, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:56:47.869530 13618 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.002s	sys 0.015s
I20260709 12:56:47.478490 15220 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.708998 15161 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:47.854044 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.460299 15211 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.847827 15221 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:47.860268 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.015s	sys 0.034s
I20260709 12:56:47.824182 15260 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:47.454723 15256 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:47.877435 13618 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.002s	sys 0.015s
I20260709 12:56:47.797757 15257 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:47.881282 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.016s	sys 0.035s
I20260709 12:56:47.477602 15246 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.851997 15123 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.854959 15214 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.476164 15271 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:47.832013 15142 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.891496 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:47.860939 14238 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:47.877203 15217 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.877014 15078 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.901068 13692 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:47.894547 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:47.894886 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.928548 14243 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:47.700524 13437 rpcz_store.cc:276] 0709 12:56:45.156618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:45.156755 (+   137us) service_pool.cc:224] Handling call
0709 12:56:47.481889 (+2325134us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:56:47.029413 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:56:47.032215 (+  2802us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:56:47.032271 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:56:47.032296 (+    25us) tablet.cc:662] Decoding operations
0709 12:56:47.087721 (+ 55425us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:56:47.092946 (+  5225us) write_op.cc:641] Partition lock acquired for write op
0709 12:56:47.093780 (+   834us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:56:47.094197 (+   417us) tablet.cc:701] Row locks acquired
0709 12:56:47.094212 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:56:47.094394 (+   182us) write_op.cc:270] Start()
0709 12:56:47.094513 (+   119us) write_op.cc:276] Timestamp: P: 1783601807094344 usec, L: 0
0709 12:56:47.094532 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:56:47.118130 (+ 23598us) log.cc:844] Serialized 1694 byte log entry
0709 12:56:47.127778 (+  9648us) op_driver.cc:464] REPLICATION: finished
0709 12:56:47.128252 (+   474us) write_op.cc:301] APPLY: starting
0709 12:56:47.128341 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 12:56:47.128525 (+   184us) tablet.cc:1370] finished BulkCheckPresence
0709 12:56:47.128631 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:56:47.129257 (+   626us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:56:47.129452 (+   195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:56:47.129506 (+    54us) write_op.cc:312] APPLY: finished
0709 12:56:47.129752 (+   246us) log.cc:844] Serialized 41 byte log entry
0709 12:56:47.129925 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:47.130044 (+   119us) write_op.cc:454] Released schema lock
0709 12:56:47.130091 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 12:56:47.140825 (+ 10734us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:47.150773 (+  9948us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":150078,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":257,"apply.run_cpu_time_us":2180,"apply.run_wall_time_us":2179,"num_ops":4,"prepare.queue_time_us":259682,"prepare.run_cpu_time_us":5085,"prepare.run_wall_time_us":112040,"replication_time_us":33082,"thread_start_us":127691,"threads_started":1}]]}
I20260709 12:56:47.887959 15193 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:47.849556 15259 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.887827 15221 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:47.807060 13438 rpcz_store.cc:276] 0709 12:56:44.575786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:44.575999 (+   213us) service_pool.cc:224] Handling call
0709 12:56:47.508637 (+2932638us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:56:45.189068 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:56:45.193354 (+  4286us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:56:45.193403 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:56:45.193444 (+    41us) tablet.cc:662] Decoding operations
0709 12:56:45.194051 (+   607us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:56:45.194133 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:56:45.194160 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:56:45.194672 (+   512us) tablet.cc:701] Row locks acquired
0709 12:56:45.194721 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:56:45.194897 (+   176us) write_op.cc:270] Start()
0709 12:56:45.195031 (+   134us) write_op.cc:276] Timestamp: P: 1783601805194839 usec, L: 0
0709 12:56:45.195069 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:56:45.238434 (+ 43365us) log.cc:844] Serialized 1293 byte log entry
0709 12:56:46.571770 (+1333336us) op_driver.cc:464] REPLICATION: finished
0709 12:56:46.854422 (+282652us) write_op.cc:301] APPLY: starting
0709 12:56:46.854533 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:56:46.854700 (+   167us) tablet.cc:1370] finished BulkCheckPresence
0709 12:56:46.909269 (+ 54569us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:56:47.005846 (+ 96577us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:56:47.006158 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:56:47.006216 (+    58us) write_op.cc:312] APPLY: finished
0709 12:56:47.006558 (+   342us) log.cc:844] Serialized 35 byte log entry
0709 12:56:47.006756 (+   198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:47.006863 (+   107us) write_op.cc:454] Released schema lock
0709 12:56:47.006927 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:56:47.007423 (+   496us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:47.007455 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21716,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":265352,"apply.run_cpu_time_us":3454,"apply.run_wall_time_us":154502,"num_ops":3,"prepare.queue_time_us":101403,"prepare.run_cpu_time_us":10835,"prepare.run_wall_time_us":145384,"raft.queue_time_us":920938,"raft.run_cpu_time_us":10986,"raft.run_wall_time_us":272147,"replication_time_us":1303681,"thread_start_us":992177,"threads_started":4,"wal-append.queue_time_us":117571}]]}
I20260709 12:56:47.902295 15085 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.892264 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:47.868747 15154 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.889818 15076 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:47.875890 15095 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:56:47.899329 15216 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.896947 15246 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:47.910689 15256 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.930893 14245 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:47.933094 15161 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.940212 15150 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.893540 15069 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:47.896061 15123 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.896471 15214 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.898773 15142 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:47.884464 15220 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:47.929914 15156 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.001s
I20260709 12:56:47.965452 13693 raft_consensus.cc:2410] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:56:47.887689 15260 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:47.929191 15217 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:47.956315 15194 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.011s	sys 0.015s
W20260709 12:56:47.968302 15242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.016s	sys 0.040s
W20260709 12:56:47.986945 15242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.016s	sys 0.040s
I20260709 12:56:47.954571 15097 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:47.961066 15172 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.004s	sys 0.022s
I20260709 12:56:47.994342 15275 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:47.994999 15275 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:47.995414 15275 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:47.973726 13610 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.655s	user 0.005s	sys 0.010s
I20260709 12:56:47.896798 15271 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:47.955396 15169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.012s	sys 0.046s
I20260709 12:56:47.941704 15078 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.970585 15246 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:47.988104 15194 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.011s	sys 0.016s
I20260709 12:56:48.022266 15265 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.952682 15085 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:47.954285 15221 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:48.036000 15169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.012s	sys 0.046s
I20260709 12:56:47.965268 15216 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:48.034072 15191 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.050s	user 0.011s	sys 0.019s
I20260709 12:56:48.090224 15278 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:47.978056 15069 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.092759 15123 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.092995 15085 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.019186 13688 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:48.026216 13610 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:47.988824 15156 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:48.028936 15131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.972s	user 0.025s	sys 0.071s
W20260709 12:56:48.084064 15244 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.032s	sys 0.062s
I20260709 12:56:48.056833 15280 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:48.030616 15097 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.067409 15246 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:48.236155 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:48.337108 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:48.205017 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:48.101866 15211 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:56:48.364755 13613 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.022s	sys 0.027s
W20260709 12:56:48.366497 13613 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.022s	sys 0.027s
W20260709 12:56:48.347287 15131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.290s	user 0.030s	sys 0.071s
W20260709 12:56:48.026780 15172 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.004s	sys 0.022s
W20260709 12:56:48.145433 15191 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.011s	sys 0.019s
I20260709 12:56:48.133040 15195 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:48.306746 15279 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:48.146239 15248 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.069s	user 0.012s	sys 0.061s
W20260709 12:56:48.349515 15244 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.450s	user 0.037s	sys 0.068s
I20260709 12:56:48.182346 15124 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.367602 15123 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:56:48.188989 15264 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.021s	sys 0.029s
I20260709 12:56:48.348033 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2252) took 1050 ms. Trace:
I20260709 12:56:48.380548 15259 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.368146 15154 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.462656 15220 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.112782 15265 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.463711 15256 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.364264 15211 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:48.459278 15217 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.461082 15069 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.461566 15246 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.444185 15195 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:48.440414 15279 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.351382 15280 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.469419 15211 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:48.467818 15264 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.033s	sys 0.035s
W20260709 12:56:48.420665 15248 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.353s	user 0.019s	sys 0.065s
I20260709 12:56:48.468931 15216 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.481086 15271 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.490008 15265 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.453768 15260 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.447396 15274 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:48.494988 15255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.008s	sys 0.050s
I20260709 12:56:48.470927 15214 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.500133 15277 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:48.501158 15280 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.502671 15279 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:48.498389 15078 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.447229 15123 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.449484 13610 rpcz_store.cc:276] 0709 12:56:47.297420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.298634 (+  1214us) service_pool.cc:224] Handling call
0709 12:56:48.345532 (+1046898us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:48.521785 15274 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.207773 15278 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.531504 15274 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:48.496341 15262 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.171s	user 0.018s	sys 0.040s
W20260709 12:56:48.503495 15275 consensus_meta.cc:287] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 12:56:48.520471 15277 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.506094 15259 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:48.512176 15255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.224s	user 0.009s	sys 0.050s
I20260709 12:56:48.508994 15265 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.527726 15280 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.531613 15123 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.492409 15195 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:48.548533 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.027s	sys 0.157s
W20260709 12:56:48.596755 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.027s	sys 0.157s
I20260709 12:56:48.539404 15156 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:48.563956 13685 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.059s	sys 0.056s
W20260709 12:56:48.600106 13685 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.059s	sys 0.056s
W20260709 12:56:48.601334 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.019s	sys 0.030s
W20260709 12:56:48.601943 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.019s	sys 0.030s
W20260709 12:56:48.597545 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 892263us
W20260709 12:56:48.548979 13616 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.178s	user 0.023s	sys 0.032s
W20260709 12:56:48.603030 13616 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.023s	sys 0.032s
I20260709 12:56:48.603332 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:48.540025 15266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.131s	user 0.012s	sys 0.041s
W20260709 12:56:48.561446 15262 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.019s	sys 0.041s
I20260709 12:56:48.598371 15278 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:48.620044 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.010s	sys 0.030s
W20260709 12:56:48.620689 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.010s	sys 0.030s
W20260709 12:56:48.622128 13618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.017s	sys 0.020s
I20260709 12:56:48.622888 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:48.623778 13618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.018s	sys 0.021s
I20260709 12:56:48.625167 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2245) took 1523 ms. Trace:
W20260709 12:56:48.628424 13583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.006s	sys 0.036s
W20260709 12:56:48.628962 13583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.006s	sys 0.036s
W20260709 12:56:48.605099 15266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.013s	sys 0.042s
W20260709 12:56:48.646085 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:48.563129 15275 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.605792 15255 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:48.625662 13618 rpcz_store.cc:276] 0709 12:56:47.102183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.102905 (+   722us) service_pool.cc:224] Handling call
0709 12:56:47.103224 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:47.103439 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:56:47.885929 (+782490us) log.cc:844] Serialized 14 byte log entry
0709 12:56:48.624694 (+738765us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:48.624770 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:56:48.624828 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:48.624935 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:48.625088 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1215099,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":1509756,"threads_started":2}
W20260709 12:56:48.657392 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.022s	sys 0.011s
I20260709 12:56:48.594710 15259 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.597229 15265 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.599666 15280 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.675019 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:48.661593 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.022s	sys 0.011s
W20260709 12:56:48.633038 15189 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.467s	user 0.016s	sys 0.049s
W20260709 12:56:48.729702 13689 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.007s	sys 0.040s
I20260709 12:56:48.732514 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:48.733929 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:48.759950 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:48.653626 15256 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:48.761129 15279 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:48.760879 15256 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.761564 15279 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:48.761960 15279 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:48.763110 15256 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.763651 15256 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.765020 15277 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:48.765291 15256 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:48.748482 15289 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:56:48.737461 15255 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:48.766295 15195 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.767067 15255 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.768330 15276 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.768800 15276 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.750948 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2249) took 1496 ms. Trace:
I20260709 12:56:48.770665 13613 rpcz_store.cc:276] 0709 12:56:47.254736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.258210 (+  3474us) service_pool.cc:224] Handling call
0709 12:56:47.264051 (+  5841us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:56:47.264400 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:47.264427 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:56:47.265051 (+   624us) log.cc:844] Serialized 52 byte log entry
0709 12:56:48.368854 (+1103803us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:48.368892 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:48.368960 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:48.750713 (+381753us) raft_consensus.cc:1685] finished
0709 12:56:48.750735 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:48.750888 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2251,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5964,"thread_start_us":1085031,"threads_started":1,"wal-append.queue_time_us":1402061}
I20260709 12:56:48.658228 15262 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 12:56:48.773974 15256 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:48.774904 13689 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.007s	sys 0.040s
I20260709 12:56:48.781854 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.739180 15131 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.782636 15131 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.784816 15131 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.739010 15124 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.785832 15124 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.751587 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:48.788820 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.754117 15265 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:48.757758 15260 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:56:48.745301 15189 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.579s	user 0.026s	sys 0.049s
I20260709 12:56:48.759814 15259 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:48.764216 15123 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:48.739351 15142 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:48.745744 15154 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.746662 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:48.797477 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.799649 13611 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:48.738775 15271 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.727341 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:48.837829 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.838768 13611 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:48.793947 15131 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.789680 15124 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.801064 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:48.850422 13613 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:48.845359 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 100101us
I20260709 12:56:48.851493 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:48.864236 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.865146 13611 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:56:48.864302 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.021s	sys 0.025s
W20260709 12:56:48.866355 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.021s	sys 0.025s
W20260709 12:56:48.866670 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 1.01482e+06us
I20260709 12:56:48.797857 13610 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:48.846170 15194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.009s	sys 0.024s
I20260709 12:56:48.800428 15142 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:48.868147 14238 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:48.798993 15123 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.877985 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:48.879647 15194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.009s	sys 0.024s
I20260709 12:56:48.890489 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:48.800586 15154 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.842288 15271 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.896348 15195 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:48.896775 15195 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:48.897109 15195 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:48.897907 15154 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.898118 15271 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.900735 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:48.901695 15154 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:48.903975 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:48.904572 13611 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:48.907668 13612 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:56:48.908011 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.908737 13619 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:48.910535 13690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.003s	sys 0.009s
W20260709 12:56:48.910893 13690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.034s	user 0.003s	sys 0.009s
I20260709 12:56:48.910964 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:48.911527 13585 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.017s	sys 0.019s
W20260709 12:56:48.911883 13585 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.017s	sys 0.019s
I20260709 12:56:48.798113 15260 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.912310 13614 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:56:48.913517 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.914196 15260 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.880249 15124 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:56:48.885531 15288 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:56:48.908155 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:48.915186 15154 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:48.915326 13617 raft_consensus.cc:2376] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2. Re-sending same reply.
I20260709 12:56:48.915477 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.915728 15154 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:48.916226 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.916873 13617 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:48.914227 13614 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:48.917938 15154 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:48.918233 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:48.874409 13684 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.006s	user 0.014s	sys 0.029s
W20260709 12:56:48.923710 13684 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.014s	sys 0.029s
I20260709 12:56:48.868714 15131 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:48.940137 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.009s	sys 0.041s
W20260709 12:56:48.945353 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.010s	sys 0.041s
I20260709 12:56:48.888515 15276 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:48.877660 15259 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:56:48.942224 13617 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:48.899709 15271 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:56:48.947557 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 401544us
I20260709 12:56:48.950525 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2254) took 1599 ms. Trace:
I20260709 12:56:48.958647 13584 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:48.959558 13611 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:48.958297 13616 rpcz_store.cc:276] 0709 12:56:47.351116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.351251 (+   135us) service_pool.cc:224] Handling call
0709 12:56:47.369272 (+ 18021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:47.369456 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:56:48.603427 (+1233971us) log.cc:844] Serialized 14 byte log entry
0709 12:56:48.949484 (+346057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:48.949553 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:56:48.949602 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:48.949979 (+   377us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:48.950445 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1003221,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":91,"thread_start_us":1578183,"threads_started":2}
I20260709 12:56:48.960512 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.949767 15295 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:48.964845 15293 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:56:48.964766 15295 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:48.969131 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:48.969213 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.969882 13611 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:48.965015 15292 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:56:48.917074 13612 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:48.956036 14239 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.005s	sys 0.036s
I20260709 12:56:48.988591 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.990267 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.991035 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:48.986200 13613 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:56:48.994102 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.994781 13616 raft_consensus.cc:2376] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3. Re-sending same reply.
I20260709 12:56:48.947335 15275 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:49.008591 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:49.010890 13611 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:56:49.009166 13617 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:49.013368 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:48.966243 15276 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:49.016531 15276 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:49.019394 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:49.020718 15300 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:56:49.024605 15298 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 12:56:49.030719 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 63401.1us
I20260709 12:56:49.032186 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:49.034646 15271 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:49.035176 15271 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:49.024658 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:49.037390 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:49.038612 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2433) took 1676 ms. Trace:
I20260709 12:56:49.043414 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2440) took 1208 ms. Trace:
I20260709 12:56:49.044006 13689 rpcz_store.cc:276] 0709 12:56:47.834914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.838989 (+  4075us) service_pool.cc:224] Handling call
0709 12:56:47.858139 (+ 19150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:47.858401 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:47.858438 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:47.878887 (+ 20449us) log.cc:844] Serialized 26 byte log entry
0709 12:56:48.775465 (+896578us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:48.775512 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:48.778371 (+  2859us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:49.043145 (+264774us) raft_consensus.cc:1685] finished
0709 12:56:49.043172 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:49.043339 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":855752,"threads_started":1,"wal-append.queue_time_us":1053342}
I20260709 12:56:49.035686 15271 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:49.047614 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2437) took 1636 ms. Trace:
I20260709 12:56:49.048095 13690 rpcz_store.cc:276] 0709 12:56:47.411382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.416136 (+  4754us) service_pool.cc:224] Handling call
0709 12:56:47.416834 (+   698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:47.417286 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:47.417356 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:47.850284 (+432928us) log.cc:844] Serialized 26 byte log entry
0709 12:56:48.757930 (+907646us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02b6a80ac 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b
0709 12:56:48.911321 (+153391us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:48.911363 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:48.911437 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:49.047282 (+135845us) raft_consensus.cc:1685] finished
0709 12:56:49.047310 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:49.047549 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5443328,"thread_start_us":1042588,"threads_started":1,"wal-append.queue_time_us":1094509}
I20260709 12:56:49.036408 15189 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:49.051410 15189 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:49.052140 15148 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:49.039059 13693 rpcz_store.cc:276] 0709 12:56:47.362202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.366286 (+  4084us) service_pool.cc:224] Handling call
0709 12:56:49.038372 (+1672086us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54002eccd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:49.038524 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371761920}
I20260709 12:56:49.059254 13611 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:49.060217 15148 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:49.065627 15131 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:49.068818 15280 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:49.072899 13616 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:56:49.073216 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 162976us
I20260709 12:56:49.077797 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:48.993121 14239 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.005s	sys 0.036s
I20260709 12:56:49.088222 13611 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:56:49.100009 15259 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:49.100672 15259 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:49.071552 15279 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:49.102706 15259 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:49.105178 15259 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:49.109695 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:49.110843 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:49.113345 13693 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:56:49.115051 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:49.116572 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.118634 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.121783 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.122435 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.086400 15148 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:49.169778 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:49.255151 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 580187us
I20260709 12:56:49.200608 13617 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:49.254738 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 71942.9us
I20260709 12:56:49.125854 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.092687 15255 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:49.126955 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:49.140012 15191 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:49.265822 15191 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:49.160039 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:49.122074 15257 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:49.270471 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.272297 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.273175 13690 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:49.274639 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:56:49.275070 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:49.275442 13690 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:49.277230 15193 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.312s	user 0.006s	sys 0.025s
I20260709 12:56:49.261089 13687 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
W20260709 12:56:49.277946 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 263624us
I20260709 12:56:49.278345 14237 raft_consensus.cc:2393] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:49.285089 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:49.227869 15211 consensus_meta.cc:287] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.722s	user 0.007s	sys 0.004s
I20260709 12:56:49.307740 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:49.283737 15193 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.325s	user 0.006s	sys 0.026s
I20260709 12:56:49.278115 15279 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:49.297672 15322 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:49.318907 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.283831 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:49.322460 13687 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:49.325563 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.328502 15097 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:49.328718 13687 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
W20260709 12:56:49.330744 15324 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 12:56:49.331321 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.333228 15148 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:49.333839 15191 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:49.333988 15148 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:56:49.339411 15148 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:49.337229 15191 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:56:49.300061 15333 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:49.354377 15123 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:49.355293 15221 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:49.356916 15189 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:49.306674 15305 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:49.319460 15193 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:49.305655 15211 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:49.307179 15328 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:56:49.369580 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2439) took 1535 ms. Trace:
I20260709 12:56:49.370520 13684 rpcz_store.cc:276] 0709 12:56:47.834545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.835039 (+   494us) service_pool.cc:224] Handling call
0709 12:56:47.835376 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:47.835759 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:56:48.924756 (+1088997us) log.cc:844] Serialized 14 byte log entry
0709 12:56:49.369244 (+444488us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:49.369308 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:56:49.369345 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:49.369415 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:49.369522 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9487,"prepare.queue_time_us":902209,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"thread_start_us":1460289,"threads_started":2}
I20260709 12:56:49.373138 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.374496 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.340345 15148 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:49.365916 15305 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:49.383153 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.369680 15079 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:49.345855 15322 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:49.378381 15333 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:49.394747 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:49.402259 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 141748us
I20260709 12:56:49.382048 15305 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:49.408447 15333 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:49.414342 15191 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:49.428238 15193 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:49.431017 15322 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:49.431939 13691 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:49.438660 15295 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:49.439689 15331 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:49.442687 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.456874 13686 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:49.443660 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.403859 15329 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:49.402736 15195 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:49.444645 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.458544 13689 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:49.462832 13692 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:49.445602 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:49.448024 15328 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:49.414608 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:56:49.460824 15148 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:49.391057 15079 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:49.479048 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 167119us
I20260709 12:56:49.481148 13687 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:49.474131 13688 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:56:49.690086 15273 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 2.179s	user 0.012s	sys 0.032s
I20260709 12:56:49.696902 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 318) took 2741 ms. Trace:
I20260709 12:56:49.697371 14244 rpcz_store.cc:276] 0709 12:56:46.955726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:46.955876 (+   150us) service_pool.cc:224] Handling call
0709 12:56:46.956361 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:46.956726 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:46.956762 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:46.957273 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:56:47.184073 (+226800us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:47.184119 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:47.184203 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:49.554133 (+2369930us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:56:49.696318 (+142185us) raft_consensus.cc:1685] finished
0709 12:56:49.696346 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:49.696761 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86030,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":10918,"spinlock_wait_cycles":26931584,"thread_start_us":213774,"threads_started":1,"wal-append.queue_time_us":540600}
I20260709 12:56:49.700762 15305 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:49.465790 15329 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:49.664714 15340 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:56:49.772014 15195 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:49.736718 15340 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:49.773969 15340 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:49.751052 15211 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:49.749199 15271 consensus_meta.cc:287] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.703s	user 0.011s	sys 0.001s
I20260709 12:56:49.780567 15271 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:49.898478 15305 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:49.948342 13610 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.063s	user 0.015s	sys 0.000s
I20260709 12:56:50.361375 15329 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:50.666442 15271 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:50.421023 15191 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.006s	sys 0.003s
W20260709 12:56:50.668087 15191 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.006s	sys 0.005s
I20260709 12:56:50.655826 13610 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:56:50.927094 13610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2256) took 2190 ms (client timeout 2256 ms). Trace:
W20260709 12:56:50.458088 15259 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.020s	sys 0.043s
W20260709 12:56:50.927763 13610 rpcz_store.cc:269] 0709 12:56:48.740337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:48.743345 (+  3008us) service_pool.cc:224] Handling call
0709 12:56:50.926958 (+2183613us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:56:50.928364 15259 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.373s	user 0.021s	sys 0.043s
W20260709 12:56:51.136142 15333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.446s	user 0.013s	sys 0.010s
W20260709 12:56:51.136678 15333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.447s	user 0.013s	sys 0.010s
W20260709 12:56:51.303121 15193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.004s	sys 0.004s
W20260709 12:56:51.303771 15193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.004s	sys 0.004s
W20260709 12:56:51.862556 13507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.961s	user 0.003s	sys 0.030s
W20260709 12:56:51.863662 13507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.962s	user 0.003s	sys 0.030s
W20260709 12:56:51.876255 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.983s	user 0.017s	sys 0.000s
W20260709 12:56:51.877058 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.984s	user 0.017s	sys 0.000s
I20260709 12:56:52.013031 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.015228 13535 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:52.016113 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.016010 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.054550 13539 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:52.060461 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:52.061851 13542 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:52.190598 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:52.583824 13508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.682s	user 0.006s	sys 0.019s
W20260709 12:56:52.590070 13508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.688s	user 0.010s	sys 0.019s
W20260709 12:56:52.589143 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 263547us
W20260709 12:56:52.590899 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 4.69904e+06us
I20260709 12:56:52.592422 13508 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:52.604347 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.605504 13539 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
W20260709 12:56:52.606318 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.216s	user 0.029s	sys 0.065s
W20260709 12:56:52.606788 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.217s	user 0.029s	sys 0.065s
I20260709 12:56:52.606824 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:52.607254 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 3.21746e+06us
I20260709 12:56:52.608017 13542 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:52.609853 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.610865 13541 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:52.612403 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.613417 13538 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:56:52.614401 14209 connection.cc:256] RPC call timeout handler was delayed by -2.161978213999646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:56:52.614954 14209 connection.cc:256] RPC call timeout handler was delayed by -2.0989782139995441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 12:56:52.615298 14209 connection.cc:256] RPC call timeout handler was delayed by -1.8343782139991163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 12:56:52.615612 14209 connection.cc:256] RPC call timeout handler was delayed by -1.7812782139990304s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
I20260709 12:56:52.616886 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.617780 13535 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:56:52.616148 14209 connection.cc:256] RPC call timeout handler was delayed by -1.7517206549991897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 12:56:52.618978 14209 connection.cc:256] RPC call timeout handler was delayed by -1.4343057730002329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:56:52.619385 14209 connection.cc:256] RPC call timeout handler was delayed by -1.43250577300023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 12:56:52.619786 14209 connection.cc:256] RPC call timeout handler was delayed by -1.3299057730000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
W20260709 12:56:52.620208 14209 connection.cc:256] RPC call timeout handler was delayed by -1.3110057730000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 12:56:52.638926 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:52.640175 13540 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:52.641943 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 223) took 3722 ms. Trace:
I20260709 12:56:52.645294 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:52.647145 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:52.662899 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 71438.2us
I20260709 12:56:52.669556 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.670794 13535 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:56:52.678027 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.679284 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.679466 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.680562 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.642477 13614 rpcz_store.cc:276] 0709 12:56:48.918975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:48.919144 (+   169us) service_pool.cc:224] Handling call
0709 12:56:48.966403 (+ 47259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:48.992796 (+ 26393us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:56:49.072187 (+ 79391us) log.cc:844] Serialized 14 byte log entry
0709 12:56:49.405753 (+333566us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:52.640890 (+3235137us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:52.641093 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:52.641248 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:52.641593 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19226,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":15628,"raft.queue_time_us":17404,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":318159,"threads_started":1,"wal-append.queue_time_us":483187,"wal-append.run_cpu_time_us":1317,"wal-append.run_wall_time_us":1076522}
I20260709 12:56:52.689230 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.690140 13543 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:52.703492 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.704536 13543 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:52.705924 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.706517 15364 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:52.711237 15345 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:52.711876 15345 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:52.713730 15345 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:52.707316 15364 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:52.753665 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 161589us
I20260709 12:56:52.770994 13540 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
W20260709 12:56:52.777624 14209 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.535s (SENT) [suppressed 16 similar messages]
W20260709 12:56:52.783854 13441 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 4.830s	user 0.017s	sys 0.038s
I20260709 12:56:52.760443 15364 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:52.792261 15364 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:52.796890 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 150067us
I20260709 12:56:52.797508 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:52.798457 13614 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:56:52.798362 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:56:52.804123 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 140065us
W20260709 12:56:52.796921 13616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 228) took 3737 ms (client timeout 2122 ms). Trace:
W20260709 12:56:52.851632 13616 rpcz_store.cc:269] 0709 12:56:49.059520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:52.796461 (+3736941us) service_pool.cc:211] Skipping call since client already timed out
0709 12:56:52.796645 (+   184us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:56:52.866711 13441 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 4.913s	user 0.017s	sys 0.041s
W20260709 12:56:52.880482 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 102981us
I20260709 12:56:52.890820 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:52.892381 13618 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:52.892802 15360 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:52.892925 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:52.894199 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:52.893769 15360 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:52.899308 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:52.900780 13619 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:52.901918 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:52.903280 13614 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:52.904995 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:52.906368 13613 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:56:52.906939 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 106996us
I20260709 12:56:52.912616 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:52.911531 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:52.922842 13618 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:52.923898 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:52.890880 13565 debug-util.cc:398] Leaking SignalData structure 0x7b080010b980 after lost signal to thread 12412
I20260709 12:56:52.913424 15345 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:52.930261 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 132396us
I20260709 12:56:52.937175 15346 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:52.939774 15357 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:52.940541 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:52.940717 15376 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:52.941519 15373 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:52.940718 15357 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:52.942552 13508 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:52.945186 15371 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:52.956167 15376 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:52.956187 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:52.957453 13616 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:52.957486 15357 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:52.949604 15373 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:52.959208 15373 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:52.962195 15345 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:52.965694 13506 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:52.966724 15380 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:52.968384 15356 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:52.971393 15356 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:52.972685 15356 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:52.977715 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:52.967904 15380 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:52.981884 15380 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:52.984809 13509 connection.cc:256] RPC call timeout handler was delayed by -4.307133788000101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 12:56:52.985517 15374 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:52.948762 14209 leader_election.cc:336] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.569s (SENT)
I20260709 12:56:52.934026 15360 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:52.985960 13509 connection.cc:256] RPC call timeout handler was delayed by -3.233789634000459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 12:56:52.989830 15374 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:52.992252 13509 connection.cc:256] RPC call timeout handler was delayed by -3.1272689200004606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2256s for a response
W20260709 12:56:52.992560 13509 connection.cc:256] RPC call timeout handler was delayed by -2.6892594669998289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.272s for a response
W20260709 12:56:52.992731 13509 connection.cc:256] RPC call timeout handler was delayed by -2.6002717379997193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27740000000000004s for a response
W20260709 12:56:52.992897 13509 connection.cc:256] RPC call timeout handler was delayed by -2.542647441000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 12:56:52.993048 13509 connection.cc:256] RPC call timeout handler was delayed by -2.1986187570000766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.321s for a response
W20260709 12:56:52.993199 13509 connection.cc:256] RPC call timeout handler was delayed by -1.6760226340002191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32480000000000003s for a response
W20260709 12:56:52.994640 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 63311us
I20260709 12:56:52.982640 15380 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:52.954777 15371 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:52.993932 15377 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:52.986143 15346 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:52.991694 15364 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:52.998793 15377 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:53.018716 13691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1080) took 3888 ms (client timeout 2284 ms). Trace:
W20260709 12:56:53.018827 14209 leader_election.cc:336] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.571s (SENT)
I20260709 12:56:53.019114 13616 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:52.997766 15371 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:53.019393 13691 rpcz_store.cc:269] 0709 12:56:49.120801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.122200 (+  1399us) service_pool.cc:224] Handling call
0709 12:56:53.007741 (+3885541us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54002981d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:53.008899 (+  1158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710182784}
I20260709 12:56:53.013765 15380 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:53.002630 15346 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:53.027242 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:56:53.023304 14209 leader_election.cc:336] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.685s (SENT)
I20260709 12:56:53.032546 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:53.026113 15346 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:53.052861 15360 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:56:53.058293 13689 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 3.581s	user 0.013s	sys 0.036s
W20260709 12:56:53.058743 13689 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.582s	user 0.013s	sys 0.036s
I20260709 12:56:53.062541 15360 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:53.063789 15360 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.066123 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:53.069657 15374 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:53.072726 15360 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:53.081707 15313 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 3.675s	user 0.011s	sys 0.033s
W20260709 12:56:53.092998 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 57834.4us
I20260709 12:56:53.094993 15374 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:53.095455 15374 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:53.095988 15374 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:53.100330 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 326) took 5271 ms. Trace:
W20260709 12:56:53.129123 13657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.814s	user 0.025s	sys 0.029s
W20260709 12:56:53.129732 13657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.820s	user 0.025s	sys 0.030s
I20260709 12:56:53.066525 15373 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:53.133762 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:53.136893 15345 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:53.138799 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2434) took 5769 ms. Trace:
I20260709 12:56:53.139261 15380 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:53.137660 15345 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:53.143147 13684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.719s	user 0.005s	sys 0.048s
I20260709 12:56:53.101061 14239 rpcz_store.cc:276] 0709 12:56:47.828110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.828213 (+   103us) service_pool.cc:224] Handling call
0709 12:56:47.828508 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:48.082538 (+254030us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:56:49.126581 (+1044043us) log.cc:844] Serialized 14 byte log entry
0709 12:56:49.372666 (+246085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:49.374536 (+  1870us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:49.383351 (+  8815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:53.099394 (+3716043us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54007102d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:53.099477 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:53.099720 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10163,"prepare.queue_time_us":209810,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":1137,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3747,"spinlock_wait_cycles":483192960,"thread_start_us":1056259,"threads_started":2,"wal-append.queue_time_us":247828,"wal-append.run_cpu_time_us":1599,"wal-append.run_wall_time_us":1001621}
I20260709 12:56:53.135614 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:53.149777 13684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.725s	user 0.007s	sys 0.049s
I20260709 12:56:53.153190 15345 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:53.154001 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 177) took 3832 ms. Trace:
I20260709 12:56:53.139228 13685 rpcz_store.cc:276] 0709 12:56:47.369601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:47.369803 (+   202us) service_pool.cc:224] Handling call
0709 12:56:47.371306 (+  1503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:47.852516 (+481210us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:56:48.600751 (+748235us) log.cc:844] Serialized 14 byte log entry
0709 12:56:49.177395 (+576644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:49.177480 (+    85us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:49.177550 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:53.138282 (+3960732us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54002986d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:53.138343 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:53.138576 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15373,"prepare.queue_time_us":914318,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":148513,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":58,"spinlock_wait_cycles":1106173184,"thread_start_us":1293177,"threads_started":3,"wal-append.queue_time_us":630036,"wal-append.run_cpu_time_us":1017,"wal-append.run_wall_time_us":1001053}
W20260709 12:56:53.163748 13687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1081) took 4042 ms (client timeout 2812 ms). Trace:
W20260709 12:56:53.169412 13687 rpcz_store.cc:269] 0709 12:56:49.125470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.330455 (+204985us) service_pool.cc:224] Handling call
0709 12:56:53.163206 (+3832751us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002963d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:53.163484 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341383552}
I20260709 12:56:53.149834 15373 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:53.170531 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 171) took 4041 ms. Trace:
I20260709 12:56:53.171864 15373 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:53.174000 15373 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:53.171197 13684 rpcz_store.cc:276] 0709 12:56:49.131049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.396998 (+265949us) service_pool.cc:224] Handling call
0709 12:56:49.423043 (+ 26045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:49.423310 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:56:49.424038 (+   728us) log.cc:844] Serialized 14 byte log entry
0709 12:56:53.169977 (+3745939us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:53.170086 (+   109us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:53.170198 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:53.170289 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:53.170469 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":1286,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":134,"thread_start_us":3742285,"threads_started":1,"wal-append.queue_time_us":3287219}
I20260709 12:56:53.174767 15373 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:53.168908 13689 rpcz_store.cc:276] 0709 12:56:49.321123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.462935 (+141812us) service_pool.cc:224] Handling call
0709 12:56:49.464227 (+  1292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:49.464493 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:56:53.059266 (+3594773us) log.cc:844] Serialized 14 byte log entry
0709 12:56:53.153322 (+ 94056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:53.153407 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:56:53.153515 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:53.153647 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:53.153837 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20530,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":4244,"raft.queue_time_us":3564,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":176,"thread_start_us":3671937,"threads_started":2}
I20260709 12:56:53.193356 15385 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:53.194123 15385 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:53.194589 15385 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:53.196285 15345 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:53.197036 15345 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.197821 15371 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:53.199126 15345 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:53.200186 15374 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.198596 15377 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:53.202405 15374 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:53.203169 15374 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:53.203235 14238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 402) took 4147 ms (client timeout 2260 ms). Trace:
I20260709 12:56:53.203797 15374 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:56:53.210494 13659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.801s	user 0.032s	sys 0.052s
W20260709 12:56:53.210990 13659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.802s	user 0.033s	sys 0.052s
W20260709 12:56:53.211287 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 3.80181e+06us
W20260709 12:56:53.203727 14238 rpcz_store.cc:269] 0709 12:56:49.055452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.072972 (+ 17520us) service_pool.cc:224] Handling call
0709 12:56:53.202720 (+4129748us) spinlock_profiling.cc:243] Waited 2.74 s on lock 0x7b54007238d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:53.202982 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6310856576}
I20260709 12:56:53.220110 15371 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:53.220256 15345 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:53.219573 15346 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:53.223202 15374 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:53.221033 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:53.224727 15345 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:53.225390 15345 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:56:53.226513 13658 connection.cc:256] RPC call timeout handler was delayed by -3.8418342699997083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 12:56:53.226877 13658 connection.cc:256] RPC call timeout handler was delayed by -3.5862342700002046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 12:56:53.227195 13658 connection.cc:256] RPC call timeout handler was delayed by -3.5529342700001507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 12:56:53.227401 13658 connection.cc:256] RPC call timeout handler was delayed by -3.4701342700000168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
W20260709 12:56:53.227684 13658 connection.cc:256] RPC call timeout handler was delayed by -3.37923426999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260709 12:56:53.227964 13658 connection.cc:256] RPC call timeout handler was delayed by -3.3234342699997796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
I20260709 12:56:53.229028 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:53.228935 15345 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.233112 15346 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:53.226091 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:53.248479 15378 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:53.251825 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.243793 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:53.252027 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:53.322443 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:53.348047 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 105311us
I20260709 12:56:53.348076 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.350383 15328 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:53.350937 15328 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:56:53.467368 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 230467us
I20260709 12:56:53.686216 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.351256 13543 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:53.704185 13540 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:53.736141 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.803817 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.738905 15328 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:53.869354 13537 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:53.869200 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.359975 15380 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:53.870412 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.870342 15380 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.870858 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:53.873032 15380 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:53.871634 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.871634 13543 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:53.674366 13541 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:56:53.334303 15378 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:53.334407 15346 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:53.905467 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.364457 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.906840 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:53.909933 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:53.362576 15385 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.905764 15378 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:53.920522 13537 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:56:53.919569 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 570063us
I20260709 12:56:53.912999 13535 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:53.920784 15373 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.744s	user 0.011s	sys 0.000s
I20260709 12:56:53.927454 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2261) took 5136 ms. Trace:
I20260709 12:56:53.933125 13543 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:53.933894 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:53.934303 13615 rpcz_store.cc:276] 0709 12:56:48.791963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:48.792432 (+   469us) service_pool.cc:224] Handling call
0709 12:56:48.792742 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:48.944413 (+151671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:48.944440 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:48.944488 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:48.944516 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:53.926972 (+4982456us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540023c8d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:53.927036 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:53.927213 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348356992}
I20260709 12:56:53.938326 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 236) took 1050 ms. Trace:
I20260709 12:56:53.938711 13617 rpcz_store.cc:276] 0709 12:56:52.888432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:53.929431 (+1040999us) service_pool.cc:224] Handling call
0709 12:56:53.938244 (+  8813us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:53.941200 15385 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:53.946578 15378 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.934646 15373 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:53.949433 15378 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:53.950176 15373 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:53.950146 15378 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:56:53.946656 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:53.954640 15266 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:53.955703 15355 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.172s	user 0.002s	sys 0.014s
I20260709 12:56:53.955565 15266 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:53.952329 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 700379us
W20260709 12:56:53.956015 13659 leader_election.cc:336] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.870s (SENT)
I20260709 12:56:53.960376 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:53.966537 15266 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:53.971450 14237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 401) took 4959 ms (client timeout 1519 ms). Trace:
I20260709 12:56:53.950240 15277 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:53.960404 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1078) took 4798 ms. Trace:
I20260709 12:56:53.979363 15079 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:53.979350 15328 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:53.977483 14237 rpcz_store.cc:269] 0709 12:56:49.011669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.019078 (+  7409us) service_pool.cc:224] Handling call
0709 12:56:53.970907 (+4951829us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b54001c57d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:53.971236 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203753344}
I20260709 12:56:53.976370 13693 rpcz_store.cc:276] 0709 12:56:49.120153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.120306 (+   153us) service_pool.cc:224] Handling call
0709 12:56:49.120706 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:49.120993 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:49.121030 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:49.121515 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:56:49.664590 (+543075us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:49.664649 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:49.664742 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:53.908106 (+4243364us) raft_consensus.cc:1685] finished
0709 12:56:53.911220 (+  3114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:53.918173 (+  6953us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":408876,"threads_started":1,"wal-append.queue_time_us":564909,"wal-append.run_cpu_time_us":1968,"wal-append.run_wall_time_us":1019649}
I20260709 12:56:53.977286 15277 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:53.987376 15332 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:53.988644 15328 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:53.988698 15277 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:53.990855 15277 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:54.004398 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:54.008846 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 52884.2us
I20260709 12:56:54.010923 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:54.011772 13690 consensus_meta.cc:287] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 4.704s	user 0.028s	sys 0.049s
W20260709 12:56:54.013088 13659 connection.cc:256] RPC call timeout handler was delayed by -4.6135903460008194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 12:56:54.015129 13659 connection.cc:256] RPC call timeout handler was delayed by -4.2076903460001631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
W20260709 12:56:54.015424 13659 connection.cc:256] RPC call timeout handler was delayed by -3.4895697720003227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 12:56:54.015717 13659 connection.cc:256] RPC call timeout handler was delayed by -2.2772697720001815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2993s for a response
I20260709 12:56:54.016654 15328 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:54.012337 13690 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:54.022351 15266 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:54.022131 15380 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:54.023521 15266 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:54.024205 15380 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.018452 15332 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:54.027123 15354 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.126s	user 0.004s	sys 0.005s
I20260709 12:56:54.026988 15332 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.019599 13658 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:54.029362 15332 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:54.029691 15277 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:54.029379 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:54.026285 15380 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:54.022034 13584 connection.cc:256] RPC call timeout handler was delayed by -3.9952961189992493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 12:56:54.032063 15380 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:54.032598 15380 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:54.034018 13656 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.655s	user 0.031s	sys 0.043s
W20260709 12:56:54.031507 13584 connection.cc:256] RPC call timeout handler was delayed by -3.8521961189999274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
W20260709 12:56:54.035481 13656 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.657s	user 0.032s	sys 0.043s
W20260709 12:56:54.021849 13690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 173) took 4769 ms (client timeout 1569 ms). Trace:
W20260709 12:56:54.036403 13690 rpcz_store.cc:269] 0709 12:56:49.253027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:49.274241 (+ 21214us) service_pool.cc:224] Handling call
0709 12:56:54.021728 (+4747487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:54.035392 15380 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:54.037722 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:54.041369 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.042371 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:54.042745 13538 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:56:54.043450 13690 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.043249 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.044586 13693 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:54.044478 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.044771 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.045545 13535 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:54.045751 13537 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 1. Current term is 3. Ops: []
I20260709 12:56:54.045547 13688 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:54.047683 15076 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:54.048146 15076 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:54.048332 15380 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:54.048516 15076 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.048563 13535 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.048947 15380 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:54.051491 15380 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:54.052220 13584 connection.cc:256] RPC call timeout handler was delayed by -3.7594961189997775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
I20260709 12:56:54.052404 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.053326 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.054091 15142 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:54.054117 13688 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:54.056322 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.056346 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:54.052632 13584 connection.cc:256] RPC call timeout handler was delayed by -3.7468961189997572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 12:56:54.057188 13538 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
W20260709 12:56:54.057601 13584 connection.cc:256] RPC call timeout handler was delayed by -3.7414961189997484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 12:56:54.058168 13584 connection.cc:256] RPC call timeout handler was delayed by -3.6676961189996291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 12:56:54.056461 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.061908 13688 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:56:54.062151 15372 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.167s	user 0.003s	sys 0.006s
I20260709 12:56:54.062649 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:54.063434 13685 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:54.063560 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:56:54.061326 13584 connection.cc:256] RPC call timeout handler was delayed by -3.6676961189996291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 12:56:54.064359 13684 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:54.065056 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.065697 13685 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
W20260709 12:56:54.064630 13584 connection.cc:256] RPC call timeout handler was delayed by -3.6330370319992653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 12:56:54.059659 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.060796 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:54.071483 15348 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.359s	user 0.013s	sys 0.001s
I20260709 12:56:54.071849 13693 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:54.072278 15142 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:54.072587 15256 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:54.072660 13584 connection.cc:256] RPC call timeout handler was delayed by -3.6330370319992653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 12:56:54.073247 13584 connection.cc:256] RPC call timeout handler was delayed by -3.4885961189993395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20640000000000003s for a response
I20260709 12:56:54.072934 15142 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:56:54.073710 13584 connection.cc:256] RPC call timeout handler was delayed by -3.4098370319998139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
I20260709 12:56:54.073323 15256 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:54.074232 13584 connection.cc:256] RPC call timeout handler was delayed by -3.3949961189991882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 12:56:54.074625 13584 connection.cc:256] RPC call timeout handler was delayed by -3.3733961190000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260709 12:56:54.074962 13584 connection.cc:256] RPC call timeout handler was delayed by -3.2388370319995374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
W20260709 12:56:54.075263 13584 connection.cc:256] RPC call timeout handler was delayed by -3.1452370319993861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 12:56:54.066627 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.075336 15142 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
W20260709 12:56:54.075598 13584 connection.cc:256] RPC call timeout handler was delayed by -3.1452370319993861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 12:56:54.076001 13584 connection.cc:256] RPC call timeout handler was delayed by -2.9906654139995226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 12:56:54.068660 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:54.076305 13584 connection.cc:256] RPC call timeout handler was delayed by -2.966137032000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22599999999999998s for a response
W20260709 12:56:54.076620 13584 connection.cc:256] RPC call timeout handler was delayed by -2.966137032000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22599999999999998s for a response
I20260709 12:56:54.059413 15340 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:54.026285 15266 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:54.070243 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:54.076915 13584 connection.cc:256] RPC call timeout handler was delayed by -2.9445370319999711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2284s for a response
W20260709 12:56:54.078279 13584 connection.cc:256] RPC call timeout handler was delayed by -2.7258370319996175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25270000000000004s for a response
I20260709 12:56:54.078547 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.079746 13690 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:54.076824 13537 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:56:54.084869 13584 connection.cc:256] RPC call timeout handler was delayed by -2.4693370319992027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2812s for a response
I20260709 12:56:54.075470 15256 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:54.018203 15346 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:54.086364 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:54.086364 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.090323 13684 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:56:54.094260 13685 raft_consensus.cc:2410] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:56:54.089648 14238 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
W20260709 12:56:54.018136 15345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.003s	sys 0.013s
W20260709 12:56:54.098178 15382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.047s	user 0.002s	sys 0.013s
W20260709 12:56:54.099426 15345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.003s	sys 0.014s
W20260709 12:56:54.101889 15382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.051s	user 0.002s	sys 0.014s
I20260709 12:56:54.078043 15142 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:54.101855 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.102577 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:54.102890 13617 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:54.103286 14239 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:54.104036 15142 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.099372 15332 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:56:54.106562 15345 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:54.106962 15345 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:54.107872 15142 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:54.107882 15345 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.107905 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.109270 13615 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:54.107255 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:54.108783 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:54.112509 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.113066 13615 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:54.114079 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.114194 15076 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:54.114895 13685 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:54.123131 15332 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:54.123242 13615 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:56:54.123752 15332 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:54.124517 15277 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.124779 15332 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.130174 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:54.131131 15346 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:54.132733 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:54.133795 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:54.134171 15346 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:56:54.135113 15346 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:54.140872 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.136183 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:54.143900 13687 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:54.143293 15328 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:54.136267 15346 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.146764 15328 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:54.147009 14237 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:56:54.147310 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.148049 13543 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
W20260709 12:56:54.148222 15361 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.188s	user 0.004s	sys 0.010s
I20260709 12:56:54.149986 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 287) took 2136 ms. Trace:
I20260709 12:56:54.151008 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.151871 13685 raft_consensus.cc:2368] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
W20260709 12:56:54.152014 15353 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.227s	user 0.006s	sys 0.000s
I20260709 12:56:54.153218 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.150360 13536 rpcz_store.cc:276] 0709 12:56:52.013132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:52.013551 (+   419us) service_pool.cc:224] Handling call
0709 12:56:52.014229 (+   678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:52.568014 (+553785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:52.568054 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:52.576076 (+  8022us) log.cc:844] Serialized 26 byte log entry
0709 12:56:52.714026 (+137950us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:56:52.769037 (+ 55011us) log.cc:844] Serialized 14 byte log entry
0709 12:56:52.874100 (+105063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:52.874243 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:53.216912 (+342669us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b4800002dd8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:56:54.147494 (+930582us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:56:54.149618 (+  2124us) raft_consensus.cc:1685] finished
0709 12:56:54.149643 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:54.149766 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4032,"prepare.queue_time_us":39600,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":897346,"raft.run_cpu_time_us":3007,"raft.run_wall_time_us":406903,"spinlock_wait_cycles":1730304256,"thread_start_us":626173,"threads_started":4,"wal-append.queue_time_us":383167}
I20260709 12:56:54.154032 13685 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:54.154265 15076 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:54.146323 14238 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.158583 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:54.145713 13615 raft_consensus.cc:2376] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3. Re-sending same reply.
I20260709 12:56:54.147214 15328 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.165366 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.166421 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:54.160480 15376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.175s	user 0.002s	sys 0.019s
I20260709 12:56:54.166476 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.168141 13685 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:54.173032 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.174258 13689 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:56:54.175096 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 106711us
I20260709 12:56:54.171554 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:54.177793 14237 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.170780 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.180781 13690 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:54.181696 14239 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:54.179787 15277 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:54.259021 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.202346 13617 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:54.269690 14237 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:54.269891 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:54.272853 14242 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:54.273056 13582 connection.cc:256] RPC call timeout handler was delayed by -3.8972288090008078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 12:56:54.273276 15154 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:54.273679 13582 connection.cc:256] RPC call timeout handler was delayed by -3.7865288090006288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 12:56:54.274156 13582 connection.cc:256] RPC call timeout handler was delayed by -3.5554373560007662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
I20260709 12:56:54.273869 15154 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:54.274554 13582 connection.cc:256] RPC call timeout handler was delayed by -3.4400288090000686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 12:56:54.274967 13582 connection.cc:256] RPC call timeout handler was delayed by -3.4283332660006636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
W20260709 12:56:54.275312 13582 connection.cc:256] RPC call timeout handler was delayed by -3.4184288090009431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
I20260709 12:56:54.275753 14238 raft_consensus.cc:2410] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:56:54.276223 15154 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:54.277101 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:54.168236 15345 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:54.167518 15376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.002s	sys 0.019s
W20260709 12:56:54.275683 13582 connection.cc:256] RPC call timeout handler was delayed by -3.3809883330004595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
W20260709 12:56:54.297638 13582 connection.cc:256] RPC call timeout handler was delayed by -3.1847883330010518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:56:54.298223 13582 connection.cc:256] RPC call timeout handler was delayed by -3.1340332660010972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2284s for a response
W20260709 12:56:54.298765 13582 connection.cc:256] RPC call timeout handler was delayed by -2.9153332660007436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25270000000000004s for a response
W20260709 12:56:54.299228 13582 connection.cc:256] RPC call timeout handler was delayed by -2.8184883330004595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 12:56:54.299933 13582 connection.cc:256] RPC call timeout handler was delayed by -2.7257883330012191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
W20260709 12:56:54.300406 13582 connection.cc:256] RPC call timeout handler was delayed by -2.6588332660003289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2812s for a response
W20260709 12:56:54.300865 13582 connection.cc:256] RPC call timeout handler was delayed by -2.5034883330008597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2493s for a response
W20260709 12:56:54.304591 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 171960us
I20260709 12:56:54.306883 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.309911 14243 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:56:54.310581 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:54.316619 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:54.340647 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:54.317345 13685 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:56:54.344934 15349 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 1.590s	user 0.009s	sys 0.001s
I20260709 12:56:54.346233 15276 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:54.258839 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:54.357683 15157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 5.030s	user 0.032s	sys 0.116s
I20260709 12:56:54.361979 15376 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:56:54.362689 15368 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.580s	user 0.008s	sys 0.006s
I20260709 12:56:54.362604 15376 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:54.365026 15376 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:54.365259 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:54.365688 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56556us
W20260709 12:56:54.366264 14241 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 395) took 5712 ms (client timeout 1865 ms). Trace:
I20260709 12:56:54.367604 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:54.368173 15157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 5.113s	user 0.033s	sys 0.117s
I20260709 12:56:54.368486 14243 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:54.369202 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:54.369621 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:54.370218 14243 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:54.366647 14241 rpcz_store.cc:269] 0709 12:56:48.654094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:48.656812 (+  2718us) service_pool.cc:224] Handling call
0709 12:56:54.364944 (+5708132us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:56:54.366193 (+  1249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39336192}
I20260709 12:56:54.373551 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:54.367547 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:54.367614 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:54.310577 15383 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.129s	user 0.003s	sys 0.010s
I20260709 12:56:54.360884 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.378229 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.378857 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.382443 13617 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.375732 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.385519 13691 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:54.375185 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:54.392976 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:54.394377 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08000e80c0 after lost signal to thread 13475
I20260709 12:56:54.394594 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:54.394910 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08001baba0 after lost signal to thread 13489
I20260709 12:56:54.394618 13688 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:56:54.396308 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08002d9140 after lost signal to thread 13568
I20260709 12:56:54.386977 15276 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 12:56:54.397540 13565 debug-util.cc:398] Leaking SignalData structure 0x7b080039a1a0 after lost signal to thread 13643
I20260709 12:56:54.398440 13618 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:54.392624 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.399713 13611 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:54.400089 13616 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:56:54.400336 14239 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:54.400636 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.401113 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.401916 13692 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:54.404469 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.405145 13618 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:54.405230 14246 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:56:54.407037 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.407610 13613 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:54.408453 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.409082 13613 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.410369 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.412169 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.404579 15076 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:54.412747 13616 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:54.416092 13614 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:56:54.416428 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.416888 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.417138 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.417461 13616 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:54.417650 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.417910 13688 raft_consensus.cc:2368] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4.
I20260709 12:56:54.418193 13611 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:54.418380 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.418911 13616 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:54.412189 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.420188 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.416296 15076 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.420188 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:54.419704 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.423543 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.424268 13693 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:54.424150 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.424890 13616 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:54.424212 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.426285 13618 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.422508 13692 raft_consensus.cc:2376] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4. Re-sending same reply.
I20260709 12:56:54.436151 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:54.436843 13536 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.439682 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.450587 15076 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:54.453850 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:54.454556 14237 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.458019 13584 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:54.446609 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.463779 13611 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:56:54.469123 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:56:54.472220 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:54.472801 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.476281 13686 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:54.477047 13691 raft_consensus.cc:2410] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:56:54.479964 13684 raft_consensus.cc:2410] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:56:54.480222 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.480904 13541 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:54.481938 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.482748 13535 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:54.483511 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.484000 13536 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:54.484309 13535 raft_consensus.cc:2393] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:54.484517 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.460726 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.486475 13539 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:54.473340 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:56:54.487607 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:54.488320 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.488517 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.489543 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.496352 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.496634 13535 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:56:54.497634 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.497774 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:54.498433 15351 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.789s	user 0.010s	sys 0.008s
I20260709 12:56:54.498829 13543 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:54.496666 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.500296 13619 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.498379 13535 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:54.493981 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.506996 14246 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:56:54.499342 13565 debug-util.cc:398] Leaking SignalData structure 0x7b0800099f20 after lost signal to thread 13717
I20260709 12:56:54.511996 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.511224 13613 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
W20260709 12:56:54.513005 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 45197.7us
I20260709 12:56:54.513748 13538 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:54.514293 13584 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.513607 15154 raft_consensus.cc:2764] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:56:54.518548 14237 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:54.519003 15123 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:54.519418 15123 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:54.520398 15123 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.520937 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.521751 15123 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:54.522362 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.523102 13610 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.527087 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.527884 13612 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:54.530014 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.529047 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.533118 13615 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:54.536360 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:54.537062 13612 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:56:54.535244 13534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.901s	user 0.019s	sys 0.015s
I20260709 12:56:54.537830 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.557209 13537 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:54.541042 13619 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:54.544526 13617 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:54.504698 13614 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:56:54.558393 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 49500.8us
I20260709 12:56:54.546079 14243 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:54.546628 14246 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:54.561764 13618 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:54.552456 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.567271 13619 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:56:54.567612 13613 raft_consensus.cc:2376] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4. Re-sending same reply.
I20260709 12:56:54.567312 13617 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:54.569624 13538 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
W20260709 12:56:54.561427 13534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.953s	user 0.020s	sys 0.016s
W20260709 12:56:54.569180 15379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.553s	user 0.005s	sys 0.030s
I20260709 12:56:54.568768 15310 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:54.577751 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.578382 13613 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:54.566498 15276 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:54.581303 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.580300 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.583755 13538 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:56:54.580678 15356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.600s	user 0.027s	sys 0.026s
W20260709 12:56:54.579713 15379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.563s	user 0.005s	sys 0.030s
I20260709 12:56:54.580088 15310 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:54.581071 15142 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:56:54.600342 15374 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:54.582810 15380 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:54.602517 13538 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:56:54.592525 15356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.615s	user 0.027s	sys 0.027s
I20260709 12:56:54.608775 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 291) took 2593 ms. Trace:
W20260709 12:56:54.611227 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08003bf2e0 after lost signal to thread 14212
I20260709 12:56:54.613191 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.598507 15310 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.615069 13509 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.604866 15374 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:54.617920 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.606519 15380 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:54.624325 13537 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:54.620733 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.629473 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:54.629863 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.614439 13534 rpcz_store.cc:276] 0709 12:56:52.015230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:52.602204 (+586974us) service_pool.cc:224] Handling call
0709 12:56:52.602875 (+   671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:53.206137 (+603262us) spinlock_profiling.cc:243] Waited 9.74 ms on lock 0x7b4800002dd8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:56:54.448853 (+1242716us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02b6a80ac 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060
0709 12:56:54.581195 (+132342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:54.582917 (+  1722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:54.601505 (+ 18588us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:54.606333 (+  4828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:54.608506 (+  2173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:54.608693 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":257547,"raft.run_cpu_time_us":3232,"raft.run_wall_time_us":329567,"spinlock_wait_cycles":2455031552,"thread_start_us":1969969,"threads_started":1}
I20260709 12:56:54.631888 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:54.632547 13541 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.611902 15385 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:54.636019 15364 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:54.636524 15364 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:54.636260 15385 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:54.636902 15364 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:54.624333 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.635752 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.641220 13684 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:56:54.652545 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.653841 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.628392 13686 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:56:54.622260 13565 debug-util.cc:398] Leaking SignalData structure 0x7b080022a9c0 after lost signal to thread 14270
I20260709 12:56:54.656872 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.674896 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:54.678030 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:54.678824 13685 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.627355 15374 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:54.684687 13691 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:54.673468 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:56:54.682988 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54112.6us
I20260709 12:56:54.686143 13687 raft_consensus.cc:2410] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:56:54.640144 15380 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:56:54.677002 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad820 after lost signal to thread 14271
W20260709 12:56:54.683530 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 54151.8us
W20260709 12:56:54.689352 14212 connection.cc:256] RPC call timeout handler was delayed by -4.8502891459993407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
I20260709 12:56:54.712990 13693 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
W20260709 12:56:54.713160 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 89213.5us
W20260709 12:56:54.711097 14212 connection.cc:256] RPC call timeout handler was delayed by -4.834989145999316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 12:56:54.716719 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:54.714970 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.628986 15328 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:54.679886 15415 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:56:54.720409 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 101520us
I20260709 12:56:54.681221 15345 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:54.722585 13684 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:54.715509 15385 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:54.726926 15345 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:54.739740 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49010us
I20260709 12:56:54.741118 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:54.734875 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:54.725090 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab680 after lost signal to thread 15097
I20260709 12:56:54.742254 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:54.744787 14211 connection.cc:256] RPC call timeout handler was delayed by -4.5063454999999522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24620000000000003s for a response
W20260709 12:56:54.746935 14211 connection.cc:256] RPC call timeout handler was delayed by -3.8648790980005288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 12:56:54.747417 14211 connection.cc:256] RPC call timeout handler was delayed by -3.7994003599997086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 12:56:54.747819 14211 connection.cc:256] RPC call timeout handler was delayed by -3.7352790980003192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 12:56:54.748229 14211 connection.cc:256] RPC call timeout handler was delayed by -3.5003790979999394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 12:56:54.748605 14211 connection.cc:256] RPC call timeout handler was delayed by -3.4805790979999074s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:56:54.749039 14211 connection.cc:256] RPC call timeout handler was delayed by -3.4576717969994206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 12:56:54.749423 14211 connection.cc:256] RPC call timeout handler was delayed by -3.429771797000285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 12:56:54.749809 14211 connection.cc:256] RPC call timeout handler was delayed by -3.4148790979998012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 12:56:54.750205 14211 connection.cc:256] RPC call timeout handler was delayed by -2.8973790979998739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2706s for a response
I20260709 12:56:54.724327 15413 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:54.740888 15345 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:54.763070 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:54.764618 15331 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:56:54.752732 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab780 after lost signal to thread 15124
I20260709 12:56:54.770409 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:54.727510 15379 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:56:54.778263 15193 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:54.779189 15328 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:54.776543 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 51248.5us
W20260709 12:56:54.781325 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:54.778923 15193 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:54.757218 15413 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:54.782853 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:54.775111 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab660 after lost signal to thread 15148
I20260709 12:56:54.720258 15421 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:56:54.779708 15312 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:54.782553 15376 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:56:54.789812 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:54.790205 13565 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab6e0 after lost signal to thread 15156
I20260709 12:56:54.900115 15322 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:54.884999 15415 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:54.885943 13691 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:54.789011 15371 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:54.787703 15413 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:54.793917 15310 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:54.891433 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:54.892498 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 1.86917e+06us
I20260709 12:56:54.893064 15157 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:56:54.893244 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 106744us
I20260709 12:56:54.887112 15428 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:54.784212 15328 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:54.891597 13541 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
W20260709 12:56:54.909149 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 123354us
W20260709 12:56:54.928736 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:54.917464 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:54.908623 15371 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:54.939291 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:54.941673 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 185025us
I20260709 12:56:54.936152 15371 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:54.948558 14209 leader_election.cc:336] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.706s (SENT)
I20260709 12:56:54.918210 13610 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:56:54.958127 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 217080us
I20260709 12:56:54.960038 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:54.901983 15346 consensus_queue.cc:1059] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:56:54.965121 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:54.923560 15376 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:54.958488 15384 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:54.928399 15277 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:56:54.961642 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:54.967221 15359 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:54.958361 15424 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:54.968827 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:54.946082 15373 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:54.902982 15312 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:54.994827 14212 connection.cc:256] RPC call timeout handler was delayed by -4.5442891459997554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24620000000000003s for a response
W20260709 12:56:55.004878 14212 connection.cc:256] RPC call timeout handler was delayed by -4.31703823800035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
I20260709 12:56:54.953478 15331 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:56:55.005273 14212 connection.cc:256] RPC call timeout handler was delayed by -4.2873382380003022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
I20260709 12:56:54.964996 15340 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:55.006809 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:55.007004 14212 connection.cc:256] RPC call timeout handler was delayed by -4.2070926010001131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:56:55.009224 14212 connection.cc:256] RPC call timeout handler was delayed by -4.0894613430000391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 12:56:55.009538 14212 connection.cc:256] RPC call timeout handler was delayed by -4.0197265839997272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:56:55.009841 14212 connection.cc:256] RPC call timeout handler was delayed by -3.9993382379998366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
I20260709 12:56:55.009735 13582 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:55.010231 14212 connection.cc:256] RPC call timeout handler was delayed by -3.9669382379997842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 12:56:55.010540 14212 connection.cc:256] RPC call timeout handler was delayed by -2.935756672000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2706s for a response
I20260709 12:56:55.012318 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:54.970947 15328 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:54.994433 15359 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:55.021273 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:55.023672 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:55.024073 15328 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:55.025038 15328 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.025619 15417 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.026021 15417 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.026377 15417 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.027987 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:55.028523 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:55.028766 14245 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:55.029616 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:54.949584 15157 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.030571 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:55.031539 15434 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.031972 15434 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:55.032409 15434 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.034409 15157 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.035074 15157 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.035398 15157 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:55.035712 15157 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.036516 13685 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:55.005352 15424 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:55.040906 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 122715us
I20260709 12:56:55.042002 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:55.011427 15421 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.055591 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:55.020215 15382 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:55.020598 15359 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.019073 15346 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:55.022593 15322 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:55.079166 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:55.094854 15322 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.013648 15373 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.081640 15432 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:55.083151 15433 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:55.096450 15432 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:55.096547 15433 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:55.096984 15432 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.097088 15433 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.097277 15322 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.059405 15340 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.052075 15385 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:55.050437 15424 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.095950 15409 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:55.097786 15076 raft_consensus.cc:3055] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:55.048852 15421 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.085889 15371 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.100332 15340 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.007902 15384 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.111125 15076 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:55.119196 15312 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.119146 15076 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.119903 15384 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:55.120543 15076 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.121158 15310 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:55.121292 15367 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 2.084s	user 0.013s	sys 0.006s
I20260709 12:56:55.123147 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 298) took 2507 ms. Trace:
I20260709 12:56:55.126353 15413 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:55.136495 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 174968us
I20260709 12:56:55.137542 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:55.116607 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:55.139078 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 58929.1us
W20260709 12:56:55.135948 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 105471us
I20260709 12:56:55.147516 14211 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:55.143417 15310 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:56:55.072681 15415 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.113442 15385 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:55.156747 15385 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.159672 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:55.133226 15340 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.080888 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:56:55.112393 15421 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.168761 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:55.169430 13690 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.132797 15193 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:55.172475 15364 consensus_meta.cc:287] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.006s
I20260709 12:56:55.172996 15364 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.173312 15418 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:56:55.181252 15276 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:55.182193 15154 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:55.183043 15364 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.185359 14211 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:55.186293 14211 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:55.187045 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.187985 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:55.188221 15418 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:55.188416 14211 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:55.188599 15373 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:55.188710 15418 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:55.189101 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.189266 15373 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:55.189672 15418 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.190380 15379 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:55.190174 15157 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.190816 15379 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:55.191380 15379 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:55.190210 15387 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:55.192597 15387 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:55.193116 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:55.193706 14242 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.193547 15387 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.196462 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.190533 15322 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:55.123507 13542 rpcz_store.cc:276] 0709 12:56:52.615494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:52.615648 (+   154us) service_pool.cc:224] Handling call
0709 12:56:52.616168 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:52.654667 (+ 38499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:52.654709 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:52.679982 (+ 25273us) log.cc:844] Serialized 26 byte log entry
0709 12:56:52.733832 (+ 53850us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:56:52.770767 (+ 36935us) log.cc:844] Serialized 14 byte log entry
0709 12:56:52.802878 (+ 32111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:52.851735 (+ 48857us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:52.943409 (+ 91674us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b4800002dd8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:56:54.772379 (+1828970us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4800002dd8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:56:54.941285 (+168906us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:56:55.122928 (+181643us) raft_consensus.cc:1685] finished
0709 12:56:55.122949 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:55.123077 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33263,"prepare.queue_time_us":20062,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":413899,"raft.run_cpu_time_us":28898,"raft.run_wall_time_us":1915711,"spinlock_wait_cycles":1848243072,"thread_start_us":122284,"threads_started":3,"wal-append.queue_time_us":243292}
W20260709 12:56:55.151788 15123 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.004s
I20260709 12:56:55.197433 15356 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.190876 15418 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.247728 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:55.249742 13615 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:55.228533 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:55.238715 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 98839us
I20260709 12:56:55.251065 13613 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:55.251181 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:55.230811 13616 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:56:55.208308 15322 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:55.205997 15076 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.217247 15356 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:55.201833 15373 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:55.237936 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:55.291492 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:55.291147 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 200757us
I20260709 12:56:55.293668 13611 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:55.256214 13441 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41440 (request call id 206) took 8433 ms. Trace:
I20260709 12:56:55.256888 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:55.239787 15413 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.140576 15376 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.214171 15193 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.256985 15322 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.255928 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.283836 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:55.247735 15346 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:55.256395 15157 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:55.314016 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:55.241791 15123 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:55.319469 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:55.326220 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 70392.1us
I20260709 12:56:55.292831 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:55.294966 15356 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.260684 15076 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:56:55.332770 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 75984.3us
I20260709 12:56:55.339062 13615 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:55.370852 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.338413 13438 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:55.325661 15277 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:56:55.341800 15380 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.001s
I20260709 12:56:55.342329 15434 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.343994 15371 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:55.345376 15345 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.002s
I20260709 12:56:55.347090 13610 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:55.348045 15433 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.348819 13614 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:55.340492 15280 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:55.296074 15435 raft_consensus.cc:2764] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:56:55.376283 13611 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:56:55.372000 13659 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.373180 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.380581 13438 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:55.335000 15428 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:55.415946 13616 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:55.349085 15432 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.351655 14239 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:56:55.476065 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 104194us
I20260709 12:56:55.335232 15312 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:56:55.447733 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:55.479090 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:55.373593 15076 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.479307 15424 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.481470 15424 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.467794 15331 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:55.484818 15331 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:55.485762 15331 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.470985 14245 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:55.488740 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.487708 15331 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.491410 15382 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:55.494622 15359 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.497593 15433 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.498353 15443 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.498775 15443 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.499111 15443 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.500113 15437 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:55.500566 15437 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.500910 15437 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.504228 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.504060 15346 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.505903 15432 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:55.412072 15380 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.509105 15380 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:55.474723 15428 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:56:55.513245 15374 consensus_meta.cc:287] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.821s	user 0.008s	sys 0.002s
I20260709 12:56:55.513664 15428 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:55.513908 15374 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.514422 15435 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:55.514863 15435 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.515223 15435 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:56:55.516076 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 201025us
I20260709 12:56:55.443472 15123 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:55.518617 15123 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.520033 13617 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:55.520282 15193 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:55.520969 15193 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.521171 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2285) took 1062 ms. Trace:
I20260709 12:56:55.521682 13617 rpcz_store.cc:276] 0709 12:56:54.459312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:55.077829 (+618517us) service_pool.cc:224] Handling call
0709 12:56:55.085899 (+  8070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:55.520436 (+434537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:55.520939 (+   503us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54002dffd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:55.521111 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997890304}
I20260709 12:56:55.379269 15328 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:55.522912 15328 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:55.523382 15328 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:55.524452 15312 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:55.524993 15312 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:55.521317 15076 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.472913 13438 catalog_manager.cc:5697] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 2, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 2 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:56:55.507289 15379 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:55.544561 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 329714us
I20260709 12:56:55.546763 13438 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:55.381740 15277 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.551318 13438 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:56:55.554183 13438 catalog_manager.cc:5697] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:56:55.581743 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:56:55.584100 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 404107us
W20260709 12:56:55.583190 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 287530us
I20260709 12:56:55.587108 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:55.588634 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.589591 15340 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.590078 15442 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:55.590463 15442 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:55.590795 15442 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:55.591770 15340 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:55.592796 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:55.556711 15142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.011s	sys 0.043s
I20260709 12:56:55.582427 13438 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:56:55.556563 13534 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:55.606212 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:55.557386 15277 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:55.544286 14242 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:55.448658 15345 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:55.605576 15142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.012s	sys 0.043s
I20260709 12:56:55.610517 15076 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:55.607491 13438 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 3, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 3 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:56:55.615151 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.009s	sys 0.019s
W20260709 12:56:55.650316 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.009s	sys 0.019s
W20260709 12:56:55.650676 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 540082us
I20260709 12:56:55.468456 15373 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.595430 15385 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:55.295183 13441 rpcz_store.cc:276] 0709 12:56:46.823249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:46.823401 (+   152us) service_pool.cc:224] Handling call
0709 12:56:55.247264 (+8423863us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:56:49.357443 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:56:49.436237 (+ 78794us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:56:49.436273 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:56:49.436291 (+    18us) tablet.cc:662] Decoding operations
0709 12:56:49.436925 (+   634us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:56:49.436986 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:56:49.437013 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:56:49.437402 (+   389us) tablet.cc:701] Row locks acquired
0709 12:56:49.437418 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:56:49.437561 (+   143us) write_op.cc:270] Start()
0709 12:56:49.437704 (+   143us) write_op.cc:276] Timestamp: P: 1783601809437514 usec, L: 0
0709 12:56:49.437726 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:56:49.438390 (+   664us) log.cc:844] Serialized 1674 byte log entry
0709 12:56:54.343554 (+4905164us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:56:54.453172 (+109618us) op_driver.cc:464] REPLICATION: finished
0709 12:56:54.584539 (+131367us) write_op.cc:301] APPLY: starting
0709 12:56:54.594290 (+  9751us) tablet.cc:1367] starting BulkCheckPresence
0709 12:56:54.600274 (+  5984us) tablet.cc:1370] finished BulkCheckPresence
0709 12:56:54.600315 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:56:54.601069 (+   754us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:56:54.601407 (+   338us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:56:54.603894 (+  2487us) write_op.cc:312] APPLY: finished
0709 12:56:54.608833 (+  4939us) log.cc:844] Serialized 41 byte log entry
0709 12:56:54.609049 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:54.609309 (+   260us) write_op.cc:454] Released schema lock
0709 12:56:54.609376 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:56:54.609927 (+   551us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:54.609962 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":159882,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":118225,"apply.run_cpu_time_us":3589,"apply.run_wall_time_us":36277,"num_ops":4,"prepare.queue_time_us":1396501,"prepare.run_cpu_time_us":44631,"prepare.run_wall_time_us":397921,"raft.queue_time_us":74176,"raft.run_cpu_time_us":31498,"raft.run_wall_time_us":89074,"replication_time_us":5015242,"spinlock_wait_cycles":2136660352,"thread_start_us":5326798,"threads_started":4,"wal-append.queue_time_us":3315885}]]}
I20260709 12:56:55.513237 15437 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.653785 13540 raft_consensus.cc:606] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:55.651669 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:55.661931 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 524541us
I20260709 12:56:55.663127 13540 raft_consensus.cc:614] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:56:55.697746 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.690537 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:55.699172 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 47569.7us
I20260709 12:56:55.700924 13506 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:55.703377 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.713845 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.698734 13686 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:55.629619 15380 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:56:55.718815 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.719933 13689 raft_consensus.cc:2410] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:56:55.724222 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.724409 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.725302 13686 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:55.725494 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:55.726560 13685 raft_consensus.cc:2410] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:56:55.676287 15364 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:55.726372 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.728461 13692 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:56:55.682932 15421 consensus_meta.cc:287] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.000s
W20260709 12:56:55.695211 15417 consensus_meta.cc:287] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.002s
W20260709 12:56:55.683516 15436 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:56:55.729297 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.731181 15454 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:55.660568 15157 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:56:55.736047 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.736749 13684 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:55.716941 15340 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:55.738780 13690 consensus_meta.cc:287] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.002s
I20260709 12:56:55.739236 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.744047 15445 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:55.734895 15076 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:55.745949 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.741254 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.718398 15331 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:55.740338 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:55.784133 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:55.787503 13691 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:55.786417 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:55.781726 13539 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:55.792688 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:55.753850 15142 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:55.783857 13689 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:55.793445 13542 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
W20260709 12:56:55.785677 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 197092us
I20260709 12:56:55.796948 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:55.801389 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:55.807494 13686 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:55.802249 13509 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:55.811877 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:55.812067 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:55.814987 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:56:55.819093 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 301481us
I20260709 12:56:55.819322 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:55.829643 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.783222 15445 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:55.784492 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.783420 13690 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:55.832923 13686 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:55.837075 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.801342 15142 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:55.735251 15421 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.809216 15374 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.847210 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:55.779567 15340 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:55.867558 15450 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:56:55.869020 15340 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:55.818907 15276 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:56:55.816277 15359 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:55.872730 15437 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:55.874071 15359 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:56:55.874753 15359 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:55.875156 15157 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:55.838096 15451 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:55.836642 15445 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:55.806689 15076 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.800539 15331 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:55.886775 13684 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:55.926434 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:55.854768 15440 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:55.856339 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.750568 15417 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:55.859125 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:55.831225 15280 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:55.837877 15434 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:55.866452 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2470) took 1228 ms. Trace:
I20260709 12:56:55.866674 13541 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:56:55.857403 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 357453us
I20260709 12:56:55.875731 15359 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:56.211100 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.019s	sys 0.018s
I20260709 12:56:55.831827 15453 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:56:56.264911 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 106511us
W20260709 12:56:56.264190 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 434537us
W20260709 12:56:55.968963 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 268065us
W20260709 12:56:56.131647 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 329482us
I20260709 12:56:56.265784 13686 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:56:56.219372 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.014s	sys 0.024s
W20260709 12:56:56.271452 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.014s	sys 0.027s
W20260709 12:56:56.271793 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 677719us
I20260709 12:56:56.267314 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:56:56.065053 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 484723us
W20260709 12:56:56.268659 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.022s	sys 0.018s
W20260709 12:56:56.274317 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 667899us
I20260709 12:56:56.279325 13689 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:56.276338 15457 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:56:56.264809 15193 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.001s
I20260709 12:56:56.282130 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:56.284289 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:56.176010 15376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.015s	sys 0.029s
W20260709 12:56:55.951243 15418 consensus_meta.cc:287] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.679s	user 0.000s	sys 0.010s
I20260709 12:56:56.280838 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:56.286547 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:56.287734 13619 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:55.933673 15437 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:56.282151 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:56.261200 15340 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:56.014071 15385 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:56.269330 15453 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:56:56.276953 15328 consensus_meta.cc:287] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.753s	user 0.011s	sys 0.003s
W20260709 12:56:56.297531 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 516094us
I20260709 12:56:55.979382 13618 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:56.276347 15276 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:56.266230 15379 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.279809 15432 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:56.072755 15345 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:56.156728 15312 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.631s	user 0.012s	sys 0.000s
I20260709 12:56:56.307191 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:56.308178 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:56.317114 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:56.264671 13690 rpcz_store.cc:276] 0709 12:56:54.628007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:54.658496 (+ 30489us) service_pool.cc:224] Handling call
0709 12:56:55.168578 (+510082us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:56:55.853667 (+685089us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13085440}
W20260709 12:56:56.292182 15376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.019s	sys 0.032s
W20260709 12:56:56.302062 15415 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.002s	sys 0.014s
I20260709 12:56:56.236080 13684 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:56:56.295779 15443 consensus_meta.cc:287] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.796s	user 0.001s	sys 0.010s
W20260709 12:56:56.301842 15409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.008s	sys 0.008s
W20260709 12:56:56.315893 15322 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.981s	user 0.009s	sys 0.001s
I20260709 12:56:56.323892 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.324358 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.302107 15384 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:56.290284 15371 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:56:55.979199 15428 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:56.302714 15356 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.889s	user 0.007s	sys 0.003s
W20260709 12:56:56.330823 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53122.3us
I20260709 12:56:56.321466 15418 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:56:56.329452 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47377.5us
I20260709 12:56:56.309754 15434 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:56.332131 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:56.327145 15415 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.002s	sys 0.015s
I20260709 12:56:56.301815 15413 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:56:56.345192 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2284) took 1927 ms. Trace:
I20260709 12:56:56.310161 15453 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:56:56.356949 15472 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:56.332154 15443 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:56.357672 15472 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.332451 15364 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:56.309463 15340 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.360262 15364 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.362051 15340 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.362449 15364 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.360282 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:56.362965 15277 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.000s
I20260709 12:56:56.178191 15142 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.364207 15421 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:56.364404 15142 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:56.364084 15385 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:56.364975 15421 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:56.334827 15384 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:56.366144 15384 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:56.338990 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:56.329583 15442 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.000s
I20260709 12:56:56.364066 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.367785 13537 raft_consensus.cc:2393] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:56.367735 15442 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.368433 13541 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:56.368826 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.323253 15312 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.335834 15387 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:56.372483 15387 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:56.372884 15387 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:56.373195 15312 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:56.374477 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.375303 13614 raft_consensus.cc:2368] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5.
I20260709 12:56:56.331947 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.375996 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:56.376201 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.376354 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2296) took 1173 ms. Trace:
I20260709 12:56:56.376870 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:56.376960 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:56.377894 13610 raft_consensus.cc:2368] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3.
I20260709 12:56:56.376016 13540 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:56.379133 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:56.379809 13616 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:56:56.376407 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:56.325968 15331 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.309760 15463 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:56.347141 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.318097 15432 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.334391 15280 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.342033 15356 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:56:56.345649 13619 rpcz_store.cc:276] 0709 12:56:54.412475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:55.090165 (+677690us) service_pool.cc:224] Handling call
0709 12:56:55.090540 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:56.288337 (+1197797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:56.339471 (+ 51134us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540013a1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:56.339649 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2752704384}
I20260709 12:56:56.336767 13688 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:56.295437 15193 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.308722 15437 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.359624 15443 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:56.442761 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 108931us
W20260709 12:56:56.330106 15409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.008s	sys 0.008s
W20260709 12:56:56.443100 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 108609us
I20260709 12:56:56.443866 13615 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:56:56.452260 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:56.446852 15418 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:56.363420 15277 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:56.509130 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 164486us
I20260709 12:56:56.377763 15379 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:56.317922 15276 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.330137 15345 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.116s
I20260709 12:56:56.370330 15442 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:56.522392 13613 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:56:56.524206 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 207208us
I20260709 12:56:56.316367 15328 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:56.334964 15322 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.511284 15364 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:56:56.531685 13508 leader_election.cc:336] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 1.515s (SENT)
I20260709 12:56:56.520515 15457 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:56.505975 15468 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:56.582520 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.612989 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:56.586316 14210 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.021s	sys 0.023s
I20260709 12:56:56.628437 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:56.629807 13693 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:56:56.629901 14210 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.021s	sys 0.024s
I20260709 12:56:56.630136 13689 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:56.630317 14212 leader_election.cc:400] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5.
I20260709 12:56:56.630699 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:56.631491 13692 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:56.631858 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:56.633328 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.520639 15463 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:56:56.578106 15435 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.062s	user 0.005s	sys 0.008s
I20260709 12:56:56.381420 15445 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.650748 13688 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:56.651417 15445 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.510214 15413 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:56:56.652701 15413 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:56.653496 15413 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:56.653239 15435 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:56.654474 15413 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:56.655222 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.510002 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.655997 13617 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:56.656414 13535 raft_consensus.cc:2393] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:56.656834 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:56.657303 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.657454 13617 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:56.658519 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.614696 15443 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:56:56.635639 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:56.664463 14246 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:56.665282 15277 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:56.667212 15384 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.668583 14243 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:56.669401 15384 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.598212 15364 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:56.669871 14239 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:56.578752 15418 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.670483 15364 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:56.672302 15418 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.637131 14212 leader_election.cc:403] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:56:56.673393 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 95679.3us
I20260709 12:56:56.674243 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.674813 14212 leader_election.cc:400] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3.
I20260709 12:56:56.675166 14212 leader_election.cc:403] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:56:56.675941 15452 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:56.676524 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.677044 15460 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:56:56.677475 14243 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:56.679188 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:56.639420 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:56.680721 13691 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:56.681034 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:56.681739 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2294) took 1480 ms. Trace:
I20260709 12:56:56.681875 15471 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:56:56.671712 15364 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:56.682446 15471 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:56.682919 15471 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:56.684481 13692 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:56.645952 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:56.685499 13690 raft_consensus.cc:606] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:56.685886 13690 raft_consensus.cc:614] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:56:56.646641 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.687191 13614 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:56.687096 15482 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:56.646863 14242 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:56.647715 15457 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.687747 15482 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.682142 13618 rpcz_store.cc:276] 0709 12:56:55.201562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:55.214352 (+ 12790us) service_pool.cc:224] Handling call
0709 12:56:55.253767 (+ 39415us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:55.290056 (+ 36289us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:56:55.290296 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:56.308010 (+1017714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:56.681514 (+373504us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b540030a7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:56.681686 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595498624}
I20260709 12:56:56.688776 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:56.689563 14242 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:56.690362 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.648178 15432 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:56.692574 15476 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:56:56.636056 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.613431 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.679678 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.652750 15123 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:56.578502 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.681957 14241 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:56.684332 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.685734 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.707001 14244 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:56.629292 15468 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:56.597136 15331 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 12:56:56.648821 15463 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.635237 13539 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:56.376892 13612 rpcz_store.cc:276] 0709 12:56:55.202571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:55.205825 (+  3254us) service_pool.cc:224] Handling call
0709 12:56:55.285282 (+ 79457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:55.351954 (+ 66672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:55.370890 (+ 18936us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:55.484122 (+113232us) log.cc:844] Serialized 26 byte log entry
0709 12:56:55.852910 (+368788us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:56:55.855290 (+  2380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:55.874171 (+ 18881us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:56.376112 (+501941us) raft_consensus.cc:1685] finished
0709 12:56:56.376144 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:56.376295 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7833,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":22318,"thread_start_us":341067,"threads_started":1,"wal-append.queue_time_us":886821}
I20260709 12:56:56.714855 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.715425 13612 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:56.717970 14245 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:56:56.642653 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:56.643640 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 111982us
I20260709 12:56:56.681252 15328 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:56.655603 15435 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:56.702139 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:56.720014 14246 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:56:56.722934 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:56.723495 13537 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:56.655696 15413 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:56.598312 15472 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.705916 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.672067 15459 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:56:56.709311 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:56.736210 13619 raft_consensus.cc:2393] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:56.711055 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.739470 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:56.741392 13539 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:56.712883 13689 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
W20260709 12:56:56.719403 13508 leader_election.cc:336] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 1.621s (SENT)
I20260709 12:56:56.742110 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:56.587375 15379 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.743512 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1141) took 2114 ms. Trace:
I20260709 12:56:56.744199 13534 rpcz_store.cc:276] 0709 12:56:54.629298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:54.631993 (+  2695us) service_pool.cc:224] Handling call
0709 12:56:54.632331 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:55.627339 (+995008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:56.743273 (+1115934us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54002837d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:56.743455 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2074647424}
I20260709 12:56:56.746093 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:56.736045 14238 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:56:56.721587 13582 leader_election.cc:336] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.660s (SENT)
I20260709 12:56:56.724215 13618 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:56.726639 15356 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:56.727149 15483 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:56.727607 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.835711 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1122) took 1093 ms. Trace:
I20260709 12:56:56.728482 15440 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:56.721802 15387 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.729803 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:56.712188 15468 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:56.704727 15123 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:56.735091 15193 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:56.747529 13610 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:56.858112 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.793107 13611 raft_consensus.cc:2410] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:56:56.840214 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 120833us
I20260709 12:56:56.819705 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.792698 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:56:56.858186 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 136631us
W20260709 12:56:56.905164 13582 leader_election.cc:336] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.736s (SENT)
I20260709 12:56:56.905607 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:56.906370 13685 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:56.778363 13686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:56:56.907151 13686 raft_consensus.cc:606] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:56.908015 13686 raft_consensus.cc:614] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:56:56.769685 14244 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:56.721386 13535 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:56:56.865713 15409 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:56.876919 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.910773 13619 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:56.875253 13692 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:56:56.892613 13543 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:56.848124 15322 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:56.903394 13539 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:56.860728 13684 rpcz_store.cc:276] 0709 12:56:55.741924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:55.793123 (+ 51199us) service_pool.cc:224] Handling call
0709 12:56:56.835463 (+1042340us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b540038fc54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:56.835624 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153704832}
W20260709 12:56:56.892395 15446 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.240s	user 0.013s	sys 0.039s
I20260709 12:56:56.917866 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:56.917898 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:56.812608 15276 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.812745 15452 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5.
I20260709 12:56:56.815804 15460 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3.
I20260709 12:56:56.903393 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:56.921140 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.819902 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.923736 14212 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:56.921516 13687 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:56.915861 13691 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
W20260709 12:56:56.902594 15453 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.000s
I20260709 12:56:56.810510 15482 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.958948 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.919308 13684 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:56.959852 13539 raft_consensus.cc:2368] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3.
W20260709 12:56:56.920118 13582 leader_election.cc:336] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.767s (SENT)
I20260709 12:56:56.769889 15472 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:56.921945 13536 raft_consensus.cc:2368] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:56:56.963753 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:56.902339 15440 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:56.818617 15417 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:56.971016 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:56.927585 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.931210 13615 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:56:56.934913 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:56.972364 13618 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:56:56.972684 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1108) took 1270 ms. Trace:
I20260709 12:56:56.913100 15428 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.936801 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.937992 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.891371 15471 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.956863 15437 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.960078 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.908456 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:56.960539 13582 leader_election.cc:336] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.957s (SENT)
W20260709 12:56:56.936803 15488 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:56:56.963070 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:56.965379 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:56.877084 15483 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.973059 13693 rpcz_store.cc:276] 0709 12:56:55.702519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:55.703051 (+   532us) service_pool.cc:224] Handling call
0709 12:56:56.972514 (+1269463us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b5400244854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:56.972633 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2127565056}
I20260709 12:56:56.937336 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:56.915433 15356 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:56:56.957870 15076 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.919451 15409 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:56.957329 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:56.937184 13534 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:56.968400 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:56.991076 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.232913 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:57.053463 13619 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:56:56.991583 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:57.269338 13613 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:56:57.234292 15276 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.282879 15276 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.284698 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.285017 15276 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.285396 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.285632 15276 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:56.973764 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:57.267747 13684 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:57.006564 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.287015 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:56.974154 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.270825 13689 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
W20260709 12:56:57.273149 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 312619us
I20260709 12:56:57.276401 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.291261 13536 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:57.277082 15487 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:57.024690 15417 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:57.278224 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 315231us
I20260709 12:56:57.278700 15312 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.293675 15471 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:57.276456 13685 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:57.294142 15417 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.280393 13539 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:57.296087 15331 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:57.296490 15331 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:57.296872 15331 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:57.298684 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:57.299033 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.299531 15459 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:57.278877 13687 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:57.300078 15459 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:57.300235 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.301119 15467 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:57.301028 15459 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.302105 15428 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:57.301021 14241 raft_consensus.cc:2368] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
I20260709 12:56:57.302964 13539 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:57.302572 15428 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.303702 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.304849 15428 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:56.996876 15482 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.305253 15417 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.305413 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.292706 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:56.994302 15452 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.316s
I20260709 12:56:57.306818 13690 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:57.307003 15452 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:56:57.293008 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.293440 15487 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:57.307772 15435 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:56:57.308290 14238 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.308429 15487 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.309183 15483 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.309440 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:56.960963 15453 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.310693 15123 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:57.311265 15437 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:57.185549 15376 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:57.311954 13541 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:56:57.295118 14244 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:57.295543 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.312207 15376 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.312680 15468 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.296200 15345 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:56:57.313865 14239 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.314365 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.305555 15482 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.016624 13688 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:56:57.270287 15484 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:57.313030 13691 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:56:57.315099 15376 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.309710 15487 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:57.316399 15482 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:56:57.316733 13612 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.000s
I20260709 12:56:57.317106 13612 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:56:57.317320 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.318245 14238 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.307415 15452 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:57.319463 15461 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:57.319203 15452 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.320050 15461 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:57.320488 15452 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:57.321012 15461 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.290546 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.326548 13537 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:57.301734 15467 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:57.328763 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.328857 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.328729 15467 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.330166 15387 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.330328 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.331542 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:56:57.332152 15489 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.312s	user 0.003s	sys 0.000s
I20260709 12:56:57.333048 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:57.291785 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 318075us
I20260709 12:56:57.338330 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:56.994398 13614 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:56:57.340299 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.340761 14242 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.291538 13535 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:57.341557 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.341893 13619 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:57.342303 13540 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:57.343204 13612 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.343044 15364 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:57.344254 13616 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.345875 15440 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:57.015123 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:57.348264 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:57.314831 15468 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:57.282331 13542 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:57.291985 13584 leader_election.cc:336] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.501s (SENT)
I20260709 12:56:57.351429 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.350338 13618 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:56:57.351804 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:57.356160 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 64220us
W20260709 12:56:57.356757 13584 leader_election.cc:336] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.519s (SENT)
I20260709 12:56:57.357143 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.359091 15193 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:57.360219 15193 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:57.362758 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.366752 15468 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:56:57.367931 15492 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.280s	user 0.003s	sys 0.000s
I20260709 12:56:57.368079 13535 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:57.367961 15468 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:57.416785 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.372452 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.420936 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:57.418274 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.430779 13535 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:57.431751 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.353528 15076 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:56:57.360543 15193 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:57.458642 15076 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:57.384443 13584 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.459179 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.361275 15276 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.460606 13616 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
W20260709 12:56:57.472043 14209 leader_election.cc:336] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.535s (SENT)
I20260709 12:56:57.456409 15123 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.461516 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.419356 15424 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:57.363147 15482 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.433214 15345 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.421927 15468 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:57.432950 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.455231 15471 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:57.355729 15312 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
W20260709 12:56:57.463258 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 172048us
I20260709 12:56:57.472707 15483 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:57.472317 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 88002.8us
I20260709 12:56:57.496475 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:57.485431 14209 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.672s (SENT)
I20260709 12:56:57.462134 15440 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:57.470899 15076 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:56:57.455724 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 93073.4us
I20260709 12:56:57.499346 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.499773 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.500200 13619 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.497469 14209 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:57.498665 13582 leader_election.cc:336] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.143s (SENT)
W20260709 12:56:57.501761 13582 leader_election.cc:336] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.168s (SENT)
W20260709 12:56:57.502247 13582 leader_election.cc:336] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.192s (SENT)
W20260709 12:56:57.502792 13582 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.246s (SENT)
I20260709 12:56:57.503441 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.503933 13619 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:56:57.504349 13619 pending_rounds.cc:85] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 2
I20260709 12:56:57.504169 13611 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:57.504603 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:57.506331 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:57.506942 13616 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:56:57.509223 13582 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.284s (SENT)
W20260709 12:56:57.510080 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:57.512264 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.495038 13535 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:57.513741 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.514353 13538 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:57.493666 15345 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.517289 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.515652 14241 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:56:57.520713 14240 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.520932 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.521595 14241 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:57.521778 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.522435 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.519981 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.507140 15123 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:56:57.516764 13658 connection.cc:256] RPC call timeout handler was delayed by -1.1081120850003572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
I20260709 12:56:57.523684 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.523005 13611 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:56:57.508677 15331 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.525930 15482 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:56:57.501669 15471 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:57.508033 15142 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:57.537770 13616 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:56:57.539692 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.540490 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.519749 13610 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:56:57.541314 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.524534 15345 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.524966 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.505023 13619 pending_rounds.cc:107] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.3
I20260709 12:56:57.522449 14240 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:57.548328 13619 raft_consensus.cc:2917] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Aborting config change with OpId 2.3: Aborted: Op aborted by new leader
I20260709 12:56:57.550865 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:57.548818 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.551266 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.551832 13688 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.550981 13619 raft_consensus.cc:2889] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:56:57.523048 14241 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:57.552726 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:39894 (request call id 9) took 14730 ms. Trace:
I20260709 12:56:57.544207 15428 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:56:57.553642 14241 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:56:57.545378 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.554430 15379 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.545621 13689 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:56:57.555488 13689 pending_rounds.cc:85] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 4
I20260709 12:56:57.555755 13689 pending_rounds.cc:107] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:56:57.556005 13689 raft_consensus.cc:2889] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:56:57.546919 13693 raft_consensus.cc:2410] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:56:57.514010 15322 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:57.516387 15437 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:56:57.553306 13619 rpcz_store.cc:276] 0709 12:56:42.822598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:42.850642 (+ 28044us) service_pool.cc:224] Handling call
0709 12:56:42.906982 (+ 56340us) log.cc:844] Serialized 499 byte log entry
0709 12:56:42.942524 (+ 35542us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b540026c3a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7296b8 00007fa02e7292fa 00007fa02e7ad965 00007fa02e7b903c 00007fa02e7b86b3 00007fa033778d29 00007fa02ccf71d4 00007fa02ccf7162 00007fa02ccf7091 00007fa02ccf700c
0709 12:56:57.552619 (+14610095us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103240,"raft.run_cpu_time_us":1742,"raft.run_wall_time_us":1741,"spinlock_wait_cycles":4386048,"thread_start_us":123732,"threads_started":2,"wal-append.queue_time_us":29378,"wal-append.run_cpu_time_us":1903,"wal-append.run_wall_time_us":1002390}
I20260709 12:56:57.500188 15467 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:56:57.523849 15417 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:57.552723 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:57.559962 15417 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:57.561882 15417 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:57.568792 14241 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:56:57.542722 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.562491 13691 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.569689 15331 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:57.547003 13538 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:57.577170 15471 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.578526 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1057) took 1050 ms. Trace:
I20260709 12:56:57.578890 13543 rpcz_store.cc:276] 0709 12:56:56.527699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:56.527872 (+   173us) service_pool.cc:224] Handling call
0709 12:56:57.578330 (+1050458us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000b90d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:57.578451 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604142464}
I20260709 12:56:57.542420 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.577754 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.580144 15445 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:56:57.580435 13614 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:57.580075 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.581314 15373 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:57.581660 15373 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:57.582021 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.582711 13543 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.582415 15373 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.583714 13543 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:57.576346 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.586894 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.588289 13538 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:56:57.553429 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.589641 13692 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:57.588195 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:57.537053 15424 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:57.590590 13538 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:56:57.593657 13542 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:57.595316 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.598690 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.600242 13543 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:56:57.601531 13538 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.599364 13509 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:57.589056 15497 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.599586 15379 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:56:57.590744 15345 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.608863 15345 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.609881 13535 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:57.610442 15483 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.611254 15483 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:57.602403 15463 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:56:57.600860 15417 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.614312 15417 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:57.614921 15417 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:57.598409 15276 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:57.616303 15413 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:57.616758 15413 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:57.617084 15413 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:57.598300 13690 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:57.620865 13535 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:56:57.622177 13535 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:57.623555 13614 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:57.650879 15468 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.653563 15468 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:57.648358 15505 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.655154 15505 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:57.653489 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 144294us
I20260709 12:56:57.625305 15123 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.625972 14238 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.661414 15123 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.665560 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1052) took 1340 ms. Trace:
W20260709 12:56:57.666704 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 136918us
I20260709 12:56:57.668016 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.630622 13542 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:56:57.607300 15497 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.611333 15345 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:57.634130 15463 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:57.647977 15142 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:56:57.636561 15076 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:57.647188 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:57.659734 14209 leader_election.cc:336] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.742s (SENT)
I20260709 12:56:57.649324 15483 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:57.657569 15505 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:57.646765 13690 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
W20260709 12:56:57.669961 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 127325us
I20260709 12:56:57.693025 15483 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.693719 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.694411 15483 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:57.695267 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:56:57.670588 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 86059.4us
I20260709 12:56:57.672852 13619 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:56:57.700529 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:56:57.701537 14209 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.036s (SENT)
I20260709 12:56:57.670375 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:57.683925 13614 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:57.703668 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.704788 15506 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.704986 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:57.705996 15506 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:57.706593 13584 leader_election.cc:400] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3.
I20260709 12:56:57.663821 15123 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:57.668947 15471 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:57.709393 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.713049 13584 leader_election.cc:403] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:56:57.674998 15371 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:57.715271 15471 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.722265 15471 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:57.723387 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.724781 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.725713 14246 raft_consensus.cc:2410] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:56:57.725653 15193 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.726807 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.717480 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.728576 14243 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:56:57.734202 15510 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:56:57.734750 15076 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:57.732719 13543 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.735503 13612 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:56:57.741119 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:57.684839 15463 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:57.720134 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:57.719437 14240 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:56:57.727025 15193 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:57.761868 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 219765us
I20260709 12:56:57.718017 15453 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:57.666129 13541 rpcz_store.cc:276] 0709 12:56:56.325538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:56.369072 (+ 43534us) service_pool.cc:224] Handling call
0709 12:56:56.369426 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:57.312423 (+942997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:57.665251 (+352828us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b5400003054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:57.665485 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2166979456}
I20260709 12:56:57.712512 15506 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:57.761332 15371 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:56:57.782965 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:56:57.781078 15506 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:57.788038 15482 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:57.789218 15497 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.789776 15463 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.791487 13610 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:56:57.767275 15484 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 3.
I20260709 12:56:57.792966 15371 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:57.799691 15076 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:57.804729 15076 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:57.804227 15371 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.805222 15076 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:57.794175 15413 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.808686 15508 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:57.810341 15413 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.820539 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:57.821131 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 122396us
I20260709 12:56:57.821640 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.795255 15417 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.823451 14211 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:57.824205 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:56:57.825810 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:57.826061 15452 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:56:57.826393 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:57.825968 15517 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:57.826341 14211 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:57.827086 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:57.827133 15452 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:57.828557 15440 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:57.831260 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.832250 13541 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:57.833595 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.833887 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.834614 13539 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:57.838654 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.839381 13539 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:57.839860 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.841190 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:57.841660 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.843734 13508 leader_election.cc:400] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
I20260709 12:56:57.844195 13508 leader_election.cc:403] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:56:57.845117 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 174840us
I20260709 12:56:57.819000 15482 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:57.861505 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.856774 13692 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:57.865499 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.858184 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:57.866842 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:57.867448 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.815876 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 346) took 1144 ms. Trace:
I20260709 12:56:57.868152 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:57.869366 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.869424 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:57.818027 15508 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:56:57.873417 13539 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:57.868706 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.875406 13691 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:57.875385 13693 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:57.851320 15497 raft_consensus.cc:2764] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:56:57.877741 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.880195 13539 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:56:57.854890 15123 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:56:57.880075 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:56:57.881186 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 106364us
I20260709 12:56:57.877247 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.890825 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:57.887843 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:57.868410 15482 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:57.887053 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.895143 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:57.895463 13689 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:57.872287 15453 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
W20260709 12:56:57.895952 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:57.899086 14243 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:56:57.877727 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:57.855129 15505 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:56:57.858469 15413 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:56:57.874136 15487 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:56:57.920483 13686 raft_consensus.cc:2368] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3.
I20260709 12:56:57.884896 15524 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:57.874293 15387 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:56:57.921937 15424 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:56:57.915998 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:57.874461 14239 rpcz_store.cc:276] 0709 12:56:56.671185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:56.671407 (+   222us) service_pool.cc:224] Handling call
0709 12:56:56.671720 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:57.314272 (+642552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:57.815381 (+501109us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54007102d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:57.815714 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476231424}
I20260709 12:56:57.900739 15417 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:56:57.947184 15417 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:57.947106 15468 consensus_queue.cc:1059] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:57.946302 15142 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:57.948653 15417 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.958608 15424 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:57.962172 14237 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:56:57.935041 15524 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:57.859637 15511 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:56:57.887288 15123 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:57.897718 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:57.915172 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:57.988413 15435 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:57.894069 15376 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:56:57.976064 15516 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:57.975761 15387 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:57.978363 15409 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:57.985615 15123 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:56:57.997195 15435 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:56:58.044888 15435 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:58.046416 15435 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:58.047914 15516 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:58.049697 15516 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:57.989988 15468 consensus_queue.cc:1243] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:56:58.008953 15524 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:58.055392 15524 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:58.009637 15424 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.985666 15482 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:58.058821 15482 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:58.110878 15364 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:58.136591 15527 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:58.138602 15364 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:58.325865 15364 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:58.141261 15527 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:58.477828 15527 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:58.562920 15506 consensus_meta.cc:287] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.765s	user 0.000s	sys 0.010s
I20260709 12:56:58.563886 15506 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:58.801692 15518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.016s	sys 0.022s
W20260709 12:56:58.810277 15518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.016s	sys 0.023s
I20260709 12:56:58.812078 15506 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:58.817261 15373 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:58.842648 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.027s	sys 0.018s
W20260709 12:56:58.843086 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.028s	sys 0.019s
W20260709 12:56:58.845520 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:58.846793 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:58.848824 15531 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:58.849408 15531 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:58.852645 15531 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:58.871011 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:58.871488 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:58.872174 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:58.886297 15518 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:58.888568 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:58.889681 15518 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:58.891075 15518 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:56:58.893774 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:58.894053 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:58.894961 15518 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:58.895147 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:58.895396 15518 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:58.896011 15518 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:58.897863 13538 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:56:58.899021 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:58.899550 13538 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:56:58.899612 15487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.924s	user 0.007s	sys 0.006s
W20260709 12:56:58.900662 15487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.008s	sys 0.007s
I20260709 12:56:58.896688 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:58.903841 13539 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:56:58.907522 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:58.908288 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:58.910281 15524 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:58.910863 15524 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:58.911260 13686 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:56:58.913237 15524 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:58.919143 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 357) took 1399 ms. Trace:
I20260709 12:56:58.920003 14237 rpcz_store.cc:276] 0709 12:56:57.519434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.554832 (+ 35398us) service_pool.cc:224] Handling call
0709 12:56:57.555144 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:57.981660 (+426516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:58.918832 (+937172us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54005f1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:58.919068 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934563072}
I20260709 12:56:58.909497 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:58.921826 13539 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
W20260709 12:56:58.922351 13508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.042s	sys 0.145s
W20260709 12:56:58.922787 13508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.042s	sys 0.145s
I20260709 12:56:58.923126 13539 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:56:58.923826 13538 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:56:58.924624 13539 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:56:58.923257 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 1.05646e+06us
I20260709 12:56:58.928076 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:58.927976 15518 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:56:58.928741 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:58.929121 15487 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:58.930717 15518 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:58.933425 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 1.01747e+06us
I20260709 12:56:58.934238 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:58.941203 15539 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:58.941887 15539 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:58.944226 15539 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:58.944957 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:58.950101 15487 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:58.945560 15518 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:56:58.942775 13584 connection.cc:256] RPC call timeout handler was delayed by -1.8735750299993015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 12:56:58.950673 15518 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:56:58.950862 13584 connection.cc:256] RPC call timeout handler was delayed by -1.7772750299991458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 12:56:58.951148 13489 auto_rebalancer.cc:629] Failed to schedule move for tablet 3f54dde33fa845098e64673b576d780c: Aborted: Op aborted by new leader
W20260709 12:56:58.951589 13584 leader_election.cc:336] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.865s (SENT)
I20260709 12:56:58.952088 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:58.952811 15524 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:58.954196 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:58.954062 15487 consensus_queue.cc:1059] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:58.955124 15487 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:58.951448 15518 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:58.955812 15487 raft_consensus.cc:3055] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:56:58.956218 15487 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:58.957011 15487 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:58.958132 15487 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:56:58.962236 13584 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 2.048s (SENT) [suppressed 61 similar messages]
I20260709 12:56:58.964164 15539 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:58.964641 15539 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:58.967978 15516 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:58.968520 15516 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:58.969537 15437 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:58.969971 15437 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:56:58.970081 13584 connection.cc:256] RPC call timeout handler was delayed by -1.7495676789994832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 12:56:58.970686 15516 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:58.970829 13584 connection.cc:256] RPC call timeout handler was delayed by -1.6944750299999214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22599999999999998s for a response
I20260709 12:56:58.963541 15518 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:58.971390 13584 connection.cc:256] RPC call timeout handler was delayed by -1.6728750299998865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2284s for a response
W20260709 12:56:58.971938 13584 connection.cc:256] RPC call timeout handler was delayed by -1.4541750299995329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25270000000000004s for a response
W20260709 12:56:58.972294 13584 connection.cc:256] RPC call timeout handler was delayed by -1.1976750299991181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2812s for a response
I20260709 12:56:58.973106 15518 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:58.968595 15539 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:58.980423 15437 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:58.981743 15437 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:58.985673 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:58.947482 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:58.986451 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2323) took 1299 ms. Trace:
I20260709 12:56:58.986760 13614 rpcz_store.cc:276] 0709 12:56:57.686716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.703222 (+ 16506us) service_pool.cc:224] Handling call
0709 12:56:58.985527 (+1282305us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:56:58.986373 (+   846us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55010176}
I20260709 12:56:58.988224 13584 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:58.987908 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:58.989117 13614 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:56:58.989168 15518 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:58.989874 15518 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:58.990338 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:58.990880 15518 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:58.991521 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:56:58.992375 15193 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.223s	user 0.015s	sys 0.004s
I20260709 12:56:58.992781 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:58.993369 13614 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:58.996735 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:58.997597 13543 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:58.998139 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1154) took 1152 ms. Trace:
I20260709 12:56:58.998436 13543 rpcz_store.cc:276] 0709 12:56:57.845304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.852457 (+  7153us) service_pool.cc:224] Handling call
0709 12:56:58.947267 (+1094810us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:56:58.998084 (+ 50817us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8941824}
I20260709 12:56:58.999382 15524 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:58.999866 15524 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:59.000259 15524 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:59.002049 13693 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:56:59.002074 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:56:59.007558 15123 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.964s	user 0.005s	sys 0.009s
I20260709 12:56:59.008286 15524 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:59.021889 13619 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.165s	user 0.016s	sys 0.010s
I20260709 12:56:59.022290 13619 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:59.022647 15524 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.009213 13693 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:56:59.009915 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.024537 14237 raft_consensus.cc:2393] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:59.025738 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:59.026196 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2313) took 1528 ms. Trace:
I20260709 12:56:59.026372 13693 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:56:59.026600 13618 rpcz_store.cc:276] 0709 12:56:57.497228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.525977 (+ 28749us) service_pool.cc:224] Handling call
0709 12:56:57.548555 (+ 22578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:57.620266 (+ 71711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:57.620302 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:57.620341 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:56:57.620370 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:59.025960 (+1405590us) spinlock_profiling.cc:243] Waited 71.5 ms on lock 0x7b540013bfd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:56:59.026012 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:59.026133 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164444544}
W20260709 12:56:59.013788 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.009949 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:59.035467 13506 connection.cc:256] RPC call timeout handler was delayed by -1.2628129790000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 12:56:59.035698 15123 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.037567 15516 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.038093 15516 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.040949 15516 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.043004 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.044123 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.044449 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.045017 13543 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:56:59.046492 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.042239 15123 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:59.047539 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.047871 14237 raft_consensus.cc:2410] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:56:59.048126 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.048027 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:59.049192 15524 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:59.050320 15518 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:59.050845 15518 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:59.052140 15371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.021s	sys 0.023s
W20260709 12:56:59.052562 15371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.224s	user 0.021s	sys 0.023s
I20260709 12:56:59.052968 15518 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.058100 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1250) took 2431 ms. Trace:
I20260709 12:56:59.058490 13615 rpcz_store.cc:276] 0709 12:56:56.626946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:56.627146 (+   200us) service_pool.cc:224] Handling call
0709 12:56:56.930782 (+303636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:56.972374 (+ 41592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:59.057870 (+2085496us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54002fb7d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:59.058040 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652221696}
W20260709 12:56:59.060931 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51068us
I20260709 12:56:59.062549 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:59.063064 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.063653 15518 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:59.064877 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.065044 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:59.065461 14245 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:59.064889 14243 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:59.065752 15518 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:59.066960 13584 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:59.067883 15518 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.068595 15506 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:59.059670 15524 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:59.068879 15518 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:59.059868 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.069115 15506 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.070099 13615 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:59.069718 15518 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.069770 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:59.071266 15373 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:59.072239 15518 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.067026 13614 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:56:59.072153 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:59.072863 15516 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:59.073146 15445 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:59.073259 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.073551 15516 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:59.073778 15445 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:59.074097 13614 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:59.074281 15516 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:59.074494 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:59.075044 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.075270 15373 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:59.074924 15445 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.075666 15373 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:59.075745 13614 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:59.076224 15373 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:59.079567 13614 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:56:59.080087 13614 pending_rounds.cc:85] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 5
I20260709 12:56:59.080386 13614 pending_rounds.cc:107] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:56:59.080470 15506 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.080667 13614 raft_consensus.cc:2889] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:56:59.082343 15193 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:56:59.068185 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:59.057896 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.083827 14245 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:56:59.096545 15506 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:59.095983 15373 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.103870 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.105154 14237 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:56:59.115782 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08000bd260 after lost signal to thread 12409
W20260709 12:56:59.116250 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08003155a0 after lost signal to thread 12412
I20260709 12:56:59.118271 13692 raft_consensus.cc:2368] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6.
I20260709 12:56:59.118494 15373 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.121526 15424 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:59.122267 15424 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.175485 15531 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.176366 15531 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.181804 15531 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.202258 15424 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:59.223536 13692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 208) took 1364 ms (client timeout 1684 ms). Trace:
W20260709 12:56:59.224174 13692 rpcz_store.cc:269] 0709 12:56:57.858825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.865166 (+  6341us) service_pool.cc:224] Handling call
0709 12:56:59.223213 (+1358047us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002ed6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:59.223446 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2878742784}
I20260709 12:56:59.231997 15524 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.232702 15524 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.091431 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:56:59.236932 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 145617us
I20260709 12:56:59.238469 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:59.240309 15534 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:59.240983 15534 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:59.242233 15534 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.244169 15532 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:56:59.245998 15532 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:59.247465 15532 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:59.231874 15518 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:59.259213 15518 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.248358 15529 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:59.265198 15529 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:56:59.266599 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.267168 15529 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:56:59.268409 15518 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.269131 15518 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:59.272626 15524 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:59.288020 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.250308 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:56:59.073753 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.253237 15371 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:59.292932 15371 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:59.269836 15518 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.083709 15516 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.295668 15371 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.097745 13618 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:56:59.299777 15371 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:59.300796 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.301576 13618 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:59.302937 14212 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.294386 13615 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:56:59.303912 15531 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:59.304412 15531 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:56:59.304720 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.305214 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.306200 15193 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:56:59.304853 15531 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:59.306900 15193 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:59.307503 15193 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:56:59.049387 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.317726 13543 raft_consensus.cc:2368] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
W20260709 12:56:59.316506 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 248380us
I20260709 12:56:59.328557 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:59.330659 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:59.332465 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:56:59.338840 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.343343 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.338392 15276 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:56:59.328837 15373 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:59.335039 15424 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.334197 15376 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:59.340337 15524 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.351603 15373 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:59.346007 15506 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.327157 15193 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.344655 15508 consensus_queue.cc:1059] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:56:59.366441 15276 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:56:59.372853 15276 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:56:59.373402 15276 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:56:59.375231 13535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.496s	user 0.002s	sys 0.018s
I20260709 12:56:59.361128 15376 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.375304 15517 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.377732 15376 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.378998 15508 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:59.379592 15508 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:59.380945 15508 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.382481 15508 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:59.375636 13535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.496s	user 0.003s	sys 0.019s
I20260709 12:56:59.383598 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1076) took 2403 ms. Trace:
W20260709 12:56:59.385293 13582 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.716s (SENT)
I20260709 12:56:59.385963 13582 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:59.364897 15424 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:56:59.376281 15517 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.390395 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.394637 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.363021 15516 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:56:59.396613 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 53336.9us
I20260709 12:56:59.366947 15373 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:59.401597 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.398350 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.406631 13610 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:56:59.408905 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.388103 15454 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.410786 15454 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.414124 15454 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.415232 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.415881 13615 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:56:59.417860 14212 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.418803 13584 leader_election.cc:400] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:56:59.419214 13584 leader_election.cc:403] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:56:59.418946 15539 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:59.395562 15524 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:56:59.422806 15524 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:59.422919 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:59.423241 15517 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.423651 13610 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:56:59.388149 15424 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:59.424232 15532 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:56:59.424835 15532 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.429255 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.434777 13615 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:59.422363 15539 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:59.427621 15506 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:56:59.427651 15193 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:59.429939 15454 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:59.439401 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:59.448812 15543 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 12:56:59.450295 15546 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:56:59.456317 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.457955 14239 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:56:59.461165 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.436529 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.440450 15539 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:56:59.474102 15440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.645s	user 0.044s	sys 0.046s
W20260709 12:56:59.474794 15440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.646s	user 0.044s	sys 0.046s
I20260709 12:56:59.475155 15440 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:59.424258 15524 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.476606 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.477664 13613 raft_consensus.cc:2393] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:59.463418 15454 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.439217 15276 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.481225 15549 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:58.073642 15076 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.481897 15549 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.483471 15276 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.384009 13535 rpcz_store.cc:276] 0709 12:56:56.979959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.623743 (+643784us) service_pool.cc:224] Handling call
0709 12:56:57.637811 (+ 14068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:56:57.639701 (+  1890us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:56:57.767643 (+127942us) log.cc:844] Serialized 14 byte log entry
0709 12:56:59.383185 (+1615542us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:56:59.383244 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:56:59.383302 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:59.383385 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:59.383517 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26784,"prepare.queue_time_us":147333,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":51204,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":29613,"thread_start_us":1671918,"threads_started":3,"wal-append.queue_time_us":73640,"wal-append.run_cpu_time_us":2115,"wal-append.run_wall_time_us":1002116}
I20260709 12:56:59.489462 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:59.463670 15373 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.496356 14237 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:56:59.496964 14237 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:59.497231 15497 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:56:59.497881 15497 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.491050 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:59.499686 13535 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:59.501014 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.490608 15531 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.498236 14237 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.545367 14237 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:56:59.547838 13509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.669s	user 0.025s	sys 0.041s
W20260709 12:56:59.548846 13509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.671s	user 0.025s	sys 0.043s
I20260709 12:56:59.534166 15497 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:59.533048 15076 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.555474 15531 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.564721 15534 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.565619 15534 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.567306 13687 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:59.581586 15193 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:56:59.583582 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.584717 15193 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:59.585276 15193 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:59.586263 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:59.586566 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:59.586823 13684 raft_consensus.cc:2376] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3. Re-sending same reply.
I20260709 12:56:59.587030 14237 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:56:59.587184 13692 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:59.587666 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:59.588253 13693 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:56:59.589022 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:56:59.589381 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:59.589824 13619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2319) took 1904 ms (client timeout 1869 ms). Trace:
I20260709 12:56:59.590237 13693 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:59.590106 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:59.590816 13684 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:59.585790 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:59.592043 13541 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:56:59.591972 13612 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:56:59.590260 13619 rpcz_store.cc:269] 0709 12:56:57.685604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.709096 (+ 23492us) service_pool.cc:224] Handling call
0709 12:56:59.589662 (+1880566us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400309dd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:56:59.589765 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331611392}
W20260709 12:56:59.595541 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 1.72606e+06us
I20260709 12:56:59.596629 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:56:59.596949 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.597033 13692 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:59.598476 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:59.599792 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:59.600723 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:59.602418 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:56:59.602799 13692 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:56:59.604348 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:59.604957 13619 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:59.534510 15532 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.610486 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:56:59.610652 13619 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:56:59.611879 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.613091 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:56:59.616212 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 79621.7us
I20260709 12:56:59.617473 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.618810 14243 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:56:59.619035 14209 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.684s (SENT)
I20260709 12:56:59.619503 14209 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:59.621413 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.622287 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.619120 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:59.608285 15517 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.625010 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:56:59.625195 14239 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:56:59.624091 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:56:59.625743 15517 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:56:59.626870 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.608515 15516 consensus_queue.cc:1059] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:56:59.629626 15516 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:56:59.630309 13584 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:56:59.633802 13610 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
W20260709 12:56:59.634297 15417 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.645s	user 0.047s	sys 0.102s
W20260709 12:56:59.634841 15417 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.646s	user 0.047s	sys 0.102s
I20260709 12:56:59.637171 15549 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:56:59.637734 15549 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:56:59.638125 15549 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:59.638993 15549 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.635766 15417 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:56:59.644507 15417 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:59.630434 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.647187 13537 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:59.614748 15276 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.631446 15376 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.648504 15276 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:59.648903 15276 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:59.648232 15417 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.649230 15276 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:59.623344 15193 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.617561 15424 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:56:59.655076 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.632104 15454 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:56:59.657807 15424 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:56:59.659427 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.616652 15552 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:56:59.661232 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.833s	user 0.036s	sys 0.063s
W20260709 12:56:59.661605 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.834s	user 0.037s	sys 0.063s
W20260709 12:56:59.661895 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 1.83414e+06us
I20260709 12:56:59.662513 14211 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:59.664289 14211 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:56:59.665093 15364 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:59.620321 15524 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:56:59.662351 15552 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:56:59.622392 15532 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:56:59.672091 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:59.677057 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 58015.4us
I20260709 12:56:59.676996 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.653391 15508 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:56:59.659366 15424 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:59.681368 15424 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:59.669054 15373 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:56:59.685922 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.647154 15516 raft_consensus.cc:3055] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:56:59.673718 14243 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
W20260709 12:56:59.692086 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 62130.5us
I20260709 12:56:59.678480 13537 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:59.690991 15454 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:56:59.695142 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.696252 15556 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:59.680258 15508 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:56:59.700688 15508 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:56:59.691116 15516 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:56:59.705014 15413 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.705359 15516 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.705765 15413 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.675329 15552 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:56:59.699786 15539 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:57.934206 13690 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
W20260709 12:56:59.728328 13690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 196) took 1889 ms (client timeout 2095 ms). Trace:
W20260709 12:56:59.728833 13690 rpcz_store.cc:269] 0709 12:56:57.839116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.852658 (+ 13542us) service_pool.cc:224] Handling call
0709 12:56:59.728165 (+1875507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:56:59.693411 15193 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:56:59.696359 13537 raft_consensus.cc:2393] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:56:59.733145 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 52266.1us
I20260709 12:56:59.736677 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.737003 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:56:59.734496 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 109508us
W20260709 12:56:59.735211 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 69848.5us
I20260709 12:56:59.737618 13537 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:56:59.746865 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.749624 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:56:59.749769 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.750523 14246 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:56:59.751055 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.751694 13613 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:56:59.752106 13615 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:56:59.752345 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2324) took 2065 ms. Trace:
I20260709 12:56:59.753103 13613 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:56:59.754529 13615 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:56:59.723881 15556 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:59.752686 13611 rpcz_store.cc:276] 0709 12:56:57.686969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.706224 (+ 19255us) service_pool.cc:224] Handling call
0709 12:56:57.708216 (+  1992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:57.712976 (+  4760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:57.713036 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:57.749326 (+ 36290us) log.cc:844] Serialized 26 byte log entry
0709 12:56:57.849510 (+100184us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:57.849582 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:57.849712 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:56:59.752120 (+1902408us) raft_consensus.cc:1685] finished
0709 12:56:59.752141 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:56:59.752289 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3204,"raft.queue_time_us":8504,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":24650,"thread_start_us":73694,"threads_started":1,"wal-append.queue_time_us":84723,"wal-append.run_cpu_time_us":10822,"wal-append.run_wall_time_us":1131415}
I20260709 12:56:59.759060 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:59.760443 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.755373 15552 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:59.757033 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.761175 13611 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:56:59.766000 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.766718 13615 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:56:59.766373 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:56:59.739552 15516 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:56:59.764499 15373 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:56:59.770818 13537 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:56:59.774326 13539 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:56:59.781764 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:56:59.783977 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:56:59.741698 15534 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.781270 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:56:59.785887 15539 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.790665 13613 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:56:59.781998 15373 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.794996 15373 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:56:59.746688 13535 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:56:59.787480 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 44037.3us
I20260709 12:56:59.786890 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:56:59.806120 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 119503us
I20260709 12:56:59.757684 15276 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:56:59.810479 15561 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:56:59.722133 15440 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.812469 13611 raft_consensus.cc:2368] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
I20260709 12:56:59.776106 15532 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:56:59.819387 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:56:59.811506 15561 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:56:59.799639 13541 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:56:59.811542 15539 consensus_queue.cc:1059] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:56:59.857972 13689 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:56:59.859964 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:56:59.861467 15553 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.257s	user 0.004s	sys 0.001s
I20260709 12:56:59.861904 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:56:59.920914 15373 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:56:59.922133 15373 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:56:59.957105 15482 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.898s	user 0.016s	sys 0.001s
W20260709 12:57:00.018492 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.037554 13535 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:57:00.041338 13535 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:56:59.835008 13541 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:57:00.086035 15364 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.086570 15384 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.102465 15561 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.104301 15556 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:56:59.830152 15532 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:00.129165 15544 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.292s	user 0.006s	sys 0.005s
I20260709 12:57:00.091836 15413 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:00.158243 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 539193us
W20260709 12:57:00.159607 14209 leader_election.cc:336] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.095s (SENT)
I20260709 12:57:00.160048 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.078202 15508 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.164244 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:00.127775 15532 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.170565 13615 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.177557 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:00.182474 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 376768us
W20260709 12:57:00.187007 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 262213us
I20260709 12:57:00.192718 15562 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:00.193560 15562 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:00.117542 15384 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.115337 15364 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.153584 15373 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.199877 15559 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:00.204660 15452 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.377s	user 0.022s	sys 0.048s
I20260709 12:57:00.157923 15552 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.205317 15559 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.221937 15453 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:00.222663 15453 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:00.223258 15559 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.194936 15562 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.224673 15562 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:00.226449 15494 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 3.139s	user 0.004s	sys 0.014s
I20260709 12:57:00.232211 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1259) took 3491 ms. Trace:
I20260709 12:57:00.232405 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:00.233708 13541 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.234756 15345 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.235505 15345 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.237969 15331 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.238874 15076 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.238584 15331 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.240088 15459 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.240242 15558 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.241052 15558 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:00.242156 13659 connection.cc:256] RPC call timeout handler was delayed by -1.1584149990003425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 12:57:00.242280 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:00.242981 13535 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:00.243090 15482 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.243373 15511 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:00.243947 15482 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.235702 13691 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:57:00.247565 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2501) took 1256 ms. Trace:
I20260709 12:57:00.248179 13691 rpcz_store.cc:276] 0709 12:56:58.991345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:00.235088 (+1243743us) service_pool.cc:224] Handling call
0709 12:57:00.235504 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:00.247261 (+ 11757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:00.247499 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:00.232590 13617 rpcz_store.cc:276] 0709 12:56:56.740665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:56.741079 (+   414us) service_pool.cc:224] Handling call
0709 12:56:56.743783 (+  2704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:56.744188 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:56:56.744222 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:56:56.841080 (+ 96858us) log.cc:844] Serialized 26 byte log entry
0709 12:56:56.968133 (+127053us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:56:56.968173 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:56:56.968278 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:00.225609 (+3257331us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:00.231819 (+  6210us) raft_consensus.cc:1685] finished
0709 12:57:00.231840 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:00.232148 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3211,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":145,"spinlock_wait_cycles":5024000,"thread_start_us":48374,"threads_started":1,"wal-append.queue_time_us":117973}
W20260709 12:57:00.254873 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.258245 13615 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:00.260681 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:00.260848 13659 connection.cc:256] RPC call timeout handler was delayed by -1.1215751970003112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:57:00.261372 13659 connection.cc:256] RPC call timeout handler was delayed by -1.0123546790000546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 12:57:00.263124 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.239404 15076 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.274325 13615 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:00.267374 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:00.271633 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.276505 13613 raft_consensus.cc:2410] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:57:00.276867 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 346460us
I20260709 12:57:00.278457 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.278929 14212 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:00.265455 15452 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.438s	user 0.022s	sys 0.049s
I20260709 12:57:00.236544 15512 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.304616 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.246882 15435 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:00.265981 15505 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.244033 15511 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:00.320065 14245 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.016s	sys 0.023s
W20260709 12:57:00.335676 14245 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.020s	sys 0.025s
I20260709 12:57:00.265769 15437 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.334923 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:00.337304 13617 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 1.
I20260709 12:57:00.244812 15532 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:57:00.289706 15280 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:57:00.343464 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:00.240715 15459 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.362517 13615 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:00.364528 13613 raft_consensus.cc:2410] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:00.300096 15452 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:00.274891 15373 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:57:00.308224 15276 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.375614 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:00.378722 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 179118us
I20260709 12:57:00.260453 15453 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:00.384063 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.385445 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.387063 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.393556 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:00.374668 15497 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:00.375207 15532 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:00.400760 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:00.397307 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 118944us
I20260709 12:57:00.402217 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.402678 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.402940 13687 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:57:00.348248 15437 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.347710 15435 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.403889 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.407622 13684 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.366003 15512 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.363713 15505 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.409363 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:00.384755 14239 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:00.415585 15585 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:00.399103 15453 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:00.401173 15497 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:00.400461 15532 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:00.408779 15579 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 12:57:00.424486 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 145651us
I20260709 12:57:00.427979 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.430737 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:00.444752 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:00.423797 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.427539 15497 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:00.455287 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.427122 15384 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.427443 15508 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.426834 15558 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.429745 15331 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.436044 15364 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.460582 13535 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:00.465938 15424 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.784s	user 0.019s	sys 0.000s
I20260709 12:57:00.475564 13684 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:00.423153 15452 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.463807 15345 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.465816 15331 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:57:00.479738 15574 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:00.499619 15565 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.483290 15482 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.464579 15532 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.489960 15584 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:00.481441 15424 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.500654 15565 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.494583 15574 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:00.496485 15459 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:00.559427 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 75246.9us
I20260709 12:57:00.560174 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.502563 15579 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:57:00.501593 15573 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:00.534528 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:00.564824 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:00.566346 15577 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:57:00.555747 15574 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.535168 15584 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:00.560559 15193 consensus_meta.cc:287] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.819s	user 0.017s	sys 0.005s
I20260709 12:57:00.577132 15581 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:57:00.577948 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:00.580158 15586 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:57:00.563542 15570 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:57:00.555965 15511 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.532955 15482 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
W20260709 12:57:00.583689 15578 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:57:00.585870 15524 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.907s	user 0.005s	sys 0.008s
I20260709 12:57:00.503947 15568 consensus_queue.cc:1059] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.163s
I20260709 12:57:00.568679 15435 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.582319 15584 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:00.573591 15558 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:00.587435 15566 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:57:00.559787 15437 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.592952 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.593757 13617 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:00.567174 15459 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:00.582424 15573 raft_consensus.cc:697] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:00.596261 15589 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:57:00.580267 15424 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:00.600226 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 380110us
W20260709 12:57:00.580992 15576 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:57:00.582592 15505 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.603770 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:00.592384 15585 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:00.599056 15331 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:57:00.625948 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 198095us
I20260709 12:57:00.628742 13617 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
W20260709 12:57:00.618897 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 197393us
W20260709 12:57:00.630957 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 48226.7us
I20260709 12:57:00.632431 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:00.632444 14212 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.627017 15591 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:57:00.635967 15565 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.625121 14238 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:00.640746 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.643564 13508 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:00.647195 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.619982 15558 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.651126 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.655184 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.634058 15437 raft_consensus.cc:2764] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:57:00.657598 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.656903 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.655146 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2497) took 1666 ms. Trace:
I20260709 12:57:00.660025 13617 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:00.634294 13541 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:00.644935 15193 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:00.653000 15424 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.658628 15482 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:57:00.665988 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.633564 13535 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:00.675289 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:00.676137 13614 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:00.666568 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:00.647003 15586 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:00.659636 13686 rpcz_store.cc:276] 0709 12:56:58.984751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:58.984931 (+   180us) service_pool.cc:224] Handling call
0709 12:56:58.985282 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:00.232091 (+1246809us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:00.232129 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:00.232593 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:57:00.380900 (+148307us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:00.380944 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:00.381006 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:00.650898 (+269892us) raft_consensus.cc:1685] finished
0709 12:57:00.650915 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:00.651038 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1246317,"raft.queue_time_us":1247601,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":147897,"threads_started":1,"wal-append.queue_time_us":237328}
I20260709 12:57:00.665870 15435 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:57:00.648005 15594 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:00.665561 15592 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:57:00.672709 15558 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.648762 15512 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.659363 15331 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:00.679878 15193 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:00.673506 15585 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.682195 15482 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:00.668380 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.683931 15595 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:00.674902 15579 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 12:57:00.698668 15580 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:57:00.692358 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:00.685047 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:00.702463 15512 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:00.709007 15497 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:00.713151 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46670.5us
I20260709 12:57:00.718554 13616 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:00.676990 15591 consensus_queue.cc:1059] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:57:00.677431 15424 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.696617 15331 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:00.679042 15437 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:57:00.728219 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:00.728883 15516 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.958s	user 0.011s	sys 0.008s
I20260709 12:57:00.728269 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2325) took 3041 ms. Trace:
I20260709 12:57:00.722049 14237 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:00.701309 15593 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:57:00.722396 15595 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:00.738543 15595 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:00.730435 13688 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.006s
W20260709 12:57:00.740190 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.740296 13537 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 2.
I20260709 12:57:00.738196 13616 rpcz_store.cc:276] 0709 12:56:57.687244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.719097 (+ 31853us) service_pool.cc:224] Handling call
0709 12:57:00.712897 (+2993800us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:00.713185 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:00.727996 (+ 14811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:00.728214 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52368640}
I20260709 12:57:00.728760 13541 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:00.691236 15573 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.735074 15331 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:00.749577 15562 consensus_meta.cc:287] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.004s
I20260709 12:57:00.750691 14244 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:00.747223 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:00.742079 13688 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:00.743512 15193 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.754091 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:00.757908 13489 auto_rebalancer.cc:629] Failed to schedule move for tablet c88adb75300c48e98d20928bce53551d: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:57:00.764329 15123 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.769114 15123 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.751675 15497 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.766034 15516 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:00.783089 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.746451 15512 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:00.748427 13541 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:57:00.730032 15581 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:57:00.769367 15558 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:57:00.745821 15435 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:00.794445 14246 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
I20260709 12:57:00.787801 15516 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:00.739964 15482 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.806173 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.806782 15581 consensus_queue.cc:1243] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer 0f34cda059124f839711dbef61bb2c28 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:57:00.810099 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.811159 13686 raft_consensus.cc:2368] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3.
I20260709 12:57:00.810124 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.812546 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.813227 14246 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:57:00.814294 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.814968 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.796957 15435 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:00.816636 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.822856 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.816771 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.824760 13691 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:00.804513 15516 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:00.808367 15552 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.002s
I20260709 12:57:00.798287 15512 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:00.817400 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.818523 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.739141 15586 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.820889 15593 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:57:00.833663 15459 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:00.833925 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.833668 15603 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.834625 13689 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:00.836916 14209 leader_election.cc:400] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3.
I20260709 12:57:00.838621 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:00.840286 15512 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.843457 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:00.845945 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.846158 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:00.844364 15531 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:57:00.849123 15571 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:57:00.854660 15518 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.868203 15593 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:57:00.868604 15598 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:57:00.869032 15593 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:00.869701 15593 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.835783 13690 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.862311 15452 raft_consensus.cc:3055] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:57:00.863862 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.879480 13689 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 1.
I20260709 12:57:00.864830 15594 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.865715 13509 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.866461 15453 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.837803 15424 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.835389 15331 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.835031 15603 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.882602 15384 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:57:00.885550 15453 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.880698 15594 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.879216 15452 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:00.895445 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.899518 13689 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:00.900430 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.904484 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.906242 15571 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:00.896648 15384 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:00.908568 15384 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:00.909538 15331 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.846997 15435 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.912761 15435 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:00.911368 15453 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.907382 15571 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:00.912890 15601 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:57:00.912940 15603 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:00.930291 15452 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.936195 15593 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:00.940887 15571 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:00.944077 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:00.918394 15424 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:57:00.918666 15563 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:00.945185 13689 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:00.948087 15594 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:00.958819 15594 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:57:00.959139 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.960147 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.960263 14243 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:00.960310 13689 raft_consensus.cc:2410] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:57:00.961081 15483 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:00.961700 15483 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:00.961793 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.962918 13689 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:00.962982 15483 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:00.964092 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:00.965422 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.965732 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:00.967098 15552 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:00.967003 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:00.968050 13689 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:00.928311 15453 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:57:00.973757 15452 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:00.974711 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:00.983457 15593 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:00.986038 15531 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:57:00.986934 14239 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:57:00.952646 15424 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:00.987540 13613 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:00.990267 13690 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
W20260709 12:57:00.990267 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 105386us
I20260709 12:57:00.996557 13689 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:00.989771 15424 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:57:00.999636 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:00.973726 15563 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.011982 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.021080 13690 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
W20260709 12:57:01.016640 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.028020 15516 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.029376 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.006064 15558 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:57:01.038086 13690 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:01.041657 13690 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:57:00.974121 15584 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.043808 15511 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:01.032313 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:01.032851 13686 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:01.003780 15527 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.031351 13687 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:57:01.049155 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.049790 13689 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.045389 13582 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:01.063005 13689 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:01.046849 15428 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:57:01.009930 14242 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 355) took 3491 ms (client timeout 1981 ms). Trace:
I20260709 12:57:01.024154 15595 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.005825 15574 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.054780 15558 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:01.047168 15511 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:01.051754 15573 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.057813 15506 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:01.078204 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08000387e0 after lost signal to thread 13475
I20260709 12:57:01.066905 15565 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:01.072285 15558 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.071516 15428 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:01.075858 15506 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.070137 15384 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.073711 15552 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:57:01.073915 15417 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:57:01.086133 15565 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:01.098742 15558 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:01.095513 15516 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:01.078542 15573 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.099649 15506 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:01.101877 14237 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: []
W20260709 12:57:01.069842 14242 rpcz_store.cc:269] 0709 12:56:57.518965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.524684 (+  5719us) service_pool.cc:224] Handling call
0709 12:57:01.009588 (+3484904us) spinlock_profiling.cc:243] Waited 3.46 s on lock 0x7b54004c2a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.009822 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7955622656}
I20260709 12:57:01.108898 15428 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.101498 15511 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:01.112757 15595 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:01.112841 15565 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.119426 15599 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:01.118119 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:01.120237 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:01.121829 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:01.122582 14245 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.129686 14211 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:01.131347 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:01.137609 14211 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:01.139470 13541 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:57:01.135069 15472 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.140108 15472 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.140272 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:01.140799 15505 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:57:01.141399 15505 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:01.143193 15472 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:01.116695 15616 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:01.114702 15584 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:01.147608 13658 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:01.148809 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.152735 15603 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:57:01.152908 15193 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:01.153169 15599 raft_consensus.cc:2764] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:57:01.143594 15505 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.152298 15280 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:01.154908 14245 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:01.155107 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.152776 15331 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.158855 15331 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.160058 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:01.155434 15603 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:01.161458 15603 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.146908 15616 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:01.162487 15331 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:01.153028 15552 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:01.162865 15603 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:57:01.140796 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.136341 15471 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:01.157475 15482 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:57:01.160079 15593 raft_consensus.cc:2764] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:57:01.163375 15552 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.168812 15384 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.169816 15471 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:01.164959 15505 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:01.170433 15471 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.175765 15574 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:01.188118 13688 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 204) took 3333 ms (client timeout 2144 ms). Trace:
W20260709 12:57:01.188855 13688 rpcz_store.cc:269] 0709 12:56:57.854204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.856382 (+  2178us) service_pool.cc:224] Handling call
0709 12:57:01.187769 (+3331387us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b5400122fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.187994 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4716780800}
I20260709 12:57:01.192342 15563 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.192876 13684 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:57:01.197227 15331 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:01.198428 15506 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:01.204979 15614 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 12:57:01.206110 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08000346c0 after lost signal to thread 13489
W20260709 12:57:01.215989 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.206664 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.209003 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:01.176889 15511 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.163085 15616 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.216807 15483 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:01.233668 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.212738 15518 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:01.280242 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 288702us
I20260709 12:57:01.219662 15417 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:01.288424 15607 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:57:01.290119 15506 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.297719 15471 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.298530 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1137) took 1711 ms. Trace:
I20260709 12:57:01.278966 15428 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:01.280743 15483 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.303565 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:01.285739 15612 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
I20260709 12:57:01.311408 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:01.217180 15461 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:57:01.284276 15563 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.279109 15552 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:01.313851 13509 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:01.308823 13693 rpcz_store.cc:276] 0709 12:56:59.586759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:59.590955 (+  4196us) service_pool.cc:224] Handling call
0709 12:56:59.732044 (+141089us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:00.239799 (+507755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:00.239831 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:00.240323 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:57:00.381404 (+141081us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:00.381457 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:00.381546 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:00.743304 (+361758us) raft_consensus.cc:1685] finished
0709 12:57:00.743323 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:01.298042 (+554719us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002968d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.298438 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":507358,"raft.queue_time_us":510747,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":323415296,"thread_start_us":140631,"threads_started":1,"wal-append.queue_time_us":178823}
I20260709 12:57:01.308260 15417 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.325240 15563 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.333465 15471 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.334228 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.334775 15563 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:01.335472 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.336412 15573 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.336813 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:01.314448 15613 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 12:57:01.312566 15506 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.314615 15487 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:01.317332 15384 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.326710 15616 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.303421 15518 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.326354 15561 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:57:01.360544 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.361466 13693 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:01.344527 14242 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:01.365741 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 51974.6us
I20260709 12:57:01.342615 15417 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.346058 15487 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.370198 15487 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.371799 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.373186 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.374290 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.375082 13534 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:01.378173 15076 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:01.380020 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.381237 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:01.381517 15483 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:01.382246 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:01.383051 15575 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.932s	user 0.006s	sys 0.000s
I20260709 12:57:01.388234 13690 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:01.358989 15428 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:01.389410 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.389344 15428 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.390723 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.391552 13686 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:01.395364 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:01.396937 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:01.407909 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:01.358505 15552 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.375816 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:01.364535 15561 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:01.415845 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:01.416354 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.416651 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.352974 15506 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.409230 15563 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:01.428853 13693 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:01.420742 15518 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.421396 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.354434 15424 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.429909 13691 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:01.426280 13686 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:01.391774 15428 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.404181 15483 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.434106 15594 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:01.434281 15483 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.435184 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:01.436245 15531 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:01.436391 15563 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:01.436678 15531 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.437147 15531 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:01.437434 15563 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:01.440748 13611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2340) took 1683 ms (client timeout 2148 ms). Trace:
W20260709 12:57:01.441165 13611 rpcz_store.cc:269] 0709 12:56:59.757535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:59.766096 (+  8561us) service_pool.cc:224] Handling call
0709 12:57:01.440537 (+1674441us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b54001298d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.440685 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103171840}
W20260709 12:57:01.442613 13539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1079) took 2400 ms (client timeout 1897 ms). Trace:
W20260709 12:57:01.443032 13539 rpcz_store.cc:269] 0709 12:56:59.042487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:59.042614 (+   127us) service_pool.cc:224] Handling call
0709 12:57:01.442351 (+2399737us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54000b86d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.442457 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1680268032}
I20260709 12:57:01.433643 13534 raft_consensus.cc:2393] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:01.432771 15373 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:57:01.440896 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.434710 14238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 434) took 2372 ms. Trace:
I20260709 12:57:01.425559 15552 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.455416 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.456140 13543 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:01.454931 14238 rpcz_store.cc:276] 0709 12:56:59.062487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:59.064469 (+  1982us) service_pool.cc:224] Handling call
0709 12:56:59.065423 (+   954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:00.644795 (+1579372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:01.434448 (+789653us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54006009d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:01.434623 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3560110336}
I20260709 12:57:01.457062 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.457166 15552 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.436259 15506 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.458380 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.460645 13692 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:57:01.461040 13692 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:01.461809 13692 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.460204 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.452726 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:01.463016 13534 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.463634 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.464190 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2347) took 1069 ms. Trace:
I20260709 12:57:01.464564 15076 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:01.464562 13613 rpcz_store.cc:276] 0709 12:57:00.394802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:00.418267 (+ 23465us) service_pool.cc:224] Handling call
0709 12:57:01.464029 (+1045762us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b540030a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.464134 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278577024}
W20260709 12:57:01.465773 13537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1084) took 2421 ms (client timeout 3096 ms). Trace:
I20260709 12:57:01.426123 15417 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 12:57:01.466149 13537 rpcz_store.cc:269] 0709 12:56:59.044041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:59.786630 (+742589us) service_pool.cc:224] Handling call
0709 12:57:01.465629 (+1678999us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b540006e0d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.465721 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150088960}
I20260709 12:57:01.467286 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.431011 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:01.463160 15595 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:01.468272 13536 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.470539 13692 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:01.477035 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:01.477824 15424 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.478035 15573 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:01.480036 15461 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:01.480598 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.485245 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.485486 15384 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:01.461903 15483 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:01.458518 15561 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:01.490630 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.487310 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:01.491571 13472 debug-util.cc:398] Leaking SignalData structure 0x7b080034c8a0 after lost signal to thread 13568
W20260709 12:57:01.493033 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:01.493357 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08002ba6a0 after lost signal to thread 13569
I20260709 12:57:01.486778 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:01.494946 13472 debug-util.cc:398] Leaking SignalData structure 0x7b0800378a20 after lost signal to thread 13643
W20260709 12:57:01.497344 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08003ca940 after lost signal to thread 13644
I20260709 12:57:01.408316 15505 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:57:01.480563 15461 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.498745 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:01.510056 13537 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:01.515110 13536 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:01.518487 14211 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:01.516954 15595 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:57:01.520881 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.521799 13536 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:01.526917 13692 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:57:01.530484 13684 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:57:01.530946 13684 pending_rounds.cc:85] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 1
I20260709 12:57:01.531226 13684 pending_rounds.cc:107] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:57:01.531498 13684 raft_consensus.cc:2889] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:01.532260 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.534029 13686 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:57:01.424880 15622 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 12:57:01.536154 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.543483 13693 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
W20260709 12:57:01.538085 15592 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:57:01.508587 15603 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:01.496667 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1265) took 2523 ms. Trace:
I20260709 12:57:01.508435 15461 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.522904 15076 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.551201 13691 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:01.508328 14240 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:01.537081 15331 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:01.555308 13537 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:01.493378 15558 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:01.568996 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 161747us
I20260709 12:57:01.569269 13684 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:01.498617 15487 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.555871 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:01.578969 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 168228us
I20260709 12:57:01.553941 15573 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:01.583133 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.584172 13691 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 6. Ops: [4.3-4.3]
W20260709 12:57:01.584774 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.572234 15471 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:01.585695 13472 debug-util.cc:398] Leaking SignalData structure 0x7b080038edc0 after lost signal to thread 13717
I20260709 12:57:01.588244 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:01.588927 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08003d4bc0 after lost signal to thread 14270
I20260709 12:57:01.586123 13509 leader_election.cc:400] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
I20260709 12:57:01.589422 13509 leader_election.cc:403] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:01.586784 13543 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:01.565752 15331 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:01.590068 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:01.597440 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.598251 13693 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:01.598455 13582 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:01.599565 13685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2498) took 2613 ms (client timeout 2569 ms). Trace:
W20260709 12:57:01.599785 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 393214us
W20260709 12:57:01.600119 13685 rpcz_store.cc:269] 0709 12:56:58.986180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.599357 (+2613177us) service_pool.cc:211] Skipping call since client already timed out
0709 12:57:01.599501 (+   144us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:57:01.601336 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.602339 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.605321 13691 raft_consensus.cc:2368] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
I20260709 12:57:01.554425 15076 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:01.609851 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.611325 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.612773 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.583241 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.614064 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.614538 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.614701 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.615208 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.547703 13612 rpcz_store.cc:276] 0709 12:56:58.973478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:58.973605 (+   127us) service_pool.cc:224] Handling call
0709 12:56:59.591554 (+617949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:59.592899 (+  1345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:01.496407 (+1903508us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b5400309dd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.496581 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420111616}
I20260709 12:57:01.596906 15573 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:01.617139 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.617515 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.606211 15622 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
I20260709 12:57:01.622145 13693 raft_consensus.cc:2410] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:57:01.622218 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:01.607028 15424 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:01.602972 15625 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:01.610265 15417 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:01.596922 15594 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:57:01.614252 15506 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:01.619220 15552 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.632002 13691 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:57:01.601899 15471 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.629864 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.638679 13685 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:57:01.629220 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:01.640146 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.634729 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.646185 13693 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:01.647043 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:01.626504 15625 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.634150 15532 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.634759 13535 raft_consensus.cc:2410] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:01.661065 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1164) took 2772 ms. Trace:
I20260709 12:57:01.665311 13540 rpcz_store.cc:276] 0709 12:56:58.888427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.469876 (+2581449us) service_pool.cc:224] Handling call
0709 12:57:01.470299 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:01.470807 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:01.470849 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:01.471482 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:57:01.610775 (+139293us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:01.610811 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:01.610875 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:01.660848 (+ 49973us) raft_consensus.cc:1685] finished
0709 12:57:01.660869 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:01.661010 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":138775,"threads_started":1,"wal-append.queue_time_us":185943}
I20260709 12:57:01.674290 13540 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:57:01.681113 13685 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:01.682363 13540 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:57:01.685897 13540 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:01.689168 13540 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:01.660547 15622 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:01.691802 13540 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:01.660387 13535 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:57:01.635321 15632 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.632507 15629 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:01.657800 15625 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.695706 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.698729 15556 raft_consensus.cc:3055] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:01.704787 15556 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:01.705169 15573 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.705709 15487 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.703605 13685 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:57:01.677667 15506 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:01.695976 15637 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:57:01.696274 15622 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.700292 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:01.707856 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.709208 13540 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:57:01.687139 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 64999us
W20260709 12:57:01.693195 15628 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:57:01.712316 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 122291us
I20260709 12:57:01.629217 15331 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.720006 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:01.716079 15442 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:57:01.725298 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:01.722975 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.741930 13613 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:01.742002 13540 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:57:01.743144 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:01.701586 15471 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:01.759205 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.719801 15622 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:01.693332 15531 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.705935 15556 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.723591 15384 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:01.734797 15594 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:01.736269 15558 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.729023 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:01.722954 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:01.797420 15594 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.799938 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.811012 13689 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:57:01.803184 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:01.812386 13618 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.804530 13685 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:01.757292 15506 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:01.807899 15428 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:01.722697 15527 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:57:01.798866 15632 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:01.780629 15440 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:57:01.790788 15331 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.816906 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.843102 13693 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:01.790437 15384 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:01.807641 15076 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:01.720889 15563 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:57:01.851336 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.816778 13612 raft_consensus.cc:2376] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4. Re-sending same reply.
I20260709 12:57:01.838253 13686 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:57:01.841542 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.875183 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:01.810118 13610 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:01.855808 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:01.805763 15638 consensus_queue.cc:1059] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:57:01.871912 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:01.885586 13612 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:01.890664 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:01.890851 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 364) took 1133 ms. Trace:
I20260709 12:57:01.853999 13685 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:57:01.892683 15531 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.841055 15440 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:01.841683 15428 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:01.856006 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.922821 13693 raft_consensus.cc:2376] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3. Re-sending same reply.
I20260709 12:57:01.897101 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:01.899226 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.891537 14237 rpcz_store.cc:276] 0709 12:57:00.757114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:00.772627 (+ 15513us) service_pool.cc:224] Handling call
0709 12:57:00.797424 (+ 24797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:01.119069 (+321645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:01.890589 (+771520us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540011f154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:01.890781 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664093952}
I20260709 12:57:01.901710 15637 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:01.901407 13534 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:01.854825 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.904112 15506 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.971741 14241 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:01.908730 15599 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:01.913575 15642 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:01.866441 13613 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:57:01.982904 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:01.921552 13618 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:01.891552 13612 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:01.984432 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:01.985450 13612 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:01.985998 13618 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:01.924594 13539 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
W20260709 12:57:01.939953 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 292984us
I20260709 12:57:01.988535 13539 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:01.988842 15603 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:01.989845 13539 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:01.940649 15512 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:57:01.948665 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:01.922896 15424 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:01.949929 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:02.005229 13693 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:01.951226 13685 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:01.952724 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:02.007823 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.902407 15384 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:01.904140 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:01.965301 15594 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:01.980751 13688 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:02.025836 15574 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:01.965843 15617 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.028417 15617 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:01.969842 14240 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 358) took 4106 ms (client timeout 2347 ms). Trace:
I20260709 12:57:01.971776 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:01.955132 13541 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:01.972889 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:02.033239 13693 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:01.922924 15440 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:01.978920 15643 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:02.003522 15552 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:01.992734 13509 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:01.996234 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:01.950588 15625 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:01.996723 15512 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:01.997139 13539 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:01.949618 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:01.997380 15599 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:02.000885 15470 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:57:02.001819 15532 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:01.987496 13612 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:02.002787 15534 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:02.012058 15561 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:02.012342 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:02.011680 15424 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:01.981113 15616 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.000s
I20260709 12:57:01.974351 15637 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:02.024626 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 120607us
I20260709 12:57:02.033640 13616 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 1.
I20260709 12:57:02.008723 13689 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:01.988857 13658 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:02.140410 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 151618us
W20260709 12:57:02.045482 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 73795.3us
W20260709 12:57:02.170243 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.045982 13693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:57:01.989881 15603 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:57:02.049230 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.032867 13535 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:57:02.056628 13613 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:02.024695 15123 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:02.207000 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 58899.1us
I20260709 12:57:02.048241 15518 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.064723 13615 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:02.034552 15642 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:02.054486 15384 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:02.059634 15512 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:57:02.089466 15556 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:02.090847 13438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49892 (request call id 317) took 8934 ms. Trace:
I20260709 12:57:02.090307 15600 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:57:02.097767 15463 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:02.100510 13687 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:02.100528 15565 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:02.106278 13537 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:57:02.102658 13441 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 3, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:57:02.110867 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:02.112377 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.112903 15539 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.115865 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 714) took 2727 ms. Trace:
W20260709 12:57:02.032155 14240 rpcz_store.cc:269] 0709 12:56:57.863970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.868679 (+  4709us) service_pool.cc:224] Handling call
0709 12:57:01.969537 (+4100858us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b54006d0954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:01.969738 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5978837376}
I20260709 12:57:02.127377 13685 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:02.048038 15640 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:02.129606 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 136937us
I20260709 12:57:02.057878 15428 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.206113 15487 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.159528 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:02.053813 15413 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:02.051146 15483 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:02.296567 15483 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:02.299206 15483 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:02.228844 13684 raft_consensus.cc:2410] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:02.084682 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1069) took 5400 ms. Trace:
I20260709 12:57:02.300082 13542 rpcz_store.cc:276] 0709 12:56:56.684478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:57.600097 (+915619us) service_pool.cc:224] Handling call
0709 12:56:57.629988 (+ 29891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:56:57.681134 (+ 51146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:02.084356 (+4403222us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b54002841d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:02.084565 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67948288}
I20260709 12:57:02.079205 13690 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:57:02.254456 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 534556us
W20260709 12:57:02.254918 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.255892 13619 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:02.305781 13619 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:57:02.258234 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.226975 15433 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:57:02.057934 15461 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.330880 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:02.332686 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:02.255257 15600 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:02.218918 15453 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:57:02.368242 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:02.204810 13693 raft_consensus.cc:606] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:02.369706 13693 raft_consensus.cc:614] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:57:02.288743 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.206282 15603 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:02.282933 15563 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.256667 15123 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:57:02.293941 15537 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:02.378394 13689 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:02.286718 15531 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.250162 15435 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 12:57:02.294600 15584 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:02.293545 15539 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.339565 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:02.305083 15640 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.393498 15461 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:02.393697 15637 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:02.251283 15518 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.303426 15642 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:02.395144 15625 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:02.396728 15453 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:02.398308 13693 pending_rounds.cc:85] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 5
I20260709 12:57:02.406040 15563 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:02.411857 15573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.034s	sys 0.059s
I20260709 12:57:02.282501 15527 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
W20260709 12:57:02.380805 15575 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:57:02.226692 15617 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:02.427454 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 94858.8us
I20260709 12:57:02.427819 15589 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.432614 13689 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:02.328449 15487 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.402200 15655 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:02.460561 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 301142us
I20260709 12:57:02.285913 14241 rpcz_store.cc:276] 0709 12:56:59.388864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:59.393356 (+  4492us) service_pool.cc:224] Handling call
0709 12:57:01.971327 (+2577971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:01.975830 (+  4503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:02.115591 (+139761us) spinlock_profiling.cc:243] Waited 2.57 s on lock 0x7b54006d0950. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:02.115789 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5913460096}
I20260709 12:57:02.365917 15413 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:02.504638 13616 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:02.517297 13689 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:57:02.511397 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 325814us
I20260709 12:57:02.440208 15531 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.439494 15483 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:57:02.381623 15647 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 12:57:02.482012 13693 pending_rounds.cc:107] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:57:02.544608 13693 raft_consensus.cc:2889] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:02.481519 15482 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:57:02.503963 15573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.037s	sys 0.063s
I20260709 12:57:02.411651 15537 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.514662 15655 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:02.518126 15312 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:57:02.580478 15655 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:02.580969 15661 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:57:02.578917 15482 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:02.525533 15652 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.869930 13441 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 2, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 2 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:57:02.875407 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.015s	sys 0.015s
W20260709 12:57:02.976419 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.017s	sys 0.016s
W20260709 12:57:02.977106 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 1.00069e+06us
W20260709 12:57:02.877266 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 730286us
I20260709 12:57:02.279201 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:02.981828 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1185) took 1823 ms. Trace:
I20260709 12:57:02.880550 13541 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:02.886910 13534 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:02.982442 13535 rpcz_store.cc:276] 0709 12:57:01.158316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.700428 (+542112us) service_pool.cc:224] Handling call
0709 12:57:01.977314 (+276886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:02.250132 (+272818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:02.981258 (+731126us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54003cb250. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:02.981526 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622442240}
I20260709 12:57:02.989833 13534 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [3.6-4.7]
W20260709 12:57:02.887712 13540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.143s	user 0.031s	sys 0.055s
I20260709 12:57:02.503059 13584 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:02.463132 13610 consensus_meta.cc:287] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.543s	user 0.006s	sys 0.006s
I20260709 12:57:02.529968 13612 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:02.515029 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:02.893778 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.251251 13438 rpcz_store.cc:276] 0709 12:56:53.157155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:53.158086 (+   931us) service_pool.cc:224] Handling call
0709 12:57:02.090773 (+8932687us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:56:56.629142 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:56:56.714288 (+ 85146us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:56:56.714353 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:56:56.714372 (+    19us) tablet.cc:662] Decoding operations
0709 12:56:56.715272 (+   900us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:56:56.715358 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:56:56.715387 (+    29us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:56:56.716188 (+   801us) tablet.cc:701] Row locks acquired
0709 12:56:56.716219 (+    31us) write_op.cc:260] PREPARE: finished
0709 12:56:56.716411 (+   192us) write_op.cc:270] Start()
0709 12:56:56.716542 (+   131us) write_op.cc:276] Timestamp: P: 1783601816716344 usec, L: 0
0709 12:56:56.716562 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:56:56.810752 (+ 94190us) log.cc:844] Serialized 3209 byte log entry
0709 12:56:57.360498 (+549746us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:56:57.544350 (+183852us) op_driver.cc:464] REPLICATION: finished
0709 12:56:57.715630 (+171280us) write_op.cc:301] APPLY: starting
0709 12:56:57.715711 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:56:57.715904 (+   193us) tablet.cc:1370] finished BulkCheckPresence
0709 12:56:57.715946 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:56:57.730616 (+ 14670us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:56:57.730912 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:56:57.730969 (+    57us) write_op.cc:312] APPLY: finished
0709 12:56:57.731297 (+   328us) log.cc:844] Serialized 65 byte log entry
0709 12:56:57.731546 (+   249us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:57.731768 (+   222us) write_op.cc:454] Released schema lock
0709 12:56:57.731851 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:56:57.763559 (+ 31708us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:56:57.767835 (+  4276us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12999,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":171055,"apply.run_cpu_time_us":3548,"apply.run_wall_time_us":42345,"num_ops":8,"prepare.queue_time_us":657343,"prepare.run_cpu_time_us":39082,"prepare.run_wall_time_us":387919,"raft.queue_time_us":161268,"raft.run_cpu_time_us":24618,"raft.run_wall_time_us":167270,"replication_time_us":827646,"spinlock_wait_cycles":82397056,"thread_start_us":890392,"threads_started":4,"wal-append.queue_time_us":276885,"wal-append.run_cpu_time_us":39771,"wal-append.run_wall_time_us":4247128}]]}
I20260709 12:57:02.894379 13617 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:02.895032 13618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.009s	sys 0.003s
W20260709 12:57:02.894965 13686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.029s	sys 0.041s
W20260709 12:57:02.729825 13688 consensus_meta.cc:287] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.704s	user 0.008s	sys 0.008s
I20260709 12:57:03.005069 15652 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:02.896224 13614 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.001s	sys 0.013s
W20260709 12:57:02.896144 13687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.018s
I20260709 12:57:02.900875 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 294) took 1083 ms. Trace:
W20260709 12:57:02.902215 13692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.019s	sys 0.013s
W20260709 12:57:02.905000 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:02.529528 13659 leader_election.cc:336] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.551s (SENT)
W20260709 12:57:02.764153 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:03.018458 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 503360us
I20260709 12:57:03.018599 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:03.019258 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:03.020325 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.020537 13619 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:03.020732 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:02.726809 15123 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:03.024017 13614 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.001s	sys 0.015s
I20260709 12:57:03.024070 15629 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:03.024705 15123 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.992180 13610 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:03.026422 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2354) took 1298 ms. Trace:
I20260709 12:57:02.922991 15373 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.602692 15413 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.040254 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.041837 15671 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:02.697899 15453 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.042757 15671 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.691905 15482 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.045529 15482 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.045751 15671 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.018605 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:02.691995 15506 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:03.048380 15506 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:03.035426 15629 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:03.049427 15629 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.050073 15506 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.051643 15506 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:02.787002 15384 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.018s
I20260709 12:57:03.026975 13610 rpcz_store.cc:276] 0709 12:57:01.728433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.728608 (+   175us) service_pool.cc:224] Handling call
0709 12:57:03.026243 (+1297635us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:02.549397 15483 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:03.061457 15483 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:57:02.782585 15518 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:02.778620 15531 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:02.734238 15537 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:02.781590 15539 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:02.761401 15561 consensus_meta.cc:287] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.629s	user 0.013s	sys 0.000s
I20260709 12:57:02.788790 15563 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:02.808350 15574 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.782s	user 0.015s	sys 0.004s
I20260709 12:57:03.107669 13543 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:02.752892 15582 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:03.108619 15574 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:03.109418 15582 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:03.110174 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 607040us
I20260709 12:57:02.517720 15589 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:02.607769 15599 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:02.527321 15600 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.506031 15603 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.125523 15603 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:03.126268 15556 consensus_meta.cc:287] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.852s	user 0.010s	sys 0.015s
W20260709 12:57:02.812690 15622 consensus_meta.cc:287] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.019s	user 0.015s	sys 0.007s
I20260709 12:57:03.127908 15123 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.129093 15123 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.129213 15622 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.129828 15123 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:57:02.533179 15627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.015s	sys 0.012s
W20260709 12:57:03.132885 15627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.396s	user 0.024s	sys 0.016s
I20260709 12:57:02.779567 15640 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:02.512634 15653 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:03.136574 15653 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:03.137538 15440 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.063s	user 0.013s	sys 0.006s
I20260709 12:57:03.138020 14241 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:03.138168 15440 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.138526 15653 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:02.606274 15657 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:57:03.140059 15642 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.140957 15642 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.143471 14239 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:03.144070 15574 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:02.821732 15655 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:02.911566 13684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2526) took 1309 ms (client timeout 1632 ms). Trace:
W20260709 12:57:03.145643 13684 rpcz_store.cc:269] 0709 12:57:01.601893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.602072 (+   179us) service_pool.cc:224] Handling call
0709 12:57:02.910924 (+1308852us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54003a0a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:02.911251 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411218816}
I20260709 12:57:02.964035 15364 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:02.968045 15592 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:02.991375 13540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.246s	user 0.031s	sys 0.055s
W20260709 12:57:03.001190 13618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.010s	sys 0.003s
W20260709 12:57:02.990643 13542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.012s	sys 0.013s
I20260709 12:57:03.004632 15532 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.005681 13688 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:57:02.556668 13689 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
W20260709 12:57:03.152987 13542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.012s	sys 0.014s
I20260709 12:57:03.153177 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2351) took 1426 ms. Trace:
I20260709 12:57:03.153728 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 237) took 1006 ms. Trace:
W20260709 12:57:03.008225 13686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.029s	sys 0.042s
I20260709 12:57:03.154663 15600 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:57:03.016937 13687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.012s	sys 0.019s
I20260709 12:57:03.156176 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2535) took 1358 ms. Trace:
W20260709 12:57:03.017557 13692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.019s	sys 0.013s
I20260709 12:57:03.156684 13688 rpcz_store.cc:276] 0709 12:57:01.798296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.809051 (+ 10755us) service_pool.cc:224] Handling call
0709 12:57:03.156081 (+1347030us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:03.017776 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.158308 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2511) took 1629 ms. Trace:
I20260709 12:57:03.025313 15472 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:57:03.159641 15472 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.025573 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2356) took 1291 ms. Trace:
I20260709 12:57:02.920634 15356 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:03.158710 13692 rpcz_store.cc:276] 0709 12:57:01.528613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.538327 (+  9714us) service_pool.cc:224] Handling call
0709 12:57:01.538617 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:01.548176 (+  9559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:01.548216 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:01.887084 (+338868us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b1800166000. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa03268658f 00007fa02b6c5571 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd
0709 12:57:02.015648 (+128564us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b2c0006a7c8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:57:02.035141 (+ 19493us) log.cc:844] Serialized 26 byte log entry
0709 12:57:03.157321 (+1122180us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:03.157409 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.157511 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.157680 (+   169us) raft_consensus.cc:1685] finished
0709 12:57:03.157695 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.158134 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9526,"raft.queue_time_us":9652,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":13235328,"thread_start_us":1109028,"threads_started":1,"wal-append.queue_time_us":783742}
I20260709 12:57:03.160284 15592 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:02.730167 15461 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:03.156684 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 234) took 1013 ms. Trace:
I20260709 12:57:03.162786 15461 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.036744 15647 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:57:03.163604 15655 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.052373 15373 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.164110 15647 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:03.164678 15647 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:03.030587 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:03.024634 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.023229 13441 catalog_manager.cc:5697] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:03.154255 13689 rpcz_store.cc:276] 0709 12:57:02.147516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.533626 (+386110us) service_pool.cc:224] Handling call
0709 12:57:02.556434 (+ 22808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:03.153110 (+596676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.153460 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:03.063769 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1173) took 1097 ms. Trace:
I20260709 12:57:03.066293 13535 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 2. Current term is 3. Ops: []
I20260709 12:57:03.086639 15455 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:03.087868 15384 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:03.089097 14237 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:03.169554 13693 rpcz_store.cc:276] 0709 12:57:01.965632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.397679 (+432047us) service_pool.cc:224] Handling call
0709 12:57:02.398089 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:02.545430 (+147341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:02.545477 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:02.546448 (+   971us) log.cc:844] Serialized 26 byte log entry
0709 12:57:02.991330 (+444882us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:02.993205 (+  1875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:02.999121 (+  5916us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.057753 (+ 58632us) raft_consensus.cc:1685] finished
0709 12:57:03.057807 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.062379 (+  4572us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":432994,"threads_started":1,"wal-append.queue_time_us":507700}
I20260709 12:57:03.092278 15537 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:57:03.112880 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 121893us
I20260709 12:57:03.112304 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.123087 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:03.147480 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.084342 15671 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:57:03.152968 15364 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.008590 13616 rpcz_store.cc:276] 0709 12:57:01.817108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.248596 (+431488us) service_pool.cc:224] Handling call
0709 12:57:02.286291 (+ 37695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:02.524184 (+237893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:02.900051 (+375867us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540013a6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:02.900557 (+   506us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501114752}
I20260709 12:57:03.157347 15532 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.161458 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 342) took 1965 ms. Trace:
I20260709 12:57:03.162581 13687 rpcz_store.cc:276] 0709 12:57:02.143517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.387538 (+244021us) service_pool.cc:224] Handling call
0709 12:57:02.387874 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:02.388207 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:02.388235 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:02.388696 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:57:03.156125 (+767429us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:03.156249 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.156373 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.156432 (+    59us) raft_consensus.cc:1685] finished
0709 12:57:03.156442 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.156635 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2145,"raft.queue_time_us":11526,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":18735,"thread_start_us":766368,"threads_started":1,"wal-append.queue_time_us":443700}
I20260709 12:57:03.030629 15076 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.037706 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:36690
I20260709 12:57:02.783775 15487 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.195672 15534 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:03.196219 15534 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:03.196255 15076 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.197337 15534 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.199170 13690 raft_consensus.cc:606] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.199645 13690 raft_consensus.cc:614] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:57:03.200327 15463 consensus_meta.cc:287] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.900s	user 0.009s	sys 0.009s
I20260709 12:57:03.201553 15463 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:03.201614 15592 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.202020 15463 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.202589 15463 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.202706 15364 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.203267 13538 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:03.204298 15440 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.117224 15629 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:03.066553 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.205839 13610 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:03.209017 15529 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.209682 15529 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.143643 15642 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.091329 13539 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:57:03.177143 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.178381 13613 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:03.179283 13615 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:57:03.160290 15599 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:57:03.160406 15472 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:03.180289 15631 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:03.181046 13536 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:03.182318 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:03.182621 15662 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.662s	user 0.017s	sys 0.020s
W20260709 12:57:03.183350 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 653814us
I20260709 12:57:03.184043 15470 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.184238 13537 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.184048 15531 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
W20260709 12:57:03.185258 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 73677us
I20260709 12:57:03.185757 15373 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.185456 15455 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.185954 15622 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.186790 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.183571 15565 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:03.192781 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 168500us
I20260709 12:57:03.204964 15589 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.205164 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:57:03.206205 13538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1080) took 4163 ms (client timeout 1785 ms). Trace:
I20260709 12:57:03.210007 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:03.148914 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 481636us
I20260709 12:57:03.156215 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2357) took 1355 ms. Trace:
I20260709 12:57:03.167083 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1168) took 1258 ms. Trace:
I20260709 12:57:03.247884 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2529) took 1580 ms. Trace:
I20260709 12:57:03.248380 13691 rpcz_store.cc:276] 0709 12:57:01.661408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.661996 (+   588us) service_pool.cc:224] Handling call
0709 12:57:01.684786 (+ 22790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:01.941340 (+256554us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:57:01.941406 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:01.942030 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:57:02.288418 (+346388us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:02.288458 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:02.288522 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.164451 (+875929us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:03.241776 (+ 77325us) raft_consensus.cc:1685] finished
0709 12:57:03.241814 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.242177 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255828,"raft.queue_time_us":255950,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":6384896,"thread_start_us":345839,"threads_started":1,"wal-append.queue_time_us":519682}
I20260709 12:57:03.247658 13618 rpcz_store.cc:276] 0709 12:57:01.800481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.986830 (+186349us) service_pool.cc:224] Handling call
0709 12:57:01.987092 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:01.987320 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:01.994804 (+  7484us) log.cc:844] Serialized 14 byte log entry
0709 12:57:03.155339 (+1160535us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:03.155454 (+   115us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:03.155550 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.155679 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.156100 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6635,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":9464,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":120,"thread_start_us":1155004,"threads_started":1,"wal-append.queue_time_us":833187}
I20260709 12:57:03.251981 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.214953 15461 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.202069 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.215230 15640 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:03.190471 13540 rpcz_store.cc:276] 0709 12:57:01.195824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.743075 (+547251us) service_pool.cc:224] Handling call
0709 12:57:01.743512 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:01.744020 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:01.744060 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:01.744614 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:57:03.160890 (+1416276us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:03.160973 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.161092 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.161184 (+    92us) raft_consensus.cc:1685] finished
0709 12:57:03.161202 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.161384 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":15694,"thread_start_us":1405123,"threads_started":1,"wal-append.queue_time_us":780479}
I20260709 12:57:03.257231 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 284) took 1484 ms. Trace:
I20260709 12:57:03.211541 15629 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.160822 13614 rpcz_store.cc:276] 0709 12:57:01.734354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.734875 (+   521us) service_pool.cc:224] Handling call
0709 12:57:01.735206 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:02.055407 (+320201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:02.055444 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:02.063253 (+  7809us) log.cc:844] Serialized 26 byte log entry
0709 12:57:03.024938 (+961685us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:03.025043 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.025191 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.025235 (+    44us) raft_consensus.cc:1685] finished
0709 12:57:03.025248 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.025484 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44257,"raft.queue_time_us":68631,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":10700,"thread_start_us":1031211,"threads_started":2,"wal-append.queue_time_us":598310}
I20260709 12:57:03.187639 15584 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.265326 15584 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.257756 13617 rpcz_store.cc:276] 0709 12:57:01.769433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.769632 (+   199us) service_pool.cc:224] Handling call
0709 12:57:01.782182 (+ 12550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:02.998839 (+1216657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.235240 (+236401us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540020a154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:03.254150 (+ 18910us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2556163328}
I20260709 12:57:03.153769 13611 rpcz_store.cc:276] 0709 12:57:01.726226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.726472 (+   246us) service_pool.cc:224] Handling call
0709 12:57:01.726806 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:01.781116 (+ 54310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:01.781168 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:01.804557 (+ 23389us) log.cc:844] Serialized 26 byte log entry
0709 12:57:02.220390 (+415833us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:02.220445 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:02.220535 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.152491 (+931956us) raft_consensus.cc:1685] finished
0709 12:57:03.152542 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.152940 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35370,"raft.queue_time_us":59866,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":11014,"thread_start_us":380637,"threads_started":1,"wal-append.queue_time_us":683482}
I20260709 12:57:03.199782 15076 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.222080 15646 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:03.268576 15646 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:03.222275 13441 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 3, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:57:03.226671 15512 consensus_meta.cc:287] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.934s	user 0.006s	sys 0.013s
I20260709 12:57:03.190434 15582 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:03.230101 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 54487.3us
I20260709 12:57:03.223711 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 296) took 1320 ms. Trace:
I20260709 12:57:03.234341 15470 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.240120 15487 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.279618 15640 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:03.279881 15540 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.280054 15487 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.280396 15629 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.241212 15529 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.241299 15531 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:03.231364 13610 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:57:03.213056 15123 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:03.247292 13538 rpcz_store.cc:269] 0709 12:56:59.042897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:59.044182 (+  1285us) service_pool.cc:224] Handling call
0709 12:57:03.205884 (+4161702us) spinlock_profiling.cc:243] Waited 4.16 s on lock 0x7b540006ead4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:03.206063 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9562401024}
I20260709 12:57:03.251289 15565 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.263734 13613 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 1. Current term is 2. Ops: []
I20260709 12:57:03.264580 15455 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.246783 15671 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.290352 15671 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.292384 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 353) took 1070 ms. Trace:
I20260709 12:57:03.247737 13686 rpcz_store.cc:276] 0709 12:57:01.897490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.048323 (+150833us) service_pool.cc:224] Handling call
0709 12:57:02.048707 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:02.049103 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:02.049139 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:02.053732 (+  4593us) log.cc:844] Serialized 26 byte log entry
0709 12:57:03.155033 (+1101301us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:03.155148 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.155340 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.155396 (+    56us) raft_consensus.cc:1685] finished
0709 12:57:03.155414 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.155774 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2610,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":10569,"thread_start_us":904677,"threads_started":1,"wal-append.queue_time_us":701521}
I20260709 12:57:03.292775 13541 rpcz_store.cc:276] 0709 12:57:02.222215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.984350 (+762135us) service_pool.cc:224] Handling call
0709 12:57:02.984997 (+   647us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:02.985833 (+   836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:02.985903 (+    70us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:02.987185 (+  1282us) log.cc:844] Serialized 52 byte log entry
0709 12:57:03.084952 (+ 97767us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:03.094033 (+  9081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.100291 (+  6258us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.292091 (+191800us) raft_consensus.cc:1685] finished
0709 12:57:03.292110 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.292301 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9345,"raft.run_cpu_time_us":634,"raft.run_wall_time_us":6808,"thread_start_us":78938,"threads_started":1,"wal-append.queue_time_us":187710}
I20260709 12:57:03.276283 15076 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.271136 13612 rpcz_store.cc:276] 0709 12:57:01.902921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.328114 (+425193us) service_pool.cc:224] Handling call
0709 12:57:02.408509 (+ 80395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:02.992434 (+583925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.223205 (+230771us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b54002fc6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:03.223501 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218670976}
I20260709 12:57:03.295499 15076 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.277158 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.300323 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.297890 15467 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:03.301419 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.298374 15424 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:03.302475 15424 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.302836 15424 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.270977 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.308071 15683 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.308929 15683 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.279067 15593 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.310032 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:03.310438 15593 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.279197 15470 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.311475 15683 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.312251 15467 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:03.312790 15593 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.313419 15593 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:03.280495 15540 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.232870 15631 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:03.316977 15631 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.318547 15631 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.294492 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.278954 15373 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:03.320595 13615 raft_consensus.cc:2410] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:57:03.320827 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.321050 15571 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:03.321290 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.321617 15571 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.302031 15076 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.313349 15461 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:03.317569 15540 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:03.281615 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.277294 15646 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.283958 15531 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:03.298592 15655 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:03.288540 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 78679.6us
I20260709 12:57:03.278007 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1192) took 1383 ms. Trace:
I20260709 12:57:03.323817 13619 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
W20260709 12:57:03.325044 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53513.3us
I20260709 12:57:03.325578 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 348) took 1672 ms. Trace:
I20260709 12:57:03.324550 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.324384 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.328738 13543 rpcz_store.cc:276] 0709 12:57:01.894952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.080604 (+185652us) service_pool.cc:224] Handling call
0709 12:57:02.081012 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:03.108629 (+1027617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.277694 (+169065us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000203d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:03.277945 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333615488}
I20260709 12:57:03.327739 13615 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:57:03.384838 13540 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:03.390110 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.280207 15640 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:03.279345 13441 catalog_manager.cc:5697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:03.326386 14212 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.295132 15681 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:03.333817 13539 rpcz_store.cc:276] 0709 12:57:01.653106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.054546 (+401440us) service_pool.cc:224] Handling call
0709 12:57:02.054841 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:03.229533 (+1174692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.325300 (+ 95767us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003b7c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:03.325504 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323331456}
I20260709 12:57:03.401201 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.325031 15487 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.293313 15671 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.324832 13535 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:03.323818 15571 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.324735 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.389626 15582 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.417021 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:03.421376 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 95107.1us
I20260709 12:57:03.326426 15467 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:03.425638 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 243561us
I20260709 12:57:03.424575 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.324180 15652 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.427510 13537 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:03.424332 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:03.326041 15540 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 12:57:03.429080 13540 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:03.286391 15512 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.431295 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.434751 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:03.435184 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 248808us
I20260709 12:57:03.424770 13441 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:03.436859 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.437651 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.439410 13538 raft_consensus.cc:2410] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:03.430846 15671 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:03.444996 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.424151 15532 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.448686 13543 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
W20260709 12:57:03.450385 13659 leader_election.cc:336] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.795s (SENT)
I20260709 12:57:03.450945 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:03.451422 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:57:03.451799 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.452275 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:03.452811 13611 raft_consensus.cc:2393] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:03.453693 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.453931 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:03.447535 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.450946 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.412432 15123 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:57:03.456593 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.428702 15482 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:03.460960 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:03.461198 15482 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.461890 13693 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:03.459383 13619 raft_consensus.cc:2376] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6. Re-sending same reply.
I20260709 12:57:03.464291 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:03.465430 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.466475 13615 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:03.466624 13619 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:03.421109 15463 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.468109 13615 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:03.468222 15627 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:03.468842 15627 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:03.470191 15482 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.413853 15655 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.460520 15076 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:03.470050 15627 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.471509 15463 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.474809 15534 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:03.433100 15531 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:03.476557 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 163474us
I20260709 12:57:03.477012 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.477721 13506 leader_election.cc:400] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4.
I20260709 12:57:03.455088 15671 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.481066 15671 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:03.484014 15674 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:57:03.444530 15640 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.485646 15640 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.487392 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 350) took 1356 ms. Trace:
I20260709 12:57:03.456490 13687 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:03.477239 15531 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.489830 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:03.489893 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.498170 13616 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:03.496296 15467 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.502107 15482 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:03.431645 15653 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:57:03.502635 15512 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.500169 15600 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:57:03.433336 15681 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.506134 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.506628 13618 raft_consensus.cc:2376] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5. Re-sending same reply.
I20260709 12:57:03.496976 13506 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.473762 15571 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:03.510962 15681 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.504660 15540 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:03.512450 13539 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:03.487910 13542 rpcz_store.cc:276] 0709 12:57:02.131109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.301212 (+170103us) service_pool.cc:224] Handling call
0709 12:57:02.301644 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:02.395329 (+ 93685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:02.395494 (+   165us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:02.396132 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 12:57:03.154062 (+757930us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:03.154151 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.154297 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:03.487034 (+332737us) raft_consensus.cc:1685] finished
0709 12:57:03.487062 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:03.487298 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93194,"raft.queue_time_us":94593,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":757156,"threads_started":1,"wal-append.queue_time_us":777326}
W20260709 12:57:03.518015 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.508174 15646 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.521065 15629 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.524227 15487 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:03.525676 15565 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.512284 15540 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.527576 15540 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.524322 15655 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.539649 13539 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:03.545784 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.560772 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.559365 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.568547 13615 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:57:03.512861 15483 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.574784 13617 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:03.540453 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.581822 13612 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:03.582896 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.583611 13612 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:03.585088 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.578258 15540 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:03.578862 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 143585us
I20260709 12:57:03.571545 15512 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:03.593837 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 96924.5us
I20260709 12:57:03.584703 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.602451 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.575050 15571 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:03.597608 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:03.603554 13615 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:03.584774 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.586097 13616 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:03.597548 15482 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:57:03.608973 15455 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:03.609741 15482 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:03.610198 15482 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:03.610554 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.610208 15600 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:57:03.612445 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.586887 15470 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:03.613507 15470 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.616029 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:03.616461 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.616499 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.616968 14240 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:03.617274 13610 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:03.618079 15529 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:57:03.619220 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.604192 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.620345 13611 raft_consensus.cc:2410] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:03.620434 15455 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:03.620848 15455 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:03.621248 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.621753 15455 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.622375 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.623106 15455 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.608273 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.619283 15529 raft_consensus.cc:3055] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:57:03.624526 15529 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:03.624825 14239 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:03.625681 15529 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.627394 15529 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:57:03.606663 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.545087 15565 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:57:03.611263 15482 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.637125 13617 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:03.643249 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.545491 15681 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:57:03.649700 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.652431 13613 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:03.605124 15571 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.633333 15483 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.665680 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:03.659698 13659 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.870s (SENT)
I20260709 12:57:03.665266 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.611479 14242 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:03.667522 13616 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.639436 15470 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:03.674373 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.675257 13617 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:57:03.659940 13506 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.680405 13537 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:03.647122 15565 consensus_queue.cc:1243] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:57:03.687045 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.682410 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.688674 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.689111 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.690641 13615 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:57:03.661214 15571 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:03.696628 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.698284 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.699229 14239 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
W20260709 12:57:03.701437 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.701601 15076 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:03.702420 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:03.703060 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.706086 13611 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:57:03.707247 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.639878 15512 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.710605 13610 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:03.716436 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 130696us
I20260709 12:57:03.702486 15076 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.719974 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.720455 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.726629 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.720441 13543 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:03.733332 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.722252 13536 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:03.694590 15470 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:03.741559 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:03.739626 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.694612 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:03.747562 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:03.748548 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.753759 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:03.754184 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:03.749099 14243 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:57:03.737442 13542 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:03.758738 15663 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:03.759035 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.759317 14211 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.763813 15540 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:03.721630 15472 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.764475 13693 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.767125 13535 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:03.759773 13617 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:57:03.759759 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 105656us
W20260709 12:57:03.781389 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 128943us
I20260709 12:57:03.736467 15582 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:03.744973 15683 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.792196 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:03.706936 15647 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.790418 13611 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
W20260709 12:57:03.797699 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.799376 15076 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.768028 15413 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:03.716325 15424 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.769620 15470 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.821044 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.821872 13617 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:03.824206 15472 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:03.781201 15655 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:03.827723 13613 raft_consensus.cc:2410] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:03.825822 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:03.832635 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.844696 13610 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
W20260709 12:57:03.844848 13659 leader_election.cc:336] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.907s (SENT)
I20260709 12:57:03.845482 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:03.814849 15584 consensus_meta.cc:287] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.004s
I20260709 12:57:03.851948 13691 raft_consensus.cc:2410] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:57:03.797647 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.786011 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:03.840027 13686 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:03.815797 15413 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:03.781175 15467 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:57:03.827337 15470 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:03.884244 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 155385us
W20260709 12:57:03.891872 15603 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.765s	user 0.000s	sys 0.013s
W20260709 12:57:03.871968 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45097.2us
I20260709 12:57:03.893577 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:03.892479 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:03.897509 15627 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:03.898175 15627 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:03.898684 15627 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.826270 15631 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:57:03.910341 15076 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:03.839093 15472 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:57:03.881093 15413 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.914176 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.880461 15622 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:57:03.926278 15658 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:57:03.927342 13612 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:03.927235 15453 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:57:03.928056 15532 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:03.929006 15453 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:03.870223 15584 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.929469 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:03.859567 15655 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:03.930307 13618 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:03.936693 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.937906 13617 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:03.906364 15539 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.221s
W20260709 12:57:03.935178 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 142143us
I20260709 12:57:03.855140 15683 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.983736 13693 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:03.984143 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:03.986585 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:03.989768 13617 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 1. Current term is 2. Ops: []
W20260709 12:57:03.990864 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 160689us
I20260709 12:57:03.991194 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:03.943374 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:03.989821 13616 raft_consensus.cc:2410] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:03.994318 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:03.994940 13618 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:57:03.912624 14239 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:57:04.001390 13686 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
W20260709 12:57:04.014990 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 255751us
I20260709 12:57:04.017849 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:04.026140 15470 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:03.987228 15622 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:04.041786 13617 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:04.042491 15076 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.038074 13691 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:57:04.046061 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.008848 15531 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:04.009491 15512 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:04.034633 15663 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:04.057741 15076 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:04.061882 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:04.068003 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:04.068363 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 154947us
I20260709 12:57:04.060742 13616 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:57:04.099298 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.074033 13689 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:57:04.107378 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 164085us
I20260709 12:57:04.115638 13693 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:04.140206 13616 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:04.119325 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:04.140901 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.074700 15467 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.183089 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:04.186859 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 273101us
I20260709 12:57:04.189018 14212 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:04.189666 13688 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:57:04.190667 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.195205 15683 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:04.119719 15682 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:57:04.226737 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:04.074043 15603 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 12:57:04.139457 15539 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:57:04.114611 15663 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:04.233776 13693 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:04.118537 15584 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:04.234333 15076 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:04.228621 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 100272us
I20260709 12:57:04.130791 15512 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.263834 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:04.190125 15647 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:04.244853 15646 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.724s	user 0.007s	sys 0.004s
W20260709 12:57:04.291160 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:04.298023 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 280354us
I20260709 12:57:04.268679 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:04.298373 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.243736 13612 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:04.303844 13684 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:04.300545 13688 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
I20260709 12:57:04.255856 15539 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:04.264987 15603 consensus_queue.cc:1243] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:57:04.223613 15467 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.262185 15682 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:04.315157 15539 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:04.139526 15622 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:04.253394 15424 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:04.323851 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:04.326426 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.309929 15720 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:04.360621 15713 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:04.365173 15657 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.223s	user 0.001s	sys 0.001s
I20260709 12:57:04.365911 15573 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:04.365594 15713 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.366986 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:04.368469 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:04.375883 13659 connection.cc:256] RPC call timeout handler was delayed by -1.0741535850002037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2905s for a response
W20260709 12:57:04.347361 15640 consensus_meta.cc:287] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.861s	user 0.009s	sys 0.006s
I20260709 12:57:04.362901 15076 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:04.401165 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:04.401942 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:04.401890 13536 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:57:04.350745 15720 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:04.426000 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 79457.9us
W20260709 12:57:04.413717 13659 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.615s (SENT) [suppressed 23 similar messages]
W20260709 12:57:04.440277 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 176541us
I20260709 12:57:04.432281 15076 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.453823 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:04.349362 15512 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.376359 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:04.469702 13614 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.000s	sys 0.016s
I20260709 12:57:04.491999 13688 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 2.
W20260709 12:57:04.492269 13614 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.000s	sys 0.017s
I20260709 12:57:04.493448 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2359) took 1286 ms. Trace:
W20260709 12:57:04.464345 15461 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.004s	sys 0.012s
W20260709 12:57:04.496944 15461 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.004s	sys 0.012s
I20260709 12:57:04.497231 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:57:04.497506 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:04.497821 15455 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.874s	user 0.007s	sys 0.006s
I20260709 12:57:04.498210 14241 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:04.489897 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.501263 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.501458 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.502602 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.502959 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.503232 13613 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
W20260709 12:57:04.506776 15705 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:57:04.513146 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:04.514735 15708 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.385s	user 0.001s	sys 0.000s
I20260709 12:57:04.494033 13614 rpcz_store.cc:276] 0709 12:57:03.207200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.260864 (+ 53664us) service_pool.cc:224] Handling call
0709 12:57:03.261485 (+   621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:03.262064 (+   579us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:03.424747 (+162683us) log.cc:844] Serialized 14 byte log entry
0709 12:57:04.492798 (+1068051us) log.cc:844] Serialized 14 byte log entry
0709 12:57:04.493033 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:04.493080 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:04.493124 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:04.493213 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:04.493374 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23997,"prepare.queue_time_us":377509,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":83,"raft.queue_time_us":29125,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5851,"thread_start_us":1161569,"threads_started":3,"wal-append.queue_time_us":968376}
I20260709 12:57:04.524014 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:04.432109 15720 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:04.526084 13611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.028s	sys 0.052s
W20260709 12:57:04.530989 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:04.537634 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:04.563213 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.521768 15663 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:04.520139 15682 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:04.587906 13618 consensus_meta.cc:287] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.004s
W20260709 12:57:04.525494 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 221042us
W20260709 12:57:04.919488 15539 consensus_meta.cc:287] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.007s
I20260709 12:57:04.449000 15373 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:04.595654 13619 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
W20260709 12:57:04.497067 13508 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 048528e413ac4c67a1cfd3dfe70673c7. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:57:04.606684 13611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.029s	sys 0.052s
W20260709 12:57:04.922327 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 456802us
W20260709 12:57:04.613868 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.503842 13612 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:04.920125 15539 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.924547 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.925969 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.926607 13612 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:57:04.613018 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:04.928637 13536 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:04.614880 13618 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:04.929908 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 318) took 1042 ms. Trace:
I20260709 12:57:04.930408 13618 rpcz_store.cc:276] 0709 12:57:03.887064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.941954 (+ 54890us) service_pool.cc:224] Handling call
0709 12:57:04.929835 (+987881us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:04.612739 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:04.923245 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 316) took 1217 ms. Trace:
W20260709 12:57:04.932407 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 319822us
I20260709 12:57:04.654301 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:04.654937 15729 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.592272 15646 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.657224 13541 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:04.659245 15717 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:04.640496 15455 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:04.605365 15725 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:04.617669 15571 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.887s	user 0.010s	sys 0.006s
W20260709 12:57:04.618105 15534 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.135s	user 0.011s	sys 0.010s
W20260709 12:57:04.661329 15703 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:57:04.661652 15600 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:57:04.662072 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 157741us
I20260709 12:57:04.662719 15512 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:04.669495 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 255724us
W20260709 12:57:04.669415 13691 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.000s	sys 0.014s
W20260709 12:57:04.682305 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.688791 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:04.692204 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:04.703850 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 239934us
I20260709 12:57:04.708454 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:04.686338 15671 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.204s	user 0.011s	sys 0.004s
I20260709 12:57:04.672066 15713 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:04.695894 14237 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:57:04.741968 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 78459.5us
I20260709 12:57:04.742259 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:04.954566 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 246239us
I20260709 12:57:04.925997 15539 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:04.759287 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.776392 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 314) took 1070 ms. Trace:
I20260709 12:57:04.779068 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.958997 13616 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:04.783207 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1202) took 1115 ms. Trace:
I20260709 12:57:04.786105 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 740) took 1205 ms. Trace:
I20260709 12:57:04.786579 15424 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:04.790009 15413 consensus_meta.cc:287] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.789s	user 0.015s	sys 0.000s
W20260709 12:57:04.758361 15593 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.046s	sys 0.047s
I20260709 12:57:04.749070 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:04.746430 15531 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:57:04.758474 15467 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:04.969532 13543 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:04.815212 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:04.846951 15655 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:04.847991 15720 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:04.851066 15627 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.952s	user 0.007s	sys 0.007s
I20260709 12:57:04.861011 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:04.863538 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.920147 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:04.931836 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:04.935536 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:04.932754 13611 rpcz_store.cc:276] 0709 12:57:03.706149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.798477 (+ 92328us) service_pool.cc:224] Handling call
0709 12:57:03.798951 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:03.852094 (+ 53143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:03.852139 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:03.884271 (+ 32132us) log.cc:844] Serialized 52 byte log entry
0709 12:57:04.922733 (+1038462us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:04.922792 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:04.922889 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:04.922947 (+    58us) raft_consensus.cc:1685] finished
0709 12:57:04.922965 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:04.923164 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9368,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":16226,"thread_start_us":990241,"threads_started":1,"wal-append.queue_time_us":910071}
I20260709 12:57:04.938902 15455 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:04.985956 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:04.965922 15740 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.988101 13543 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:04.988468 15740 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.989205 15470 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:04.992156 15691 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:04.939870 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:04.994427 13614 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:04.995308 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:04.995857 13614 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:04.937880 15729 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:04.927781 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.000567 15729 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:05.001029 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.001981 13612 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 2: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:05.002280 15729 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:05.002939 15679 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.091s	user 0.000s	sys 0.004s
I20260709 12:57:04.953349 13540 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:04.986006 15534 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:05.003755 15534 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:05.004225 15534 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.006623 15627 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.977972 15424 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.008960 15627 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.010927 13535 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:57:05.011152 15720 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.011411 15424 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:04.714293 15681 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.956148 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:04.955150 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 212995us
I20260709 12:57:04.955597 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:05.025486 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:04.959538 13542 rpcz_store.cc:276] 0709 12:57:03.668072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.693699 (+ 25627us) service_pool.cc:224] Handling call
0709 12:57:04.772597 (+1078898us) spinlock_profiling.cc:243] Waited 9.94 ms on lock 0x7b54003cb754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:04.783097 (+ 10500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22865280}
I20260709 12:57:04.958516 13610 rpcz_store.cc:276] 0709 12:57:03.705682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.720215 (+ 14533us) service_pool.cc:224] Handling call
0709 12:57:04.776046 (+1055831us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540020ec54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:04.776176 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275765504}
I20260709 12:57:04.936975 15640 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:04.945200 15600 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:04.940128 15725 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:04.940387 13534 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 2.
I20260709 12:57:04.977254 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:04.966833 15593 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.047s	sys 0.047s
I20260709 12:57:04.984000 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 367) took 1405 ms. Trace:
I20260709 12:57:04.984236 14246 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:05.030002 15600 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.043287 15470 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:04.942986 15571 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.044090 15470 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.045991 15571 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:05.046149 15470 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.046833 14238 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:04.985129 13584 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:05.042053 15482 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:05.048414 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 63379.8us
I20260709 12:57:05.049978 15483 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:04.985258 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.050743 13618 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
W20260709 12:57:04.985514 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 65512.7us
I20260709 12:57:05.036876 15123 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:05.054031 15123 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.054404 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:04.986191 15455 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.056448 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.057240 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.057971 13540 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
W20260709 12:57:04.952797 13691 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.000s	sys 0.015s
I20260709 12:57:04.955085 15671 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:04.936021 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:05.014533 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.014827 13537 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:04.991521 15740 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:05.003651 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.072379 13538 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:04.938033 15717 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:05.016201 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.072624 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:05.072822 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.073249 13543 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:04.960464 14244 rpcz_store.cc:276] 0709 12:57:03.581018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.604468 (+ 23450us) service_pool.cc:224] Handling call
0709 12:57:03.608595 (+  4127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:03.659899 (+ 51304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:03.659970 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:03.693885 (+ 33915us) log.cc:844] Serialized 26 byte log entry
0709 12:57:03.860485 (+166600us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:03.860791 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.861274 (+   483us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:04.785728 (+924454us) raft_consensus.cc:1685] finished
0709 12:57:04.785751 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:04.785899 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38702,"raft.queue_time_us":67954,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":13735,"thread_start_us":98622,"threads_started":1,"wal-append.queue_time_us":1037408}
I20260709 12:57:04.965035 15373 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:04.964449 15512 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:04.966038 15713 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:05.029727 13614 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:05.059378 15691 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:05.031280 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:05.013940 14241 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.515s	user 0.004s	sys 0.009s
I20260709 12:57:05.096139 14241 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:57:05.086475 15719 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:05.036479 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 80938.6us
I20260709 12:57:05.098330 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:05.034332 15725 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:05.042090 15747 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:57:05.002768 15729 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.043054 13539 rpcz_store.cc:276] 0709 12:57:03.578548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.592381 (+ 13833us) service_pool.cc:224] Handling call
0709 12:57:03.592734 (+   353us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:03.606264 (+ 13530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:03.606297 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:03.648925 (+ 42628us) log.cc:844] Serialized 52 byte log entry
0709 12:57:03.887655 (+238730us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:03.914529 (+ 26874us) log.cc:844] Serialized 14 byte log entry
0709 12:57:03.921343 (+  6814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:03.921440 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:04.983720 (+1062280us) raft_consensus.cc:1685] finished
0709 12:57:04.983743 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:04.983940 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11184,"prepare.queue_time_us":114146,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":19482,"raft.queue_time_us":14988,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":174,"thread_start_us":191677,"threads_started":1,"wal-append.queue_time_us":744282}
I20260709 12:57:05.013482 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:05.073189 13616 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:04.938246 15646 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.076313 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:05.072088 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 94913.7us
I20260709 12:57:05.080606 15622 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:57:05.029783 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 77054.5us
I20260709 12:57:04.970928 15413 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.141737 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:05.088714 14243 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:57:05.089900 15752 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:57:05.072239 15593 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.081174 15717 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.080263 15512 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.148977 15512 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.150820 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.150540 15534 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.095135 13540 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:57:05.083033 15713 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.098484 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:05.041229 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.106181 13543 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:05.071260 15584 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:57:05.109144 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 185928us
I20260709 12:57:05.035671 15681 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.027165 15640 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:05.138756 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:05.156560 13659 leader_election.cc:336] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.518s (SENT)
I20260709 12:57:05.166568 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:05.060352 15731 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:05.107664 15627 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:05.096925 15719 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.145148 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.083562 15470 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 12:57:05.146358 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 48120.3us
W20260709 12:57:05.162180 15582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.255s	user 0.022s	sys 0.031s
I20260709 12:57:05.082652 15373 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.127595 15123 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.120376 15691 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:05.180437 15755 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:05.192255 15755 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:05.192917 15755 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.192549 15691 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.170266 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1185) took 1112 ms. Trace:
I20260709 12:57:05.196430 13691 rpcz_store.cc:276] 0709 12:57:04.058110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:04.060004 (+  1894us) service_pool.cc:224] Handling call
0709 12:57:04.060333 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:04.060561 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:05.060966 (+1000405us) log.cc:844] Serialized 14 byte log entry
0709 12:57:05.169234 (+108268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:05.169314 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:05.169375 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:05.169446 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:05.170182 (+   736us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20562,"prepare.queue_time_us":748992,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"thread_start_us":1076498,"threads_started":3,"wal-append.queue_time_us":65439}
I20260709 12:57:05.157977 15531 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:57:05.175262 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.160000 15713 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.180945 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:05.081355 15740 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:05.163097 15584 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:05.186861 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 48231us
I20260709 12:57:05.185340 13534 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:05.188758 15735 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:05.188884 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.190567 15681 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.162356 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:05.233906 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 71639.1us
I20260709 12:57:05.194712 15719 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:05.169502 15640 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:05.181844 15582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.295s	user 0.023s	sys 0.031s
W20260709 12:57:05.207089 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.169931 13535 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:05.206837 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.242509 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:05.244272 13658 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:05.208122 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 109705us
I20260709 12:57:05.200294 13541 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:05.182562 13616 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
W20260709 12:57:05.218334 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.229327 15600 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.230566 15534 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.224915 15655 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:57:05.231582 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.200249 15603 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:57:05.235366 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:05.200660 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.232466 15373 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.220423 15584 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.232609 15735 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.271000 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.179421 15731 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.252871 15681 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:05.260485 13617 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:05.238502 15627 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:05.249559 15532 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:57:05.266942 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:05.266433 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.271129 13542 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:05.266351 15766 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:57:05.246963 15582 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:05.263036 15603 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.271554 15663 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:05.249959 15740 raft_consensus.cc:697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:05.260481 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:05.277400 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.269742 15671 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.263080 15531 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:05.292529 15646 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.281543 15373 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:57:05.294787 15413 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.317641 15603 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:05.296221 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.298388 15455 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.281087 15681 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:05.306938 15600 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.282990 15764 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:05.266834 15761 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:05.329183 15532 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:05.274827 15735 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.329106 13617 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:05.345029 15767 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:57:05.344225 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 187701us
I20260709 12:57:05.293200 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.273244 15470 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:57:05.317389 15663 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:05.360659 15582 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.348608 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.361259 15455 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:05.357097 15681 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:05.362649 14244 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:05.272958 15736 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:05.336874 15123 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:05.353358 15729 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.369159 15413 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:57:05.362759 15717 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.363552 15764 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:05.401860 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.363361 14237 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
W20260709 12:57:05.402010 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 135137us
W20260709 12:57:05.403545 13619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1297) took 1724 ms (client timeout 1746 ms). Trace:
I20260709 12:57:05.387104 15646 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:05.416054 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.416343 15646 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.418942 15646 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.358999 15765 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:05.420167 15765 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.422906 15765 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.416059 15123 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:05.425015 15123 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.425479 15531 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.391381 15663 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:05.404868 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 117216us
I20260709 12:57:05.387688 15373 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:05.428442 15740 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.429159 15373 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.407729 15764 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.412649 15717 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:05.409039 13619 rpcz_store.cc:269] 0709 12:57:03.679724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.693994 (+ 14270us) service_pool.cc:224] Handling call
0709 12:57:05.403368 (+1709374us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54002b3c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:05.403487 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554454400}
I20260709 12:57:05.369194 15715 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:05.411139 15532 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:05.441409 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:05.445699 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 201480us
I20260709 12:57:05.447841 15076 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.446650 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.450213 13619 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:05.450664 15076 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:05.451102 15076 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.451476 15076 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.451253 15729 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:05.452096 15729 consensus_queue.cc:1243] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:57:05.450847 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 101441us
I20260709 12:57:05.454492 13508 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.451643 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.460482 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.461313 13617 raft_consensus.cc:2393] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:05.445523 15761 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:05.466094 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.452597 15729 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:05.468586 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.470237 13616 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:05.470997 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.471644 13614 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:05.406544 15725 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.476819 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.477253 15763 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:05.478032 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.478528 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.446959 15755 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.465545 15774 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:05.480139 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.480592 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.471203 15729 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:05.464774 15622 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:05.487046 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:05.446081 15715 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.406450 15736 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.470023 15482 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:05.496502 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.514042 15736 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.525374 15470 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:05.513832 15755 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.511831 15076 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.514051 15622 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:05.515844 15534 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:57:05.513921 15715 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.489004 13612 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:05.490803 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.493072 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.541797 13690 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:05.538710 13614 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:05.543077 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.544615 15725 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.519117 15593 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:57:05.548646 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.550176 14239 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:57:05.550568 14239 pending_rounds.cc:85] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 6
I20260709 12:57:05.550863 14239 pending_rounds.cc:107] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:57:05.551308 14239 raft_consensus.cc:2889] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:05.539965 15622 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.552773 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.553728 14240 raft_consensus.cc:2410] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:57:05.492652 13684 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:57:05.554848 13684 pending_rounds.cc:85] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 4
I20260709 12:57:05.555096 13684 pending_rounds.cc:107] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:57:05.555955 13684 raft_consensus.cc:2889] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:05.557252 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.514999 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.494693 15779 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:57:05.559456 15779 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:05.475445 15761 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.560731 15779 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.472904 15512 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.562620 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.562994 14212 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:05.543834 13690 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:05.525056 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:05.524878 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.567209 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:05.585414 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.586144 13616 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:57:05.549937 15767 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:57:05.611822 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.612576 13610 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:05.612974 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 45855.6us
I20260709 12:57:05.505812 15584 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.625496 15076 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:05.537473 15731 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.630782 13617 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:57:05.628729 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.513499 15482 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.630581 13686 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:05.619318 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:05.592736 15725 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:05.591995 15470 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:05.513159 15729 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.636642 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 251) took 1235 ms. Trace:
I20260709 12:57:05.642601 13688 rpcz_store.cc:276] 0709 12:57:04.401109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:04.401710 (+   601us) service_pool.cc:224] Handling call
0709 12:57:05.636417 (+1234707us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b540034e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:05.636565 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147838336}
W20260709 12:57:05.649971 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.657335 13616 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:05.621888 15715 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:05.664333 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.557662 15763 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.664858 13687 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
I20260709 12:57:05.665268 13618 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:05.664618 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.666792 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:05.505549 15603 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.670915 13688 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.607362 15640 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.671720 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.644495 15482 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:05.677392 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:05.647872 15470 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:05.696250 13584 leader_election.cc:336] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.785s (SENT)
I20260709 12:57:05.647903 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.708005 13693 raft_consensus.cc:2410] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:05.713392 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:05.721678 13610 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:05.512261 15774 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.664498 15715 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.715972 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.662053 15725 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:05.665355 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.699429 15720 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:05.673745 15413 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:05.675992 15540 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:05.678328 15781 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:05.727792 13685 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:05.727425 13611 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:05.719693 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.733428 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:05.734017 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56698.3us
I20260709 12:57:05.674031 15600 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:57:05.737447 14239 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:57:05.738404 15603 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.739697 15584 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.743041 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:05.726501 15750 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:57:05.752998 13688 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:05.724184 15470 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:05.725636 15776 raft_consensus.cc:2764] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:57:05.727038 15715 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:05.757954 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.756942 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.762475 13689 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.734958 15720 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.765425 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.770241 13614 raft_consensus.cc:2410] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:05.765687 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.765758 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:05.766593 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.767561 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.745323 15413 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.771421 13610 raft_consensus.cc:2410] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:05.771661 13615 raft_consensus.cc:2376] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4. Re-sending same reply.
I20260709 12:57:05.774181 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.776850 13688 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:05.777503 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:05.778106 13536 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.778230 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.778864 13610 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:05.779791 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.781689 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.782748 13610 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:05.783529 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.784628 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.785167 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:05.785491 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.785948 13616 raft_consensus.cc:2410] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:05.786506 13614 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:05.784274 13610 raft_consensus.cc:2410] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:57:05.789358 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.791208 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.791718 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.792086 13618 raft_consensus.cc:2393] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:05.754817 15761 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:05.791209 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:05.792738 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.795360 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.763399 15540 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:05.796887 15540 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.761318 15789 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:57:05.798393 15789 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:05.798769 15789 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:05.769210 15671 raft_consensus.cc:2764] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:57:05.773842 15720 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.808957 13688 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
W20260709 12:57:05.810307 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.754530 15681 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.816807 14245 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:05.778901 15413 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.828722 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:05.835539 14209 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:05.806630 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.837642 14209 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:05.827864 13582 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:05.801661 15512 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.828040 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.844415 13688 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.840219 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:05.854250 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.857893 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.768311 15734 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:57:05.860378 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.862960 13691 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:57:05.864708 15785 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:57:05.864742 15723 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:05.865453 15584 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:05.866139 15584 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:05.866418 14238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 741) took 2284 ms. Trace:
I20260709 12:57:05.866544 15584 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:05.843168 15455 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:05.754611 15781 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.869923 15781 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:05.857281 15776 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:05.871040 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.871696 15776 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:05.872479 15776 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.873080 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.873710 15445 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:05.874655 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.874341 15445 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:05.880067 15791 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:05.880473 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.809557 15719 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:57:05.880779 15791 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.881197 13687 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:05.881717 15719 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:05.882087 15719 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:05.882275 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.883049 15791 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.882977 15719 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:05.883474 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.864588 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:05.873338 15787 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:57:05.929677 13582 leader_election.cc:400] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
I20260709 12:57:05.930042 13582 leader_election.cc:403] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:57:05.865509 15723 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:05.887516 13508 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:05.934398 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46978.2us
W20260709 12:57:05.935532 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.936080 15781 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:05.887943 13542 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:05.866920 14238 rpcz_store.cc:276] 0709 12:57:03.581656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:03.611985 (+ 30329us) service_pool.cc:224] Handling call
0709 12:57:05.866165 (+2254180us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400631354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:05.866327 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3296591232}
I20260709 12:57:05.874786 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.939594 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.940315 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.940647 14238 raft_consensus.cc:2368] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4.
I20260709 12:57:05.941800 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.943009 14238 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:05.911571 13689 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
W20260709 12:57:05.925271 15786 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:57:05.933133 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.836725 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.945372 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:05.945955 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.945393 13541 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:05.948299 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:05.948580 15781 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:05.950254 14238 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:05.951992 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:05.953203 15781 consensus_queue.cc:1059] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:05.932119 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:05.843958 15593 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:57:05.956442 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 91934.7us
I20260709 12:57:05.854983 15565 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:05.844305 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:05.960160 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:05.967048 13687 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:05.969512 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:05.975371 13658 leader_election.cc:400] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4.
I20260709 12:57:05.859169 15534 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:05.958786 15565 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:05.884608 15640 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:05.799638 15789 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:05.976760 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 183966us
I20260709 12:57:05.974315 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:05.975868 13658 leader_election.cc:403] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:05.998946 13688 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:05.955488 15483 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:05.932857 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:05.960022 15655 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:05.971436 15719 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:06.009142 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:06.010156 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.023756 13614 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:05.982658 15565 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:05.980504 15593 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:06.026537 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:06.026829 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:05.854314 15725 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:06.009244 15713 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.003s
I20260709 12:57:06.027503 13615 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:06.035322 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:06.041966 13614 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:05.956192 15736 consensus_queue.cc:1175] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "058c47afa94546dba814cf345f414f4a" responder_term: 3 status { last_received { term: 3 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 3 index: 6 } }
I20260709 12:57:06.016371 15717 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:06.053706 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:06.054466 13692 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:06.045226 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 64610.6us
I20260709 12:57:06.015682 15761 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:57:06.060238 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.060914 13538 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
W20260709 12:57:06.037655 15764 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.003s
I20260709 12:57:06.065557 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:05.997028 15534 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.046155 15789 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:06.067425 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:06.052502 15646 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.010s	sys 0.027s
I20260709 12:57:06.079483 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:06.082789 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:06.084586 15445 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:06.086906 15715 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:06.088044 15768 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:57:06.089952 15768 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:06.090830 15777 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:57:06.092108 15771 consensus_queue.cc:1059] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 12:57:06.093264 15771 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:06.093685 15771 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:06.094791 15771 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.074833 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:06.101799 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.052776 15671 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:06.102634 13538 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:06.074288 15646 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.010s	sys 0.027s
W20260709 12:57:06.119342 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 48950.5us
I20260709 12:57:06.068676 15717 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:06.049611 15713 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:06.124228 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:06.077615 15781 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:06.125735 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.049291 15795 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:57:06.132856 13692 raft_consensus.cc:2368] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5.
I20260709 12:57:06.083899 13612 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:06.068774 15764 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:06.136693 15663 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.000s
I20260709 12:57:06.144747 14246 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
I20260709 12:57:06.130105 15770 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:06.130404 15781 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:06.131058 15771 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:06.147938 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.128975 15801 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:06.119341 15671 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.156749 13689 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:06.140908 15681 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:06.070990 15723 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.155570 15781 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:06.161442 15791 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:06.171947 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.182116 15715 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:06.183437 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:06.184514 15805 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:06.185052 15805 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:06.187747 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.195185 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.136983 15593 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.198932 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.230020 15764 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:06.188606 13541 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
W20260709 12:57:06.231647 15797 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:57:06.230233 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.234086 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.236044 13539 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:57:06.242209 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.259397 15729 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:06.262044 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.292730 13543 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 2: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:06.152417 15663 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:06.306134 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:06.305627 15788 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.516s	user 0.000s	sys 0.003s
I20260709 12:57:06.321619 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.174439 15717 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.305624 15774 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:06.329810 14240 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:57:06.330693 13693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 249) took 2026 ms (client timeout 1850 ms). Trace:
I20260709 12:57:06.331060 15713 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:06.331107 13693 rpcz_store.cc:269] 0709 12:57:04.304714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:04.304935 (+   221us) service_pool.cc:224] Handling call
0709 12:57:06.330481 (+2025546us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400297cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:06.330618 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3159402240}
I20260709 12:57:06.332428 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.338925 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.340008 13539 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:06.354379 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:06.357810 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:06.378955 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:06.382795 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:06.389387 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:06.390411 13536 consensus_meta.cc:287] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.024s
I20260709 12:57:06.390491 13539 raft_consensus.cc:2368] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
I20260709 12:57:06.391155 13536 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:06.393939 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:06.394047 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 238034us
I20260709 12:57:06.395867 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.400097 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.323854 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.399711 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:06.415839 13539 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:06.420893 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:06.411995 15740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.013s	sys 0.025s
I20260709 12:57:06.408267 15534 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:06.433429 13539 raft_consensus.cc:2410] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:06.415400 13535 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:06.441561 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:06.412611 15663 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:06.448825 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.449976 13536 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:06.451334 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:06.427309 15740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.014s	sys 0.025s
I20260709 12:57:06.462143 13687 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:06.477743 15720 consensus_meta.cc:287] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.652s	user 0.019s	sys 0.000s
I20260709 12:57:06.478451 15720 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.485533 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:06.486621 13617 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:06.487145 13538 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
W20260709 12:57:06.495096 15470 consensus_meta.cc:287] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.726s	user 0.027s	sys 0.012s
I20260709 12:57:06.495635 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:06.497067 14237 raft_consensus.cc:2393] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:06.498001 15805 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:06.498768 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 253) took 1016 ms. Trace:
I20260709 12:57:06.503715 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:06.505899 15767 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:06.506626 15781 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:06.507959 13692 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
I20260709 12:57:06.508180 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:06.510162 13617 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:06.499197 13691 rpcz_store.cc:276] 0709 12:57:05.481983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.486257 (+  4274us) service_pool.cc:224] Handling call
0709 12:57:05.503073 (+ 16816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:05.863514 (+360441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:06.498352 (+634838us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54003a0a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:06.498680 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826855808}
I20260709 12:57:06.442574 15472 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:06.514739 13684 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:57:06.515700 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.519425 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.495690 15470 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:06.521961 15482 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.791s	user 0.003s	sys 0.017s
I20260709 12:57:06.522415 15482 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:06.522758 15470 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:06.523164 15603 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:06.523584 13686 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:06.525574 15482 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:06.522593 15445 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:06.528143 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.520635 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:06.529242 13543 raft_consensus.cc:2368] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6.
I20260709 12:57:06.530344 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:06.535537 13536 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:06.537062 13539 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:06.523947 15470 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:06.530867 15472 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:06.545476 15472 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:06.551126 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:06.554868 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.538517 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.540081 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:06.511961 15671 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:06.559892 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:06.560686 13687 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:06.562534 13691 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:06.588580 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.589996 13692 raft_consensus.cc:2410] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:06.596696 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.597484 13539 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:06.529536 15445 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:06.598513 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:06.563738 15413 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.023s
I20260709 12:57:06.542769 15470 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:06.600379 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.552246 15774 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:06.612964 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:06.618108 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 97544.8us
I20260709 12:57:06.618101 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:06.619175 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:06.620534 13536 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:06.616381 13687 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:06.564782 15767 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:06.629742 13692 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:06.630510 13539 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:06.585085 13535 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:06.628322 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:06.636375 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.625315 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:06.622125 15584 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.755s	user 0.016s	sys 0.011s
W20260709 12:57:06.624365 15655 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.018s
W20260709 12:57:06.624751 15622 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.072s	user 0.015s	sys 0.008s
I20260709 12:57:06.643648 13540 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:06.623194 15735 consensus_meta.cc:287] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.229s	user 0.022s	sys 0.001s
W20260709 12:57:06.624974 15776 consensus_meta.cc:287] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.750s	user 0.008s	sys 0.019s
I20260709 12:57:06.631613 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:06.672647 13539 raft_consensus.cc:2368] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6.
I20260709 12:57:06.673985 13539 pending_rounds.cc:85] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 5
I20260709 12:57:06.674379 13539 pending_rounds.cc:107] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:57:06.674623 13539 raft_consensus.cc:2889] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:06.652603 15720 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:06.675637 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:06.676638 15776 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:06.654340 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.678951 13692 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:06.679391 15776 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:06.680462 15819 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:57:06.680383 15735 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.656111 13536 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:57:06.685170 15735 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:06.685531 13536 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:06.657163 13535 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 12:57:06.625048 15532 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.122s	user 0.007s	sys 0.018s
I20260709 12:57:06.667016 13658 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:06.667774 15719 consensus_meta.cc:287] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.637s	user 0.014s	sys 0.009s
I20260709 12:57:06.665647 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:06.692281 13691 raft_consensus.cc:2410] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:57:06.669422 15770 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:06.671429 13540 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:57:06.688175 15761 consensus_meta.cc:287] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.018s
I20260709 12:57:06.612437 15413 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.700377 15413 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:06.709743 15565 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.649s	user 0.015s	sys 0.011s
I20260709 12:57:06.710268 15565 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.711666 14245 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:06.712316 15781 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:06.683655 15622 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.714989 15622 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:06.715646 13537 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:06.716642 13541 raft_consensus.cc:2393] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:06.671952 15764 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:06.667802 13687 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
W20260709 12:57:06.694350 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 75371us
I20260709 12:57:06.664659 15584 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.694598 13540 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:06.668357 15655 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
I20260709 12:57:06.695606 14244 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:06.637035 15774 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.690299 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:06.690995 15532 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.007s	sys 0.018s
I20260709 12:57:06.665248 15445 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:06.691167 15470 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:06.724407 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.083858 13616 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:06.833204 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:06.776150 13543 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:57:07.085239 13691 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:07.085501 13543 pending_rounds.cc:85] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 2
W20260709 12:57:06.896858 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.086584 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:07.087615 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1188) took 1329 ms. Trace:
W20260709 12:57:07.084434 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 394175us
I20260709 12:57:07.088088 13687 rpcz_store.cc:276] 0709 12:57:05.758570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.723528 (+964958us) service_pool.cc:224] Handling call
0709 12:57:07.087536 (+364008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.088884 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:07.089574 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:07.090586 13687 raft_consensus.cc:2393] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:07.091318 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 369742us
I20260709 12:57:06.848376 13541 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:07.093068 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:06.774143 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 259) took 1289 ms. Trace:
I20260709 12:57:06.906272 13692 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:06.724570 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.095450 13613 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:06.755821 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 197333us
I20260709 12:57:06.845165 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:06.695837 15770 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:07.100190 13688 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:07.116407 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:07.085846 13543 pending_rounds.cc:107] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:57:07.102963 13506 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.125329 13543 raft_consensus.cc:2889] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:07.102545 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 390) took 1102 ms. Trace:
I20260709 12:57:07.126109 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:07.126437 13542 rpcz_store.cc:276] 0709 12:57:06.000520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.677581 (+677061us) service_pool.cc:224] Handling call
0709 12:57:06.677944 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:06.678160 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:06.678378 (+   218us) log.cc:844] Serialized 14 byte log entry
0709 12:57:07.102073 (+423695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:07.102143 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:07.102225 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.102314 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:07.102461 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54484,"prepare.queue_time_us":1282,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"thread_start_us":368725,"threads_started":1}
I20260709 12:57:07.097632 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:07.094316 13685 rpcz_store.cc:276] 0709 12:57:05.485226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.485460 (+   234us) service_pool.cc:224] Handling call
0709 12:57:06.773908 (+1288448us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540031d154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:06.774071 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402014080}
I20260709 12:57:07.114884 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:06.778672 15784 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.968s	user 0.002s	sys 0.000s
W20260709 12:57:06.849412 15801 consensus_meta.cc:287] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.681s	user 0.012s	sys 0.005s
I20260709 12:57:07.129180 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:06.720769 15767 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:06.721549 15761 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4.
I20260709 12:57:07.099718 15729 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:06.843008 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:07.131139 15789 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:07.133744 15470 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:07.131320 15770 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:07.177659 13616 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:07.133752 15776 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:07.139395 15655 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:07.155174 15582 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:07.214253 15582 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.191838 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:07.152662 15801 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:07.228307 13685 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 5. Ops: []
I20260709 12:57:07.244709 15582 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:07.248430 13688 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:07.249315 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2545) took 1608 ms. Trace:
I20260709 12:57:07.249752 13688 rpcz_store.cc:276] 0709 12:57:05.640566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.146036 (+1505470us) service_pool.cc:224] Handling call
0709 12:57:07.248257 (+102221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.249040 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.249250 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.250182 13693 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:07.251416 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2542) took 1718 ms. Trace:
I20260709 12:57:07.253837 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1126) took 1108 ms. Trace:
I20260709 12:57:07.254895 13543 rpcz_store.cc:276] 0709 12:57:06.145196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.775406 (+630210us) service_pool.cc:224] Handling call
0709 12:57:06.775734 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.252041 (+476307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.253706 (+  1665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.255102 13685 rpcz_store.cc:276] 0709 12:57:05.532825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.137797 (+1604972us) service_pool.cc:224] Handling call
0709 12:57:07.202937 (+ 65140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.251053 (+ 48116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.251353 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.221016 13535 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:07.230227 15808 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:07.268380 15808 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:07.269400 15808 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.270684 15808 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:07.273387 15815 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.827s	user 0.000s	sys 0.002s
I20260709 12:57:07.253064 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 462) took 1606 ms. Trace:
I20260709 12:57:07.276342 14241 rpcz_store.cc:276] 0709 12:57:05.641752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.646165 (+  4413us) service_pool.cc:224] Handling call
0709 12:57:06.842805 (+1196640us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:07.233879 (+391074us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9519872}
I20260709 12:57:07.230509 15584 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.277628 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 384) took 1516 ms. Trace:
I20260709 12:57:07.277745 15584 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:07.278049 13537 rpcz_store.cc:276] 0709 12:57:05.761720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.762862 (+  1142us) service_pool.cc:224] Handling call
0709 12:57:05.776577 (+ 13715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:06.716073 (+939496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.277374 (+561301us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b54000b86d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:07.277549 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2159243008}
I20260709 12:57:07.278290 15584 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.216287 15565 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:07.266760 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2383) took 1808 ms. Trace:
I20260709 12:57:07.280395 13619 rpcz_store.cc:276] 0709 12:57:05.458346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.458479 (+   133us) service_pool.cc:224] Handling call
0709 12:57:05.468138 (+  9659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:05.468515 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:05.468539 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:05.474033 (+  5494us) log.cc:844] Serialized 499 byte log entry
0709 12:57:05.620471 (+146438us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:05.620504 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:05.620558 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:06.777900 (+1157342us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:07.263784 (+485884us) raft_consensus.cc:1685] finished
0709 12:57:07.263814 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:07.266659 (+  2845us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5512448,"thread_start_us":142854,"threads_started":1,"wal-append.queue_time_us":275546}
I20260709 12:57:07.248868 15761 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:57:07.285368 15791 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:07.286041 15791 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:07.290216 13685 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:57:07.290983 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2550) took 1553 ms. Trace:
I20260709 12:57:07.291567 13685 rpcz_store.cc:276] 0709 12:57:05.737972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.258236 (+1520264us) service_pool.cc:224] Handling call
0709 12:57:07.289948 (+ 31712us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:07.290676 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.290893 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.293690 13685 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:57:07.294390 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2554) took 1555 ms. Trace:
I20260709 12:57:07.294757 13685 rpcz_store.cc:276] 0709 12:57:05.739150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.293012 (+1553862us) service_pool.cc:224] Handling call
0709 12:57:07.293350 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.294110 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.294335 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.296108 13685 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:57:07.296711 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2556) took 1556 ms. Trace:
I20260709 12:57:07.297056 13685 rpcz_store.cc:276] 0709 12:57:05.748124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.295532 (+1547408us) service_pool.cc:224] Handling call
0709 12:57:07.295860 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:07.296495 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.296666 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:07.297432 14245 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 394) took 1504 ms (client timeout 1662 ms). Trace:
I20260709 12:57:07.298367 13685 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:07.299052 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1187) took 1549 ms. Trace:
I20260709 12:57:07.299417 13685 rpcz_store.cc:276] 0709 12:57:05.749966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.297913 (+1547947us) service_pool.cc:224] Handling call
0709 12:57:07.298197 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.298831 (+   634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.299004 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:07.298079 14245 rpcz_store.cc:269] 0709 12:57:05.793298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.839608 (+ 46310us) service_pool.cc:224] Handling call
0709 12:57:07.297161 (+1457553us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54005f1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.297357 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952272256}
I20260709 12:57:07.215893 15763 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:07.247503 15821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.041s	sys 0.106s
I20260709 12:57:07.251953 15445 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:07.316016 13613 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:07.229308 15655 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:07.309193 15821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.041s	sys 0.107s
I20260709 12:57:07.331679 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 389) took 1333 ms. Trace:
I20260709 12:57:07.332139 13539 rpcz_store.cc:276] 0709 12:57:05.998734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.673311 (+674577us) service_pool.cc:224] Handling call
0709 12:57:06.673730 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:06.675626 (+  1896us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:06.675765 (+   139us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:06.682550 (+  6785us) log.cc:844] Serialized 52 byte log entry
0709 12:57:07.253554 (+571004us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:07.253609 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.253692 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:07.331451 (+ 77759us) raft_consensus.cc:1685] finished
0709 12:57:07.331469 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:07.331612 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25497,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":488788,"threads_started":1,"wal-append.queue_time_us":607901}
I20260709 12:57:07.333693 15584 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.335551 15655 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:07.336290 15655 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:07.336457 13684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2541) took 1804 ms (client timeout 2125 ms). Trace:
W20260709 12:57:07.337091 13684 rpcz_store.cc:269] 0709 12:57:05.532461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.556958 (+ 24497us) service_pool.cc:224] Handling call
0709 12:57:07.336162 (+1779204us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54000488d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.336368 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2199913344}
I20260709 12:57:07.339493 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 321) took 2360 ms. Trace:
I20260709 12:57:07.339948 13611 rpcz_store.cc:276] 0709 12:57:04.979471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.012346 (+ 32875us) service_pool.cc:224] Handling call
0709 12:57:05.012702 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:05.754881 (+742179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.339210 (+1584329us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b5400332054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:07.339395 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1638254336}
I20260709 12:57:07.336704 15655 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:07.342434 15655 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:07.351608 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:07.352936 14243 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 460) took 1745 ms (client timeout 2025 ms). Trace:
I20260709 12:57:07.353293 15808 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:57:07.353325 14243 rpcz_store.cc:269] 0709 12:57:05.616383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.626293 (+  9910us) service_pool.cc:224] Handling call
0709 12:57:07.351437 (+1725144us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:07.352864 (+  1427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3916032}
I20260709 12:57:07.354125 15808 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:07.354491 15808 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:07.360313 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 221332us
I20260709 12:57:07.362339 14209 leader_election.cc:400] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5.
I20260709 12:57:07.321008 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1128) took 1141 ms. Trace:
I20260709 12:57:07.387264 14209 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:07.413765 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 284675us
I20260709 12:57:07.423666 14243 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:07.425678 14246 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 450) took 2375 ms (client timeout 2216 ms). Trace:
W20260709 12:57:07.427089 14246 rpcz_store.cc:269] 0709 12:57:05.050334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.056116 (+  5782us) service_pool.cc:224] Handling call
0709 12:57:07.425454 (+2369338us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54007102d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.425593 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2492718976}
W20260709 12:57:07.432723 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 339730us
I20260709 12:57:07.357556 15593 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:57:07.423188 15565 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:07.355162 15770 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.448300 15571 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:07.413025 15720 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:57:07.439888 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.467453 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 391) took 1340 ms. Trace:
W20260709 12:57:07.467170 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 352407us
I20260709 12:57:07.355299 15808 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.366848 13535 rpcz_store.cc:276] 0709 12:57:06.178969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.817738 (+638769us) service_pool.cc:224] Handling call
0709 12:57:07.142868 (+325130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.265519 (+122651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.319798 (+ 54279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.478470 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:07.483407 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:07.487447 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 351252us
I20260709 12:57:07.453527 15565 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:07.488886 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 386261us
I20260709 12:57:07.488891 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:07.475134 13540 rpcz_store.cc:276] 0709 12:57:06.127266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.728792 (+601526us) service_pool.cc:224] Handling call
0709 12:57:06.729134 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:06.729390 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:06.774630 (+ 45240us) log.cc:844] Serialized 14 byte log entry
0709 12:57:07.114632 (+340002us) log.cc:844] Serialized 14 byte log entry
0709 12:57:07.454509 (+339877us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b1800424300. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 12:57:07.467060 (+ 12551us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:07.467117 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:07.467175 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.467242 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:07.467384 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19010,"prepare.queue_time_us":67464,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":88,"raft.queue_time_us":59778,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":33274,"spinlock_wait_cycles":23356928,"thread_start_us":263213,"threads_started":1,"wal-append.queue_time_us":476130}
I20260709 12:57:07.492630 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:07.493098 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:07.486917 15776 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:07.495584 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:07.468441 15774 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.493793 14242 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:07.484396 15584 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:57:07.503587 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.491504 15814 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:07.486955 15461 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:57:07.511132 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.455088 15571 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:07.520819 13689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2553) took 1782 ms (client timeout 1965 ms). Trace:
W20260709 12:57:07.521404 13689 rpcz_store.cc:269] 0709 12:57:05.738876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.947761 (+208885us) service_pool.cc:224] Handling call
0709 12:57:07.520551 (+1572790us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b540002c454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.520730 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475308416}
I20260709 12:57:07.490173 15789 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:07.496347 15413 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:07.549387 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:07.549562 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.500979 15843 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:07.511816 15763 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:07.532066 15841 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:57:07.565603 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 89542.2us
I20260709 12:57:07.517366 15764 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:07.524070 15847 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:07.516054 15808 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:07.554531 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 59014.9us
I20260709 12:57:07.569628 13691 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:07.552343 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:07.572722 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:07.583493 13688 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 6. Ops: [4.3-4.3]
I20260709 12:57:07.574726 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 754) took 1637 ms. Trace:
I20260709 12:57:07.584473 14244 rpcz_store.cc:276] 0709 12:57:05.937395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.191810 (+254415us) service_pool.cc:224] Handling call
0709 12:57:06.192377 (+   567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.084060 (+891683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.574499 (+490439us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54005ce754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:07.574667 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156875008}
I20260709 12:57:07.585155 13690 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:07.576145 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.586928 13615 raft_consensus.cc:2393] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:07.589954 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.520056 15461 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:07.515156 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:07.548888 15832 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:07.582939 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:07.551664 15814 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:07.581889 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.594195 13691 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 3: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:07.566573 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:07.568140 15540 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:07.614024 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 121002us
I20260709 12:57:07.617672 13611 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:57:07.618379 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.618884 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.619040 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.619560 13611 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:07.623574 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 134749us
I20260709 12:57:07.572594 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:07.625160 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:07.625780 13691 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:07.577531 15828 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.908s	user 0.001s	sys 0.006s
I20260709 12:57:07.629223 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.629951 13619 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:07.575394 15853 raft_consensus.cc:2764] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:57:07.640524 13613 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:07.572340 15849 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:07.571137 15763 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:07.645761 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:07.646584 13613 raft_consensus.cc:2410] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:57:07.649549 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 67790.5us
I20260709 12:57:07.650386 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.651010 15755 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:07.644755 14238 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:07.645048 15849 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:07.653970 15849 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:07.616627 15540 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:07.656955 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:07.657996 13584 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:07.582535 15584 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 12:57:07.588392 15847 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.628023 15861 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:57:07.662497 13534 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:07.665923 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1192) took 1886 ms. Trace:
I20260709 12:57:07.577844 15776 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:07.691341 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:07.692667 13540 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:57:07.667836 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:07.628084 15844 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:57:07.670429 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 103923us
W20260709 12:57:07.670548 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 154812us
I20260709 12:57:07.651538 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.581595 15764 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.676527 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:07.701015 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:07.624680 15822 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:57:07.659350 15842 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:57:07.694610 14246 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:07.695788 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:07.696995 13508 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.580018 15789 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:07.716702 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 97906.4us
I20260709 12:57:07.705513 13613 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:07.707258 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:07.672649 15761 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:07.651592 15755 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.670084 15540 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:07.692591 13691 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:07.705709 13619 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:07.718111 13542 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:07.682085 13684 rpcz_store.cc:276] 0709 12:57:05.779192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.338278 (+1559086us) service_pool.cc:224] Handling call
0709 12:57:07.338880 (+   602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:07.339350 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:07.339390 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:07.341322 (+  1932us) log.cc:844] Serialized 26 byte log entry
0709 12:57:07.519241 (+177919us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:07.519294 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.519363 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:07.665659 (+146296us) raft_consensus.cc:1685] finished
0709 12:57:07.665688 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:07.665844 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12817,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":8485,"thread_start_us":177409,"threads_started":1,"wal-append.queue_time_us":289260}
I20260709 12:57:07.717846 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:07.719332 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 73220.6us
I20260709 12:57:07.616143 15565 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.694083 15770 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.708442 15853 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:07.724233 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.773538 13543 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:07.717636 15861 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:07.776556 15767 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:07.777494 15847 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.779336 15755 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.779773 15761 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:57:07.769479 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 118008us
I20260709 12:57:07.807523 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:07.809118 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.784993 13690 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:07.810627 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:07.775514 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:07.679800 15833 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:07.723862 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:07.830574 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 173679us
W20260709 12:57:07.835041 14211 connection.cc:256] RPC call timeout handler was delayed by -1.2398835269996198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 12:57:07.799000 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 74832.5us
I20260709 12:57:07.788604 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:07.799314 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 81577.5us
I20260709 12:57:07.838768 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.839607 13542 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:07.772256 15470 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:07.622678 15832 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.810361 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.724661 15663 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:07.815822 13690 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:07.816516 15584 raft_consensus.cc:2764] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:57:07.816735 15770 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:07.825166 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:07.826408 13537 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:57:07.702373 15461 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.788123 15861 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:07.840641 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 116860us
I20260709 12:57:07.781757 15844 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:07.863049 15534 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:07.787470 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.922931 13614 raft_consensus.cc:2376] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5. Re-sending same reply.
I20260709 12:57:07.801548 15767 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:07.804349 15822 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:57:07.926277 13686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2547) took 2193 ms (client timeout 1662 ms). Trace:
W20260709 12:57:07.926757 13686 rpcz_store.cc:269] 0709 12:57:05.731183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.764923 (+ 33740us) service_pool.cc:224] Handling call
0709 12:57:07.923863 (+2158940us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540012f054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.924013 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686689536}
I20260709 12:57:07.844566 13611 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:07.803701 15801 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:07.846784 13508 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:07.840368 15833 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:07.850822 14237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 393) took 2076 ms (client timeout 1558 ms). Trace:
I20260709 12:57:07.842708 15810 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:07.851945 13693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 263) took 2233 ms (client timeout 2247 ms). Trace:
W20260709 12:57:07.928735 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 82050.9us
W20260709 12:57:07.928887 13693 rpcz_store.cc:269] 0709 12:57:05.618670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.332169 (+713499us) service_pool.cc:224] Handling call
0709 12:57:07.851761 (+1519592us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b540034e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.851872 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952440576}
W20260709 12:57:07.853497 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65032.9us
W20260709 12:57:07.930286 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.856475 15600 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:07.932145 15600 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:07.932586 15600 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:07.932763 13584 leader_election.cc:400] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
I20260709 12:57:07.933203 13584 leader_election.cc:403] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:07.858763 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 455) took 2378 ms. Trace:
I20260709 12:57:07.874495 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2557) took 2076 ms. Trace:
I20260709 12:57:07.786041 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:07.934827 15862 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4.
I20260709 12:57:07.934521 13690 rpcz_store.cc:276] 0709 12:57:05.786637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.815051 (+2028414us) service_pool.cc:224] Handling call
0709 12:57:07.815396 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.862308 (+ 46912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.862548 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.708122 15814 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:07.935822 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.842082 13506 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.887022 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:07.887445 13537 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
W20260709 12:57:07.938064 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 96072.5us
I20260709 12:57:07.888005 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2544) took 2281 ms. Trace:
I20260709 12:57:07.939975 13687 rpcz_store.cc:276] 0709 12:57:05.611463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.142999 (+1531536us) service_pool.cc:224] Handling call
0709 12:57:07.143445 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:07.265519 (+122074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:07.265553 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:07.311375 (+ 45822us) log.cc:844] Serialized 26 byte log entry
0709 12:57:07.504017 (+192642us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:07.504089 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.504188 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:07.887698 (+383510us) raft_consensus.cc:1685] finished
0709 12:57:07.887719 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:07.887900 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28948,"raft.run_cpu_time_us":45,"raft.run_wall_time_us":45,"thread_start_us":144875,"threads_started":1,"wal-append.queue_time_us":477957}
W20260709 12:57:07.890595 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:07.892011 15844 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:07.903208 15791 consensus_meta.cc:287] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.000s
I20260709 12:57:07.904047 15861 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.905833 13535 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:07.879215 15761 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:07.907413 15843 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:07.908303 15858 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:57:07.863286 15627 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 12:57:07.912112 15851 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 12:57:07.879689 15859 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:57:07.917038 15764 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.918355 15832 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:07.949159 15844 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:07.844666 15655 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:07.934911 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2409) took 1168 ms. Trace:
I20260709 12:57:07.880985 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:07.934333 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:07.981076 13542 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:07.923863 15461 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:07.927557 15534 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:07.956753 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 125022us
W20260709 12:57:07.961457 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 122769us
I20260709 12:57:07.961598 13538 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:57:07.962555 14243 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:07.963640 13686 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:07.946192 15810 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.954603 15791 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:07.942054 15822 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:07.968772 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:07.968878 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:07.973969 13691 raft_consensus.cc:2410] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:57:07.920138 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 109840us
I20260709 12:57:07.985626 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1195) took 2178 ms. Trace:
I20260709 12:57:07.941421 15413 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:07.991111 13686 rpcz_store.cc:276] 0709 12:57:05.806779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.945706 (+2138927us) service_pool.cc:224] Handling call
0709 12:57:07.963407 (+ 17701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.985397 (+ 21990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.985570 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:07.986276 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:07.970498 15801 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:57:08.002501 15763 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:07.993912 15866 ts_tablet_manager.cc:933] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:57:08.005872 15720 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:08.008061 15840 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.007727 15790 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:57:08.008689 15840 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:07.986176 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:07.934137 14242 rpcz_store.cc:276] 0709 12:57:05.479677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.480014 (+   337us) service_pool.cc:224] Handling call
0709 12:57:05.867676 (+387662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:07.518474 (+1650798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.857425 (+338951us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400631354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:07.858332 (+   907us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400631350. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.858506 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4609619584}
W20260709 12:57:07.988446 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 54218.5us
W20260709 12:57:07.928488 14237 rpcz_store.cc:269] 0709 12:57:05.774181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.788890 (+ 14709us) service_pool.cc:224] Handling call
0709 12:57:07.850521 (+2061631us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b5400686d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:07.850712 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217287552}
I20260709 12:57:07.993606 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:08.021812 14239 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:08.022794 15655 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:07.987836 15833 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:08.026126 15445 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.710s	user 0.012s	sys 0.000s
I20260709 12:57:08.026563 15600 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:07.981739 15627 consensus_queue.cc:1243] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer 0f34cda059124f839711dbef61bb2c28 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:57:07.984778 15776 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:07.991200 15470 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:08.004148 15872 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:08.011543 13541 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:08.013909 13659 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:08.017536 15761 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:57:08.017964 15534 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.018172 15873 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:07.980756 13612 rpcz_store.cc:276] 0709 12:57:06.766333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.813735 (+ 47402us) service_pool.cc:224] Handling call
0709 12:57:07.784636 (+970901us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:07.934842 (+150206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5245184}
W20260709 12:57:08.019587 15868 log.cc:927] Time spent T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:57:08.027177 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.046905 15810 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:08.048895 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:07.980247 15861 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:08.029886 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:07.984653 15729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.006s	sys 0.015s
I20260709 12:57:08.034515 14245 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:57:08.013010 15413 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:08.044061 13542 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
W20260709 12:57:08.041914 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:08.032344 15763 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:08.059170 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:08.062275 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:08.064726 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 72274.9us
I20260709 12:57:08.067883 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:08.068612 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2546) took 2393 ms. Trace:
I20260709 12:57:08.069111 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:08.070683 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:08.068961 13688 rpcz_store.cc:276] 0709 12:57:05.680455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.250829 (+1570374us) service_pool.cc:224] Handling call
0709 12:57:07.583174 (+332345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:07.584113 (+   939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:08.068428 (+484315us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54002396d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:08.068557 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763388160}
W20260709 12:57:08.060114 15729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.006s	sys 0.015s
I20260709 12:57:08.053565 15764 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:08.049816 15873 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.063902 15789 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:08.088172 15877 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:08.088650 13582 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:08.088495 15565 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:57:08.084003 14209 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:08.082625 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.054433 15840 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:08.061746 15853 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:08.095757 15850 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:08.096508 15853 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:08.078271 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:08.097323 15850 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:08.098206 13688 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:08.098592 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:08.039886 15777 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:57:08.099074 13584 leader_election.cc:400] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6.
I20260709 12:57:08.078984 15776 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:57:08.102159 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 178434us
I20260709 12:57:08.077365 15655 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:08.105892 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.106858 13584 leader_election.cc:403] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:08.111757 13538 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:08.112166 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:08.063689 15810 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 12:57:08.124290 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 52822.4us
I20260709 12:57:08.095285 15774 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:08.080161 15729 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:08.113675 15413 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.114705 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:08.116806 15600 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:08.117216 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:08.121063 13612 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:08.121996 15873 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.097211 15853 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:08.150774 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 69412.8us
I20260709 12:57:08.110445 15872 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.154153 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.127349 13688 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:08.107357 15764 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:08.144513 14242 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
W20260709 12:57:08.168502 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 60812us
I20260709 12:57:08.168982 15873 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:08.169381 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:08.171592 15761 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:08.104168 15850 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:08.173219 15729 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.173856 15850 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.175138 15850 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:08.109913 15789 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:08.079811 15859 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:57:08.079265 15849 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.184744 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:08.158078 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.087419 15540 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.186061 13614 raft_consensus.cc:2410] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:08.163627 15764 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.146420 15866 tablet_copy_client.cc:323] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:57:08.145226 15755 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6.
W20260709 12:57:08.134061 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 98858.3us
I20260709 12:57:08.157141 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:08.162918 15655 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:08.187484 13611 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:57:08.155344 15774 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:08.159338 15763 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:08.193405 13688 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:08.159832 15865 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.203127 13689 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:08.177358 15846 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:57:08.196846 15761 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:08.214892 13688 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:08.215772 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.217280 13688 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:08.208628 15859 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:57:08.219273 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 66616.7us
I20260709 12:57:08.206156 15879 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:08.204595 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.223131 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:08.202994 15600 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:57:08.222085 14239 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:08.196128 15789 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.228945 13584 leader_election.cc:400] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6.
I20260709 12:57:08.228399 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:08.249042 13584 leader_election.cc:403] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:08.237717 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.220270 15859 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:08.230723 13543 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:08.266346 13538 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:08.208561 15865 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.226265 13612 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:08.242689 15808 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.001s
W20260709 12:57:08.275844 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 55234.3us
I20260709 12:57:08.210242 15844 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.245436 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 048528e413ac4c67a1cfd3dfe70673c7 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:57:08.277961 15761 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.278379 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:08.216311 15767 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.232138 15540 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:08.246487 15470 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:08.298311 13688 raft_consensus.cc:2376] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4. Re-sending same reply.
I20260709 12:57:08.300904 13538 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:08.297758 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:08.244124 15801 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:08.232937 15600 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:57:08.319003 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.319905 13693 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:08.240262 15720 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:57:08.289319 15889 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:57:08.330364 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 99870.6us
W20260709 12:57:08.330394 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 182917us
I20260709 12:57:08.240249 15774 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:08.338620 15565 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:08.339140 15565 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.334681 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:08.282706 15888 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6.
I20260709 12:57:08.139967 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:57:08.294226 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 048528e413ac4c67a1cfd3dfe70673c7 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-048528e413ac4c67a1cfd3dfe70673c7
I20260709 12:57:08.303604 15755 consensus_queue.cc:1059] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:57:08.229398 15872 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.289018 15761 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.325456 15846 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:57:08.383287 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 64732.2us
I20260709 12:57:08.346359 15373 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:08.433674 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 51966.4us
I20260709 12:57:08.319777 15801 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.383378 15565 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.456378 15849 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:08.456923 15373 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.463783 15761 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.454764 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.328090 15862 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:57:08.476619 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:08.436746 13534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 386) took 2635 ms (client timeout 1556 ms). Trace:
I20260709 12:57:08.383095 15889 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.455565 15846 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:57:08.509377 14212 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:08.455587 15767 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:08.512876 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.513167 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.513706 13611 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:08.513897 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.515990 13615 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:08.513562 13614 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
W20260709 12:57:08.537809 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:08.511130 15565 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:08.329706 15859 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:08.524235 15886 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 12:57:08.519629 15881 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:57:08.490665 15600 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:57:08.511307 15861 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:57:08.568166 15889 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.570374 15853 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.572477 15865 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.573107 15865 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.573565 15865 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.575544 15865 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.525535 15887 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:08.576315 15887 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.576720 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:08.577376 15887 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:08.580318 15848 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.953s	user 0.005s	sys 0.000s
I20260709 12:57:08.503494 15720 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 12:57:08.509245 13534 rpcz_store.cc:269] 0709 12:57:05.764944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:05.770077 (+  5133us) service_pool.cc:224] Handling call
0709 12:57:06.519294 (+749217us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:08.376877 (+1857583us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54000b81d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:08.399806 (+ 22929us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1746914432}
I20260709 12:57:08.561173 15373 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.561798 15790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:45554 (request call id 18) took 6826 ms. Trace:
W20260709 12:57:08.579668 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:08.519611 15893 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:08.594290 13582 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:08.604738 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:08.599270 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1124) took 2521 ms. Trace:
I20260709 12:57:08.545677 15879 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.550452 15894 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:08.609163 15373 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.605386 15472 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.592941 15844 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:08.621031 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 236849us
W20260709 12:57:08.622609 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:08.640594 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:08.605899 13536 rpcz_store.cc:276] 0709 12:57:06.077988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:06.717141 (+639153us) service_pool.cc:224] Handling call
0709 12:57:06.724412 (+  7271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:07.095139 (+370727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:07.095168 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:07.115663 (+ 20495us) log.cc:844] Serialized 26 byte log entry
0709 12:57:07.487069 (+371406us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:07.615521 (+128452us) log.cc:844] Serialized 14 byte log entry
0709 12:57:07.671027 (+ 55506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.671156 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:08.579556 (+908400us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:08.583120 (+  3564us) raft_consensus.cc:1685] finished
0709 12:57:08.599065 (+ 15945us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:08.599207 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78680,"prepare.queue_time_us":209043,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":5210,"raft.queue_time_us":54759,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":18442,"spinlock_wait_cycles":3560832,"thread_start_us":393103,"threads_started":2,"wal-append.queue_time_us":414017}
I20260709 12:57:08.616681 15865 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:08.607157 15470 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:08.615327 15893 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.621901 15895 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:08.652101 15895 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:08.627000 13535 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:57:08.656776 15801 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.657522 15893 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.659468 15853 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.660187 13612 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:08.660367 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.655928 15895 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:08.645040 15472 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.620493 15846 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:08.660683 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.644019 15894 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:08.648146 15789 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:08.680521 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.684150 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.684804 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.685319 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.682910 15846 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:08.652925 15373 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.695362 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:08.685834 13534 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:08.698505 13690 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:08.663035 15865 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:08.707733 14242 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:08.709117 15571 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:08.696406 15853 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.709760 15853 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.710184 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.711000 13542 raft_consensus.cc:2376] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5. Re-sending same reply.
I20260709 12:57:08.713135 15894 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.713974 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.671193 15843 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:08.715003 15843 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.717249 14239 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:08.717324 15843 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:08.647776 15470 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.671123 15816 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.669196 15755 consensus_queue.cc:1243] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:57:08.728688 13542 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:08.701858 15846 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:08.788969 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.790089 13690 raft_consensus.cc:2410] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:57:08.791285 13540 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:57:08.800772 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 120382us
I20260709 12:57:08.678361 15801 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:57:08.731292 15879 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.604387 15790 rpcz_store.cc:276] 0709 12:57:01.735204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:02.110877 (+375673us) service_pool.cc:224] Handling call
0709 12:57:03.063742 (+952865us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b5800552bd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7ad70a 00007fa02e7ae76a 00007fa02e7ad91b 00007fa02e7b903c 00007fa02e7b86b3 00007fa033778d29 00007fa02ccf71d4 00007fa02ccf7162
0709 12:57:03.327816 (+264074us) log.cc:844] Serialized 499 byte log entry
0709 12:57:03.615502 (+287686us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b18003f8c20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:04.328406 (+712904us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b4c0001b200. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02ca7ea20 00007fa02ca8fa29 00007fa02ca9013f 00007fa02ca542ab 00007fa02ca81f3c 00007fa02ca823fe 00007fa02cd0f50b 00007fa02e70e381 00007fa02e70c26b 00007fa02e712d53 00007fa02e712cd9
0709 12:57:08.561693 (+4233287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25748,"raft.queue_time_us":33390,"raft.run_cpu_time_us":4749,"raft.run_wall_time_us":785544,"spinlock_wait_cycles":2042909056,"thread_start_us":168070,"threads_started":1,"wal-append.queue_time_us":252599,"wal-append.run_cpu_time_us":2293,"wal-append.run_wall_time_us":1011986}
I20260709 12:57:08.810660 15903 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.811084 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.834337 15903 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.835423 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.811344 15472 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:08.838860 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 277) took 1283 ms. Trace:
I20260709 12:57:08.728479 15816 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.840184 15843 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:08.840710 15843 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.842075 15816 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.842986 15843 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:08.844775 15850 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.000s
I20260709 12:57:08.832422 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.788725 15755 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:08.845579 13690 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:08.845711 15755 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:08.845580 13692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2551) took 3107 ms (client timeout 3326 ms). Trace:
W20260709 12:57:08.847201 13692 rpcz_store.cc:269] 0709 12:57:05.738263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.116033 (+1377770us) service_pool.cc:224] Handling call
0709 12:57:08.845381 (+1729348us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54001018d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:08.845516 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705823488}
I20260709 12:57:08.848717 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:08.704854 15789 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.807698 15763 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.839308 13691 rpcz_store.cc:276] 0709 12:57:07.555337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.810275 (+254938us) service_pool.cc:224] Handling call
0709 12:57:08.838525 (+1028250us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54002391d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:08.838789 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373200640}
I20260709 12:57:08.859171 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.863240 13539 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:08.863384 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:08.865736 13584 connection.cc:256] RPC call timeout handler was delayed by -1.0568992210000943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2816s for a response
I20260709 12:57:08.867434 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.868099 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.869362 13618 raft_consensus.cc:2368] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:08.869448 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.869678 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.870311 13616 raft_consensus.cc:2393] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:08.870867 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.850422 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.872056 13538 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:08.730713 15455 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:57:08.877619 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:08.878916 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.804250 15853 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:08.875409 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:08.877702 13691 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:08.888314 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1197) took 2890 ms. Trace:
I20260709 12:57:08.888798 13691 rpcz_store.cc:276] 0709 12:57:05.998038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.877202 (+2879164us) service_pool.cc:224] Handling call
0709 12:57:08.877491 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:08.887999 (+ 10508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:08.888247 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:08.890988 15843 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:08.891566 15843 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:08.892105 15843 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:08.808993 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.894217 13535 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:08.895701 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:08.900525 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.884956 13619 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:08.885070 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 268) took 3073 ms. Trace:
I20260709 12:57:08.912384 13688 rpcz_store.cc:276] 0709 12:57:05.811667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.327684 (+2516017us) service_pool.cc:224] Handling call
0709 12:57:08.346392 (+ 18708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:08.346691 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:08.554066 (+207375us) log.cc:844] Serialized 14 byte log entry
0709 12:57:08.884576 (+330510us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:08.884653 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:08.884740 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:08.884829 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:08.884981 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2891,"prepare.queue_time_us":339406,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":10632,"raft.queue_time_us":3047,"thread_start_us":494483,"threads_started":2}
I20260709 12:57:08.880826 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:08.922590 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:08.925426 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 44785.8us
I20260709 12:57:08.926126 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:08.926909 13692 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:08.859787 15755 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:08.850854 15900 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:08.929699 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.929766 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:08.842988 15905 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:08.877764 15850 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:08.852089 15789 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:08.936931 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:08.954658 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:08.933238 13610 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:08.948840 13539 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:08.887564 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:08.886225 15790 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:08.886394 15889 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:08.856105 15763 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.966881 15763 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.812537 15865 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.957173 13658 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:08.924979 15453 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:08.956493 15900 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:08.972289 15453 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.972656 14237 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:57:08.974397 15453 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.975428 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:08.967686 15846 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:08.977037 15913 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:08.977857 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.979003 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.962285 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:08.963238 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:08.960053 15905 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:08.960237 15850 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:08.963963 13534 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:08.979749 13615 raft_consensus.cc:2393] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:08.961381 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:08.974290 15900 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:57:08.986600 13684 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:57:08.937573 15413 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:08.963603 15903 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.983448 14242 raft_consensus.cc:2410] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:57:09.018296 13611 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:08.991720 13688 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:57:09.019780 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 271) took 2918 ms. Trace:
I20260709 12:57:09.020262 13688 rpcz_store.cc:276] 0709 12:57:06.101463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.937909 (+2836446us) service_pool.cc:224] Handling call
0709 12:57:08.985888 (+ 47979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.019334 (+ 33446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.019658 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.001394 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:09.028748 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 153200us
I20260709 12:57:09.034089 13584 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:09.035349 15879 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:09.037906 15903 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:57:08.993939 15816 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:08.995747 15850 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:09.040169 13688 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:57:08.972697 15789 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:09.043906 13615 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:57:09.045178 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 272) took 2837 ms. Trace:
I20260709 12:57:09.000496 14239 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:08.983408 15859 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:08.977895 15846 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:08.985450 15470 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:09.046356 13659 leader_election.cc:400] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:09.057605 13659 leader_election.cc:403] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:57:08.979728 15913 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:57:09.045610 13688 rpcz_store.cc:276] 0709 12:57:06.208342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.021373 (+2813031us) service_pool.cc:224] Handling call
0709 12:57:09.038668 (+ 17295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:09.044912 (+  6244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.045122 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.049612 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 270) took 2986 ms. Trace:
I20260709 12:57:08.983013 15843 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:09.053094 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 123397us
I20260709 12:57:09.065589 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:09.054457 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:09.060318 13691 rpcz_store.cc:276] 0709 12:57:06.063676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.890037 (+2826361us) service_pool.cc:224] Handling call
0709 12:57:08.890393 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:08.890675 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:08.945730 (+ 55055us) log.cc:844] Serialized 14 byte log entry
0709 12:57:09.049257 (+103527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:09.049316 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:09.049371 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.049434 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:09.049556 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33286,"prepare.queue_time_us":102982,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":100,"thread_start_us":103123,"threads_started":1}
I20260709 12:57:09.071755 13691 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:09.055251 13611 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:57:09.089048 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2567) took 2486 ms. Trace:
I20260709 12:57:09.089607 13691 rpcz_store.cc:276] 0709 12:57:06.586497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.069699 (+2483202us) service_pool.cc:224] Handling call
0709 12:57:09.071587 (+  1888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.072275 (+   688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.072477 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.091427 13691 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:09.092623 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2568) took 2505 ms. Trace:
W20260709 12:57:09.093236 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47087.7us
I20260709 12:57:09.093276 13691 rpcz_store.cc:276] 0709 12:57:06.586769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.090616 (+2503847us) service_pool.cc:224] Handling call
0709 12:57:09.090975 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.092297 (+  1322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.092554 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.094916 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:09.095872 13691 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:57:09.069530 13612 raft_consensus.cc:2410] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:09.058754 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:09.096489 13691 pending_rounds.cc:85] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:57:09.088814 15894 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:57:09.088896 15900 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:09.101491 15844 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:57:09.058459 15789 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:09.102852 13691 pending_rounds.cc:107] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:57:09.103410 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:09.127482 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:09.089170 15877 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:09.109897 15894 consensus_queue.cc:1243] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:57:09.143250 15843 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:09.144773 15911 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
W20260709 12:57:09.152626 15908 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:57:09.153776 15789 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.106128 15373 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:09.138490 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:09.140460 15903 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.091642 15470 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:09.157065 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:09.168380 13537 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:09.169375 13537 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:09.168293 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:09.139210 13691 raft_consensus.cc:2889] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:09.171545 13611 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:09.171658 13659 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:09.139834 15763 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:09.181037 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 278) took 1625 ms. Trace:
I20260709 12:57:09.185245 13691 rpcz_store.cc:276] 0709 12:57:07.555643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.094945 (+1539302us) service_pool.cc:224] Handling call
0709 12:57:09.095348 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.177076 (+ 81728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.180980 (+  3904us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.188777 13611 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:09.165294 15843 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:09.174481 15764 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:09.199154 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:09.193257 13537 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:09.179273 15877 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:09.205157 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:09.168171 15903 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.211347 13537 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:09.195637 15470 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:09.207655 15895 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.005s
I20260709 12:57:09.222793 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:09.223794 13691 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:09.231335 13537 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:09.232101 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 400) took 1321 ms. Trace:
I20260709 12:57:09.204475 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:09.202239 15843 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:09.232416 13537 rpcz_store.cc:276] 0709 12:57:07.910546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.947047 (+ 36501us) service_pool.cc:224] Handling call
0709 12:57:07.961200 (+ 14153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.168274 (+1207074us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54000265a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 12:57:09.231761 (+ 63487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.232035 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344285056}
W20260709 12:57:09.233686 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 62162.2us
I20260709 12:57:09.234513 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:09.240749 13611 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 2.
I20260709 12:57:09.240582 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:09.242874 13537 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:09.245016 13537 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:57:09.245119 15891 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.857s	user 0.001s	sys 0.003s
W20260709 12:57:09.239584 15663 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:57:09.249904 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:09.249750 15843 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.220429 15877 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:09.252806 15877 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:09.256366 15470 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:09.258194 15914 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:57:09.257916 13612 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:09.267040 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:09.233533 15895 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.269555 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2566) took 2683 ms. Trace:
I20260709 12:57:09.534408 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2563) took 3661 ms. Trace:
I20260709 12:57:09.409170 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1228) took 1732 ms. Trace:
I20260709 12:57:09.533531 13688 rpcz_store.cc:276] 0709 12:57:06.586190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.066432 (+2480242us) service_pool.cc:224] Handling call
0709 12:57:09.110154 (+ 43722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.110915 (+   761us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:09.167761 (+ 56846us) log.cc:844] Serialized 14 byte log entry
0709 12:57:09.269050 (+101289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:09.269135 (+    85us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:09.269216 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.269310 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:09.269477 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27645,"prepare.queue_time_us":56814,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":4218,"raft.queue_time_us":32395,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":86,"thread_start_us":47195,"threads_started":1}
I20260709 12:57:09.535181 13684 rpcz_store.cc:276] 0709 12:57:05.873318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.856254 (+2982936us) service_pool.cc:224] Handling call
0709 12:57:08.856646 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:09.006486 (+149840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.534122 (+527636us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540042ccd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:09.534348 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298168192}
I20260709 12:57:09.388387 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:09.371052 13613 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:57:09.538357 13613 pending_rounds.cc:85] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 6
I20260709 12:57:09.538769 13613 pending_rounds.cc:107] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:57:09.539494 13613 raft_consensus.cc:2889] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:09.382050 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2414) took 1239 ms. Trace:
I20260709 12:57:09.387446 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:09.383540 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:09.541515 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:09.375524 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1203) took 1391 ms. Trace:
I20260709 12:57:09.542718 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:09.378723 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1193) took 3588 ms. Trace:
I20260709 12:57:09.541481 13619 rpcz_store.cc:276] 0709 12:57:08.143088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.143261 (+   173us) service_pool.cc:224] Handling call
0709 12:57:08.158178 (+ 14917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:08.912277 (+754099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.380878 (+468601us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b540026b9a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b5a02 00007fa02e7b1dd6 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:57:09.381548 (+   670us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b540013bfd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:09.381961 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702898048}
W20260709 12:57:09.539117 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 118270us
I20260709 12:57:09.313408 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:09.544946 13693 rpcz_store.cc:276] 0709 12:57:05.790521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.942416 (+2151895us) service_pool.cc:224] Handling call
0709 12:57:07.942667 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:08.320393 (+377726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.378153 (+1057760us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b5400244854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:09.378624 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867113728}
I20260709 12:57:09.414058 13618 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:09.393198 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:09.551534 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:09.538590 13617 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:09.535391 13541 rpcz_store.cc:276] 0709 12:57:07.677675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.693615 (+ 15940us) service_pool.cc:224] Handling call
0709 12:57:07.711488 (+ 17873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:08.038078 (+326590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.408700 (+1370622us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54003ca354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:09.409079 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689617152}
I20260709 12:57:09.240484 15764 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:09.583865 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:09.357477 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:09.549126 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:09.543375 13542 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:57:09.587189 13542 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:57:09.589936 13542 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:09.542976 13686 rpcz_store.cc:276] 0709 12:57:07.984153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.011246 (+ 27093us) service_pool.cc:224] Handling call
0709 12:57:09.375249 (+1364003us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b54002968d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:09.375443 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021110656}
I20260709 12:57:09.295321 13659 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:09.592039 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 296818us
I20260709 12:57:09.568157 13615 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:57:09.598620 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49609.8us
I20260709 12:57:09.602876 13616 raft_consensus.cc:2368] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5.
I20260709 12:57:09.375370 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:09.384876 15863 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.663s	user 0.004s	sys 0.000s
W20260709 12:57:09.273644 15925 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:57:09.296169 15930 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:57:09.596899 13619 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:09.305224 15903 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:09.364521 15932 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:57:09.302119 15922 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:57:09.598095 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:09.670130 14212 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:09.602808 13506 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 3.326s (SENT) [suppressed 23 similar messages]
I20260709 12:57:09.546486 13538 raft_consensus.cc:2410] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:09.362504 15888 raft_consensus.cc:3055] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:09.270475 15761 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:09.608667 13684 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:57:09.714696 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 280) took 2134 ms. Trace:
I20260709 12:57:09.715039 13684 rpcz_store.cc:276] 0709 12:57:07.580280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.582203 (+2001923us) service_pool.cc:224] Handling call
0709 12:57:09.608343 (+ 26140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.714412 (+106069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.714636 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:09.605086 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 229808us
I20260709 12:57:09.716616 13582 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:09.369120 15847 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:09.366248 15791 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:09.369673 15717 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:09.295167 13612 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:09.613937 13693 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:09.725123 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2573) took 1728 ms. Trace:
I20260709 12:57:09.725834 13693 rpcz_store.cc:276] 0709 12:57:07.996470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.596513 (+1600043us) service_pool.cc:224] Handling call
0709 12:57:09.613733 (+ 17220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.724773 (+111040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.725049 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.301973 15763 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.348277 15846 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.614836 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1191) took 3844 ms. Trace:
I20260709 12:57:09.723860 13509 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:09.384521 15767 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:09.739360 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 269) took 3880 ms. Trace:
I20260709 12:57:09.739523 15767 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.724467 15859 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:09.741086 15859 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:09.741521 15859 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:09.296072 15844 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.335s
I20260709 12:57:09.624061 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:57:09.592756 15850 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.001s
W20260709 12:57:09.746070 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 122128us
I20260709 12:57:09.739787 13690 rpcz_store.cc:276] 0709 12:57:05.859312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.846354 (+2987042us) service_pool.cc:224] Handling call
0709 12:57:08.846689 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:08.877252 (+ 30563us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:08.922453 (+ 45201us) log.cc:844] Serialized 14 byte log entry
0709 12:57:09.568652 (+646199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:09.568731 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:09.568803 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.739113 (+170310us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5400238cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:09.739159 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:09.739300 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":283,"prepare.queue_time_us":46244,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":22332,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":6812,"spinlock_wait_cycles":67096448,"thread_start_us":645475,"threads_started":2,"wal-append.queue_time_us":343016}
I20260709 12:57:09.622030 15816 consensus_queue.cc:1059] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:57:09.642468 13691 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:57:09.588117 15600 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.589998 15849 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.600895 15534 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.602245 15905 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:09.614135 15887 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:57:09.598243 15655 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:57:09.625869 13553 tablet_copy_source_session.cc:215] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:57:09.612643 15840 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:57:09.623588 15622 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:57:09.615742 15894 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:09.611290 15928 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:09.611171 15531 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:09.610913 15593 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.651335 15873 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.379119 15865 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.254s
I20260709 12:57:09.615970 15923 consensus_queue.cc:1059] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:57:09.660969 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:09.661806 13489 auto_rebalancer.cc:629] Failed to schedule move for tablet 5406eb61fd534169b3c1645f80144779: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:57:09.662254 13692 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:57:09.834612 13692 pending_rounds.cc:85] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 5
I20260709 12:57:09.835219 13692 pending_rounds.cc:107] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:57:09.662850 15895 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:09.613173 15663 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:57:09.273962 15729 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.360s
I20260709 12:57:09.698873 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1236) took 1101 ms. Trace:
I20260709 12:57:09.856130 13538 rpcz_store.cc:276] 0709 12:57:08.597745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.874908 (+277163us) service_pool.cc:224] Handling call
0709 12:57:09.366997 (+492089us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:09.698670 (+331673us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5550336}
I20260709 12:57:09.271245 15898 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.359s
I20260709 12:57:09.722184 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2558) took 3904 ms. Trace:
I20260709 12:57:09.597846 15764 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.735199 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:09.736671 15763 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.737991 15846 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.738094 15777 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:57:09.738291 13693 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:57:09.738806 15833 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:09.742692 15801 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:09.712248 15888 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:09.744724 13619 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:57:09.638067 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:09.749231 13684 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:57:09.927556 13693 pending_rounds.cc:85] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 3
I20260709 12:57:09.800846 15849 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:09.751363 13691 pending_rounds.cc:85] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 1
I20260709 12:57:09.935549 13691 pending_rounds.cc:107] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:57:09.936326 13691 raft_consensus.cc:2889] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:09.732695 13685 rpcz_store.cc:276] 0709 12:57:05.770589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.300192 (+1529603us) service_pool.cc:224] Handling call
0709 12:57:07.300740 (+   548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:07.324645 (+ 23905us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:07.516781 (+192136us) log.cc:844] Serialized 14 byte log entry
0709 12:57:07.616057 (+ 99276us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:07.630081 (+ 14024us) log.cc:844] Serialized 26 byte log entry
0709 12:57:07.630238 (+   157us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:07.630268 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:07.630332 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:09.384066 (+1753734us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:09.614515 (+230449us) raft_consensus.cc:1685] finished
0709 12:57:09.614610 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:09.614756 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":248602,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":256,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":3954,"spinlock_wait_cycles":112304000,"thread_start_us":180273,"threads_started":2,"wal-append.queue_time_us":97599}
W20260709 12:57:09.742447 15930 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:57:09.722121 15847 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:09.727135 15903 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.732924 15717 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:09.945495 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 275468us
I20260709 12:57:09.797173 13690 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:57:09.801964 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 349) took 1289 ms. Trace:
I20260709 12:57:09.742216 15767 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:09.819583 13659 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:09.762224 15926 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:09.835529 13692 raft_consensus.cc:2889] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:09.813144 15936 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:09.860617 13542 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:57:09.989140 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 169646us
I20260709 12:57:09.862288 13553 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:45566 (request call id 405) took 1618 ms. Trace:
I20260709 12:57:09.865091 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1232) took 1268 ms. Trace:
I20260709 12:57:10.002362 13535 rpcz_store.cc:276] 0709 12:57:08.596780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.920411 (+323631us) service_pool.cc:224] Handling call
0709 12:57:08.921273 (+   862us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:09.098677 (+177404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:09.098721 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:09.099320 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 12:57:09.554924 (+455604us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:09.554961 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.555023 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:09.864786 (+309763us) raft_consensus.cc:1685] finished
0709 12:57:09.864816 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:09.865010 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8861,"raft.queue_time_us":68197,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":11079,"thread_start_us":582166,"threads_started":2,"wal-append.queue_time_us":736443}
I20260709 12:57:09.824028 15840 consensus_queue.cc:1243] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer 0f34cda059124f839711dbef61bb2c28 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:57:09.826506 15928 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:09.928951 13693 pending_rounds.cc:107] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:57:10.014285 13693 raft_consensus.cc:2889] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:09.929173 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2577) took 1930 ms. Trace:
I20260709 12:57:10.015949 13684 rpcz_store.cc:276] 0709 12:57:07.999206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.722494 (+1723288us) service_pool.cc:224] Handling call
0709 12:57:09.748762 (+ 26268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.928857 (+180095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.929101 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.750341 13686 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:57:09.854413 15593 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.017575 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2579) took 2017 ms. Trace:
I20260709 12:57:09.801295 15534 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:09.794337 15600 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.022354 13688 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:57:10.024322 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 421527us
W20260709 12:57:10.025295 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:10.027989 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2578) took 2028 ms. Trace:
I20260709 12:57:10.028647 13693 rpcz_store.cc:276] 0709 12:57:07.999494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.736936 (+1737442us) service_pool.cc:224] Handling call
0709 12:57:09.737911 (+   975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.027603 (+289692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.027884 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.946549 15790 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:09.764065 15850 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:09.939209 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 204111us
I20260709 12:57:09.871675 15865 consensus_queue.cc:1243] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:57:09.764045 13537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.007s	sys 0.024s
W20260709 12:57:10.056917 13537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.008s	sys 0.024s
I20260709 12:57:09.946429 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 282) took 1888 ms. Trace:
I20260709 12:57:10.058203 13690 rpcz_store.cc:276] 0709 12:57:08.058380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.748354 (+1689974us) service_pool.cc:224] Handling call
0709 12:57:09.796958 (+ 48604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:09.946140 (+149182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.946369 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.869961 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:09.987727 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:09.827490 15894 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:09.997831 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:09.855391 15873 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:09.996263 13613 rpcz_store.cc:276] 0709 12:57:08.512184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.513548 (+  1364us) service_pool.cc:224] Handling call
0709 12:57:08.526867 (+ 13319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.586118 (+1059251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.801669 (+215551us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b540020a154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:09.801882 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940483328}
I20260709 12:57:10.012804 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2576) took 2014 ms. Trace:
I20260709 12:57:10.018152 13686 rpcz_store.cc:276] 0709 12:57:08.000297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.745079 (+1744782us) service_pool.cc:224] Handling call
0709 12:57:09.745432 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.017215 (+271783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.017504 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:10.019873 13535 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:10.083181 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.083511 13535 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:10.030377 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.084375 13535 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.085496 13535 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:10.031038 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2575) took 2034 ms. Trace:
I20260709 12:57:10.086162 13691 rpcz_store.cc:276] 0709 12:57:07.997044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.641447 (+1644403us) service_pool.cc:224] Handling call
0709 12:57:09.641866 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.030690 (+388824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.030960 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:10.087656 13691 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:10.088392 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2584) took 1417 ms. Trace:
I20260709 12:57:10.088821 13691 rpcz_store.cc:276] 0709 12:57:08.671171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.087090 (+1415919us) service_pool.cc:224] Handling call
0709 12:57:10.087355 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.088111 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.088322 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:09.948041 15833 consensus_queue.cc:1243] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:57:09.944501 13540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1141) took 1300 ms (client timeout 1577 ms). Trace:
I20260709 12:57:09.987768 15926 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:10.038699 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:09.989040 15887 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:10.113530 13690 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:10.114903 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1206) took 1531 ms. Trace:
I20260709 12:57:10.018090 15767 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.010027 15936 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:09.942025 15470 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:09.929598 15777 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:10.132368 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:10.022584 15840 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.021692 15928 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:09.985942 15903 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:09.927604 13689 rpcz_store.cc:276] 0709 12:57:05.818139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.381539 (+2563400us) service_pool.cc:224] Handling call
0709 12:57:08.441672 (+ 60133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:08.475274 (+ 33602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:08.475324 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:08.858214 (+382890us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b2c0006a7c8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:57:08.858610 (+   396us) log.cc:844] Serialized 26 byte log entry
0709 12:57:09.044035 (+185425us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:09.044091 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.044284 (+   193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:09.667808 (+623524us) raft_consensus.cc:1685] finished
0709 12:57:09.667837 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:09.722113 (+ 54276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1143,"raft.queue_time_us":24564,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":8971,"spinlock_wait_cycles":6715776,"thread_start_us":184935,"threads_started":1,"wal-append.queue_time_us":306277}
I20260709 12:57:10.082293 13692 rpcz_store.cc:276] 0709 12:57:07.998045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.644988 (+1646943us) service_pool.cc:224] Handling call
0709 12:57:09.661792 (+ 16804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.001745 (+339953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.012689 (+ 10944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:10.103439 15833 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.094170 13614 raft_consensus.cc:2368] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7.
I20260709 12:57:10.033461 15789 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:57:10.095283 13540 rpcz_store.cc:269] 0709 12:57:08.643840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.685082 (+ 41242us) service_pool.cc:224] Handling call
0709 12:57:09.944254 (+1259172us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b540028b754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:09.944420 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222154624}
I20260709 12:57:10.037127 15937 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:10.100831 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 191423us
I20260709 12:57:10.212124 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.115442 13690 rpcz_store.cc:276] 0709 12:57:08.583245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.084406 (+1501161us) service_pool.cc:224] Handling call
0709 12:57:10.113326 (+ 28920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.114294 (+   968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.114809 (+   515us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:10.073966 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 86307.3us
W20260709 12:57:10.126374 13616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1350) took 1252 ms (client timeout 1550 ms). Trace:
I20260709 12:57:10.078938 15729 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.080992 15898 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.162007 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.101925 15790 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.162017 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:10.163151 15777 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:10.164100 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:10.106222 15926 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:10.180670 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:10.130071 15801 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.100s
W20260709 12:57:10.180703 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 102732us
W20260709 12:57:10.182380 13659 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.577s (SENT)
W20260709 12:57:10.270737 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 88387.6us
I20260709 12:57:10.076337 13685 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:10.206610 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.206610 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:10.274415 13540 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:10.207494 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.208074 13535 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:57:10.275449 13535 pending_rounds.cc:85] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 5
I20260709 12:57:10.275704 13535 pending_rounds.cc:107] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:57:10.120927 15887 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:10.209580 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 114911us
I20260709 12:57:10.280596 13693 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:57:10.213495 13616 rpcz_store.cc:269] 0709 12:57:08.873525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.876489 (+  2964us) service_pool.cc:224] Handling call
0709 12:57:10.126120 (+1249631us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54002dffd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:10.126305 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561707776}
I20260709 12:57:10.113675 15877 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:10.002215 13553 rpcz_store.cc:276] 0709 12:57:08.244326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.245153 (+   827us) service_pool.cc:224] Handling call
0709 12:57:09.625319 (+1380166us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54002846d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7bc3f7 00007fa033752036 00007fa03375ba3e 00007fa02b01f27b 00007fa03375b9ba 00007fa033756acc 00007fa033751c7b 00007fa0337468d1 00007fa0327fa4b4 00007fa0327fa442
0709 12:57:09.862116 (+236797us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659251968}
I20260709 12:57:10.212903 14244 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:10.218801 13689 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:57:10.105605 15884 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:57:10.199168 15840 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.240465 14246 raft_consensus.cc:2368] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 2. Current term is 4.
I20260709 12:57:10.248801 13690 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:10.037943 15888 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:10.180065 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 146024us
I20260709 12:57:10.211509 15937 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:10.076112 15717 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.276058 13535 raft_consensus.cc:2889] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:10.157372 15767 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.289906 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.228247 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.296723 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 286) took 1109 ms. Trace:
I20260709 12:57:10.318367 13689 rpcz_store.cc:276] 0709 12:57:09.186980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.185179 (+998199us) service_pool.cc:224] Handling call
0709 12:57:10.218058 (+ 32879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.296345 (+ 78287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.296652 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:10.228344 15966 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.336138 15936 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.335848 15966 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.336522 15534 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.337082 15776 raft_consensus.cc:3055] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:10.337519 15776 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:10.338196 15791 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:57:10.182085 15470 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.338573 15776 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.340319 15470 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.341522 15593 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.342684 15593 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:10.343160 15593 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.343564 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1209) took 1726 ms. Trace:
I20260709 12:57:10.344007 13693 rpcz_store.cc:276] 0709 12:57:08.617516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.084793 (+1467277us) service_pool.cc:224] Handling call
0709 12:57:10.275062 (+190269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.281288 (+  6226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.343344 (+ 62056us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b540031e050. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:10.343496 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436500864}
I20260709 12:57:10.301770 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:10.082515 15945 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.131s
I20260709 12:57:10.256820 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:10.276746 15942 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 12:57:10.307773 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 287) took 1115 ms. Trace:
I20260709 12:57:10.362226 13690 rpcz_store.cc:276] 0709 12:57:09.192026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.213141 (+1021115us) service_pool.cc:224] Handling call
0709 12:57:10.247773 (+ 34632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.307442 (+ 59669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.307713 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:10.257519 14240 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:10.313534 13685 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:10.230989 15729 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.318936 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.321856 15763 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:10.322336 15956 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:57:10.323180 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.324165 15814 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.325636 15947 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:10.326287 15903 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.326943 15849 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.327667 13684 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:10.387486 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 285) took 1202 ms. Trace:
I20260709 12:57:10.387853 13684 rpcz_store.cc:276] 0709 12:57:09.184849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.185179 (+1000330us) service_pool.cc:224] Handling call
0709 12:57:10.327290 (+142111us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:10.327470 (+   180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.387222 (+ 59752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.387427 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5438592}
I20260709 12:57:10.328053 15764 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.315593 15976 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:57:10.240286 15898 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.217666 15833 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.329594 15963 raft_consensus.cc:2764] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:57:10.332373 15600 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.418285 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.420089 15764 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.420733 14242 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:57:10.420635 15764 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.421137 14242 pending_rounds.cc:85] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 6
I20260709 12:57:10.421411 14242 pending_rounds.cc:107] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:57:10.423067 15764 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.424530 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:10.425711 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:10.428073 15969 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:10.428403 15958 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.428541 15969 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:10.428936 15969 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:10.429401 15761 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:10.432117 15808 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.427001 15777 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.332754 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.433954 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.434263 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.434669 14238 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
W20260709 12:57:10.333925 15943 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 12:57:10.435603 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.334106 15627 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:10.436805 15627 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:10.436983 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.437767 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:10.437713 15627 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.438813 15627 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:10.334295 15767 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.440200 15767 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.440647 15767 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.443041 15767 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:10.334914 15948 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:57:10.335309 15873 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.346014 13689 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:10.287711 15887 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:10.446476 15729 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.446935 15847 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.449594 13542 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:10.449977 13542 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:10.450006 15840 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.354859 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:10.358156 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 144548us
I20260709 12:57:10.452605 13617 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:57:10.393918 15944 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.365s	user 0.000s	sys 0.002s
I20260709 12:57:10.256496 15790 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.368927 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 290) took 1093 ms. Trace:
I20260709 12:57:10.462744 13685 rpcz_store.cc:276] 0709 12:57:09.275256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.274579 (+999323us) service_pool.cc:224] Handling call
0709 12:57:10.313153 (+ 38574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.368621 (+ 55468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.368860 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:10.383101 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.463234 14243 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:10.350749 15776 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:10.390249 13690 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:10.338665 15966 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.392884 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.360996 15791 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.403833 13686 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:57:10.301787 15877 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.408917 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1202) took 2832 ms. Trace:
I20260709 12:57:10.397787 15814 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.481808 13688 rpcz_store.cc:276] 0709 12:57:07.576983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.582192 (+2005209us) service_pool.cc:224] Handling call
0709 12:57:09.607747 (+ 25555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.022762 (+415015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.408689 (+385927us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5400002154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:10.408856 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952609024}
I20260709 12:57:10.385299 15947 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:10.332928 15941 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 12:57:10.117022 15663 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.110s
I20260709 12:57:10.454502 13684 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:57:10.345423 15937 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:10.464028 13540 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:10.516422 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.518009 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:10.469170 13614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1336) took 1585 ms (client timeout 1577 ms). Trace:
I20260709 12:57:10.467646 13535 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [3.6-5.7]
W20260709 12:57:10.518774 13614 rpcz_store.cc:269] 0709 12:57:08.866943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.867094 (+   151us) service_pool.cc:224] Handling call
0709 12:57:10.452077 (+1584983us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540020f654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:10.452366 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2797467392}
I20260709 12:57:10.421665 14242 raft_consensus.cc:2889] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:10.394286 15846 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.432649 15808 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.450784 13542 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:10.525990 13542 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:10.486600 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.469730 15853 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:57:10.494040 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1147) took 1839 ms. Trace:
I20260709 12:57:10.494890 15877 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.474123 15847 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:10.500190 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.501318 15777 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:10.501276 15859 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.759s	user 0.010s	sys 0.004s
W20260709 12:57:10.502028 15953 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.201s	user 0.000s	sys 0.002s
I20260709 12:57:10.504536 15947 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.505148 15898 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.478798 15963 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:57:10.505746 15884 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:10.509768 15814 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.462131 15850 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.463196 15593 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.519613 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.473711 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.523697 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.527004 14245 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:57:10.428893 15958 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.528946 13537 rpcz_store.cc:276] 0709 12:57:08.654901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.245939 (+591038us) service_pool.cc:224] Handling call
0709 12:57:09.246277 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:09.246933 (+   656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:09.246964 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:09.247573 (+   609us) log.cc:844] Serialized 52 byte log entry
0709 12:57:10.057543 (+809970us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:10.112789 (+ 55246us) log.cc:844] Serialized 14 byte log entry
0709 12:57:10.113004 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.113078 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:10.493763 (+380685us) raft_consensus.cc:1685] finished
0709 12:57:10.493782 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:10.493979 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42601,"prepare.queue_time_us":180234,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":210260,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":194,"thread_start_us":809494,"threads_started":1,"wal-append.queue_time_us":774495}
I20260709 12:57:10.539181 15790 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:10.541939 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 260808us
I20260709 12:57:10.488312 15791 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.526741 15945 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.528648 15894 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.600853 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 412) took 2004 ms. Trace:
I20260709 12:57:10.615881 13535 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:10.580161 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:10.651683 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.647259 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2430) took 1419 ms. Trace:
I20260709 12:57:10.662300 13535 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:10.545192 15853 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:10.541623 15600 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.663596 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.678903 13536 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:57:10.644196 15972 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:10.579967 14238 raft_consensus.cc:2410] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:10.664721 14245 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:10.661345 13610 rpcz_store.cc:276] 0709 12:57:09.228201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.542050 (+313849us) service_pool.cc:224] Handling call
0709 12:57:09.542347 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:09.605558 (+ 63211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:09.605611 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:09.606287 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 12:57:10.035200 (+428913us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:10.035263 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.035353 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:10.611477 (+576124us) raft_consensus.cc:1685] finished
0709 12:57:10.611508 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:10.647171 (+ 35663us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48325,"raft.queue_time_us":50452,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":13696,"thread_start_us":319034,"threads_started":1,"wal-append.queue_time_us":587307}
W20260709 12:57:10.686455 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 207107us
I20260709 12:57:10.616447 13543 rpcz_store.cc:276] 0709 12:57:08.596202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.598148 (+  1946us) service_pool.cc:224] Handling call
0709 12:57:09.368362 (+770214us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:09.670949 (+302587us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:09.671224 (+   275us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:09.711068 (+ 39844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:09.711119 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:09.744712 (+ 33593us) log.cc:844] Serialized 52 byte log entry
0709 12:57:09.906700 (+161988us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:09.942273 (+ 35573us) log.cc:844] Serialized 14 byte log entry
0709 12:57:10.079425 (+137152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.079513 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:10.435057 (+355544us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b1800424060. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:10.600464 (+165407us) raft_consensus.cc:1685] finished
0709 12:57:10.600593 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:10.600783 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34625,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":94091,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":328,"spinlock_wait_cycles":52013952,"thread_start_us":298289,"threads_started":2,"wal-append.queue_time_us":338268}
I20260709 12:57:10.664575 14246 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:10.645529 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:10.674743 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:10.558593 15859 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:10.692302 15859 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:57:10.693010 15859 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:10.693427 15859 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:10.694278 15859 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.695534 15859 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:10.695638 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.696332 13685 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:10.697979 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.589936 15957 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.698545 13685 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:10.698644 15957 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.700133 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.700703 13685 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:10.674443 15789 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.656626 13542 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:10.673949 15877 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.606375 15898 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
W20260709 12:57:10.702358 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56931.6us
I20260709 12:57:10.671830 15808 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.703558 13684 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:10.588924 15947 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:10.674284 15764 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.703579 15968 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:10.686002 15972 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.715077 15972 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.606995 15962 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:57:10.723995 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.571051 15969 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:10.701973 15981 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:57:10.671556 15850 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:57:10.740634 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:10.741463 13689 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:10.726056 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 211749us
W20260709 12:57:10.681947 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 412020us
I20260709 12:57:10.724884 13684 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:10.648453 15945 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.747702 13614 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:10.760298 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:10.762778 13614 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:10.792575 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 289) took 1578 ms. Trace:
I20260709 12:57:10.792834 15847 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:10.794618 15958 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:10.794857 15945 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.709146 15877 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.795989 15991 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.726126 15853 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.797714 13614 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:10.818375 13692 rpcz_store.cc:276] 0709 12:57:09.214702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.216957 (+1002255us) service_pool.cc:224] Handling call
0709 12:57:10.217423 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.246177 (+ 28754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:10.246218 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:10.246724 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 12:57:10.371975 (+125251us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:10.372104 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.372304 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:10.792129 (+419825us) raft_consensus.cc:1685] finished
0709 12:57:10.792325 (+   196us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:10.792483 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107358,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"spinlock_wait_cycles":14080,"thread_start_us":124170,"threads_started":1,"wal-append.queue_time_us":298067}
I20260709 12:57:10.824877 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:10.824023 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 310619us
I20260709 12:57:10.799211 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1148) took 2077 ms. Trace:
I20260709 12:57:10.800244 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:10.827688 13684 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:10.791788 15791 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:10.804451 15767 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:10.806532 15761 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:10.807116 15729 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:10.807210 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.858071 13693 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:10.817238 15979 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 12:57:10.711736 15926 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.808280 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:10.809011 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:10.810068 15777 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:10.810381 15865 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:10.810889 15984 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:57:10.814059 15969 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:10.815140 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:10.709074 15789 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.711597 15764 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.817868 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:10.818037 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:10.775254 15963 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:57:10.825176 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:10.825459 13692 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:10.825470 13614 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:10.833576 15957 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:10.834476 15866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 048528e413ac4c67a1cfd3dfe70673c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:57:10.835623 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2432) took 1461 ms. Trace:
I20260709 12:57:10.829807 15847 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.838238 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2582) took 2651 ms. Trace:
I20260709 12:57:10.841315 13687 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:10.856042 13690 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:10.680361 15600 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.817821 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:10.817710 15814 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:57:10.859697 13542 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:57:11.037175 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 212079us
I20260709 12:57:10.842286 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2585) took 2120 ms. Trace:
I20260709 12:57:11.105808 13691 rpcz_store.cc:276] 0709 12:57:08.721433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.089776 (+1368343us) service_pool.cc:224] Handling call
0709 12:57:10.090172 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.090447 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:57:10.352334 (+261887us) log.cc:844] Serialized 14 byte log entry
0709 12:57:10.826132 (+473798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:10.826207 (+    75us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:10.826277 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.826406 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:10.842218 (+ 15812us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3981,"prepare.queue_time_us":298253,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":5343,"raft.queue_time_us":4300,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"thread_start_us":720172,"threads_started":2}
I20260709 12:57:11.106822 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:10.881404 13543 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:10.858464 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:10.851492 15928 consensus_meta.cc:287] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.622s	user 0.001s	sys 0.014s
I20260709 12:57:11.108057 13540 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:11.036692 13617 rpcz_store.cc:276] 0709 12:57:09.374216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.553466 (+179250us) service_pool.cc:224] Handling call
0709 12:57:10.452348 (+898882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.453132 (+   784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.835399 (+382267us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b540020f650. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:10.835550 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1966567680}
W20260709 12:57:11.042953 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 288898us
W20260709 12:57:10.938450 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 589597us
I20260709 12:57:11.110143 13584 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:10.956992 13689 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
W20260709 12:57:11.084462 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.008s	sys 0.023s
I20260709 12:57:11.101305 13686 rpcz_store.cc:276] 0709 12:57:08.186691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.083210 (+1896519us) service_pool.cc:224] Handling call
0709 12:57:10.118923 (+ 35713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.481306 (+362383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.837666 (+356360us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54002986d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:10.838151 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654026496}
I20260709 12:57:11.111931 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:11.112650 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.009s	sys 0.023s
I20260709 12:57:10.923062 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1233) took 2326 ms. Trace:
I20260709 12:57:11.114281 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:11.102546 13616 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:11.112679 13542 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:11.115543 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:11.115742 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:11.116837 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:11.117235 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:11.117401 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.118119 13614 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:11.118674 13543 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:11.119596 13614 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:11.119994 13614 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:11.120893 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:11.116189 13542 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:11.123751 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:11.124074 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 640501us
I20260709 12:57:11.124178 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:11.125123 13540 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:11.125031 13691 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:57:11.114461 13539 rpcz_store.cc:276] 0709 12:57:08.597017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:08.958604 (+361587us) service_pool.cc:224] Handling call
0709 12:57:09.945016 (+986412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:09.945248 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:09.945286 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:09.945343 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:09.945397 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:09.945472 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:10.922803 (+977331us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b540028b750. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:10.922976 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2267589504}
I20260709 12:57:11.116210 13685 raft_consensus.cc:1217] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Deduplicated request from leader. Original: 1.1->[2.2-4.3]   Dedup: 2.2->[4.3-4.3]
I20260709 12:57:11.131240 13684 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:11.136269 13685 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:57:11.139411 13693 raft_consensus.cc:2410] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:11.154421 13692 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
W20260709 12:57:11.155107 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:10.830507 15887 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:10.860275 15995 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:10.922698 15877 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.120733 13614 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.166112 15949 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:11.117264 13612 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:11.168433 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.169234 15980 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:11.169718 15894 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:11.171506 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.072278 15945 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:11.172204 15808 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:11.117874 15928 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:11.172312 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 48642.3us
I20260709 12:57:10.829013 15991 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:10.831543 15853 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:11.173877 15776 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.001s
I20260709 12:57:10.844846 15958 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.347s
I20260709 12:57:10.882668 15937 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:11.511889 13618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.015s	sys 0.029s
W20260709 12:57:11.544539 13618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.019s	sys 0.029s
W20260709 12:57:11.503063 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 393000us
W20260709 12:57:11.182228 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 72592.7us
I20260709 12:57:11.549096 13614 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:11.547931 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:11.582952 13688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.047s	sys 0.059s
W20260709 12:57:11.583664 13688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.047s	sys 0.060s
I20260709 12:57:11.585006 13616 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
W20260709 12:57:11.582999 13541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.013s	sys 0.038s
W20260709 12:57:11.589000 13541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.013s	sys 0.038s
I20260709 12:57:11.590138 13506 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:11.593441 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 49210.3us
W20260709 12:57:11.595001 13584 connection.cc:256] RPC call timeout handler was delayed by -1.2289611779997358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
W20260709 12:57:11.595458 13584 connection.cc:256] RPC call timeout handler was delayed by -1.2136611779997111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
I20260709 12:57:11.595662 13506 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:11.597447 13584 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.577s (SENT)
I20260709 12:57:11.598771 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:11.162108 15969 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:11.560719 15663 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.007s	sys 0.016s
W20260709 12:57:11.605289 16002 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:57:11.411386 15777 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:11.621243 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 774) took 1199 ms. Trace:
I20260709 12:57:11.282876 15729 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:11.285765 15790 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.320s
I20260709 12:57:11.608300 15872 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:11.113243 15865 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.608305 15928 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.610822 15853 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.611225 15963 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:11.588080 15995 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:11.586134 15461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.020s	sys 0.027s
I20260709 12:57:11.378238 15968 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:11.266362 15761 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:11.177461 15789 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.162681 15983 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:11.615438 13584 connection.cc:256] RPC call timeout handler was delayed by -1.0262879980000434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
I20260709 12:57:11.567055 15980 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:11.621626 14243 rpcz_store.cc:276] 0709 12:57:10.422224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.610910 (+188686us) service_pool.cc:224] Handling call
0709 12:57:10.611385 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.611876 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:10.617026 (+  5150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:10.649403 (+ 32377us) log.cc:844] Serialized 26 byte log entry
0709 12:57:11.036867 (+387464us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:11.036912 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.036984 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:11.620829 (+583845us) raft_consensus.cc:1685] finished
0709 12:57:11.620863 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:11.621175 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35748,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":9099,"thread_start_us":355371,"threads_started":1,"wal-append.queue_time_us":489414}
I20260709 12:57:10.826651 13534 rpcz_store.cc:276] 0709 12:57:08.721519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.557718 (+836199us) service_pool.cc:224] Handling call
0709 12:57:09.636856 (+ 79138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:09.738895 (+102039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:09.738963 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:09.739572 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 12:57:10.020434 (+280862us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:10.105800 (+ 85366us) log.cc:844] Serialized 14 byte log entry
0709 12:57:10.158030 (+ 52230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.183185 (+ 25155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:10.798860 (+615675us) raft_consensus.cc:1685] finished
0709 12:57:10.798892 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:10.799119 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81718,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":15130,"raft.queue_time_us":208065,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":10061,"thread_start_us":254888,"threads_started":1,"wal-append.queue_time_us":932422}
I20260709 12:57:11.179368 15988 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:11.592476 15814 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:11.178413 15994 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:11.630954 13614 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:11.612259 15946 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 12:57:11.618674 15663 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.008s	sys 0.016s
I20260709 12:57:11.601046 15979 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:11.181517 15986 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.505s
I20260709 12:57:11.615309 15969 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:11.585091 15791 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:57:11.613898 15847 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.617915 15776 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:11.594630 15976 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:57:11.653131 14211 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:11.718593 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65582.9us
W20260709 12:57:11.684552 15846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.032s	sys 0.043s
I20260709 12:57:11.600144 15627 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:57:11.617883 15470 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:57:11.588385 15808 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.636516 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.733312 13617 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 3.
I20260709 12:57:11.666929 15980 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:11.681631 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.706998 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:11.685351 13619 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:57:11.739744 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.017s	sys 0.025s
W20260709 12:57:11.654721 15461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.021s	sys 0.028s
W20260709 12:57:11.686226 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.009s	sys 0.036s
W20260709 12:57:11.740165 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.017s	sys 0.025s
I20260709 12:57:11.653348 14239 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:11.683138 15861 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:11.634977 15777 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.680943 15994 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:11.716408 13610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.000s	sys 0.034s
I20260709 12:57:11.634455 15729 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:11.586001 15945 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.722249 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:11.749425 13610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.000s	sys 0.035s
I20260709 12:57:11.749960 13615 raft_consensus.cc:2368] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5.
I20260709 12:57:11.722711 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 422) took 2066 ms. Trace:
I20260709 12:57:11.750308 13536 rpcz_store.cc:276] 0709 12:57:09.656117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.435159 (+779042us) service_pool.cc:224] Handling call
0709 12:57:10.435473 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.679396 (+243923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.722356 (+1042960us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54003cb254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:11.722621 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559056256}
I20260709 12:57:11.705019 15853 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:11.686445 15926 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:11.713668 14242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.019s	sys 0.030s
I20260709 12:57:11.681473 15968 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.714761 15969 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:11.706257 15764 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.706936 16003 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:11.704231 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 423) took 1441 ms. Trace:
W20260709 12:57:11.764554 14242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.019s	sys 0.031s
I20260709 12:57:11.737105 13614 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:11.754104 15994 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:11.718439 15814 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:11.728484 15846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.032s	sys 0.043s
I20260709 12:57:11.652719 15963 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.752005 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:11.755884 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 370666us
I20260709 12:57:11.761925 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.651463 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.734753 15894 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:11.763550 13659 leader_election.cc:400] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5.
I20260709 12:57:11.761579 15461 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:57:11.740418 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.009s	sys 0.052s
I20260709 12:57:11.793040 13535 rpcz_store.cc:276] 0709 12:57:10.262998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.663110 (+400112us) service_pool.cc:224] Handling call
0709 12:57:10.663411 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.668965 (+  5554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:10.669012 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:10.669682 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:57:11.083418 (+413736us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:11.083477 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.083579 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:11.704022 (+620443us) raft_consensus.cc:1685] finished
0709 12:57:11.704039 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:11.704174 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4032,"raft.queue_time_us":8921,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7931,"thread_start_us":366924,"threads_started":1,"wal-append.queue_time_us":500143}
I20260709 12:57:11.843047 13659 leader_election.cc:403] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:57:11.843353 13615 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:11.823391 13619 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3.
W20260709 12:57:11.801219 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 187845us
I20260709 12:57:11.762897 15853 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:11.852408 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 89034.2us
I20260709 12:57:11.854107 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.666460 15761 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.809942 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.871466 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.872730 13614 raft_consensus.cc:2410] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:11.873734 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.875027 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.816112 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1220) took 1083 ms. Trace:
W20260709 12:57:11.793843 16004 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:57:11.819348 15988 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:11.659395 15983 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:11.752776 15861 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:11.792282 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 196709us
W20260709 12:57:11.846962 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 740131us
I20260709 12:57:11.914125 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:11.799495 13584 leader_election.cc:336] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 2.025s (SENT)
I20260709 12:57:11.850656 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.916415 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:11.917367 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 117898us
W20260709 12:57:11.852771 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 103852us
W20260709 12:57:11.919852 13584 leader_election.cc:336] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 2.216s (SENT)
I20260709 12:57:11.804381 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2443) took 1052 ms. Trace:
I20260709 12:57:11.794507 15893 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:57:11.804862 16003 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:11.810582 15814 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:11.936323 15814 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:11.937444 15600 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.937582 16003 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.876091 13693 rpcz_store.cc:276] 0709 12:57:10.732238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.114018 (+381780us) service_pool.cc:224] Handling call
0709 12:57:11.815643 (+701625us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54001eebd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:11.815969 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54500224}
I20260709 12:57:11.803831 15776 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:11.940778 15945 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.940683 15776 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.941412 16011 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:11.943039 15776 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.944294 15729 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:11.944964 15976 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:11.945076 15791 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:11.946246 15888 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:11.946907 15865 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.949600 15983 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.952160 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:11.952796 13536 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
W20260709 12:57:11.956602 16012 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:57:11.879123 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.880322 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1240) took 3042 ms. Trace:
I20260709 12:57:11.958292 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2437) took 1213 ms. Trace:
I20260709 12:57:11.957280 13541 rpcz_store.cc:276] 0709 12:57:08.810329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.554375 (+744046us) service_pool.cc:224] Handling call
0709 12:57:10.686078 (+1131703us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:10.686280 (+   202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:10.686503 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:10.709325 (+ 22822us) log.cc:844] Serialized 14 byte log entry
0709 12:57:11.851431 (+1142106us) log.cc:844] Serialized 14 byte log entry
0709 12:57:11.852357 (+   926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:11.852405 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:11.852453 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.852519 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:11.852641 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17164,"prepare.queue_time_us":275807,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":91,"raft.queue_time_us":136973,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":37674,"spinlock_wait_cycles":5108992,"thread_start_us":1127172,"threads_started":2,"wal-append.queue_time_us":997731}
W20260709 12:57:11.896653 16000 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:57:11.903797 15833 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:11.959621 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:11.958658 13610 rpcz_store.cc:276] 0709 12:57:10.744708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.744844 (+   136us) service_pool.cc:224] Handling call
0709 12:57:10.745180 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.758603 (+ 13423us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:11.072950 (+314347us) log.cc:844] Serialized 14 byte log entry
0709 12:57:11.749924 (+676974us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:11.759138 (+  9214us) log.cc:844] Serialized 26 byte log entry
0709 12:57:11.759346 (+   208us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:11.759395 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.759489 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:11.958092 (+198603us) raft_consensus.cc:1685] finished
0709 12:57:11.958110 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:11.958235 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":419232,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":64159,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":8197,"spinlock_wait_cycles":4024704,"thread_start_us":818891,"threads_started":2,"wal-append.queue_time_us":673505}
I20260709 12:57:11.960244 13540 raft_consensus.cc:606] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:57:11.960954 13614 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:11.907341 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:11.907748 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:11.961503 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 294) took 1268 ms. Trace:
I20260709 12:57:11.962128 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:11.962498 15593 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:11.962633 15833 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:11.963357 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:11.963570 15833 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:11.964005 15833 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:11.964624 16001 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
W20260709 12:57:11.842070 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:11.840253 15928 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:11.842905 15894 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:11.920645 13618 rpcz_store.cc:276] 0709 12:57:10.752078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.752229 (+   151us) service_pool.cc:224] Handling call
0709 12:57:10.752556 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.752915 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:10.752944 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:10.825937 (+ 72993us) log.cc:844] Serialized 26 byte log entry
0709 12:57:11.585410 (+759473us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:11.585474 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.585567 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:11.804135 (+218568us) raft_consensus.cc:1685] finished
0709 12:57:11.804158 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:11.804323 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57320,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":681369,"threads_started":1,"wal-append.queue_time_us":807721}
I20260709 12:57:11.882593 15764 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5.
I20260709 12:57:11.990039 13618 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:57:11.990425 13618 pending_rounds.cc:85] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 2
I20260709 12:57:11.927491 14212 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:11.933327 16009 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:57:11.933456 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:11.933997 15903 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:57:11.880600 15937 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:11.935156 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:11.906919 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:11.962905 15593 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:12.006235 16008 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:57:12.006647 15593 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.818959 15847 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:12.008112 15847 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.010954 15847 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.013736 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.016199 15995 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.964912 15833 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:12.017554 15777 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.018262 15968 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.019227 15808 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.990378 15928 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.022094 15928 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:11.965777 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:12.026937 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 99560.8us
I20260709 12:57:11.851038 15846 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:11.908437 16005 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:57:11.961858 13688 rpcz_store.cc:276] 0709 12:57:10.692851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.693095 (+   244us) service_pool.cc:224] Handling call
0709 12:57:10.693491 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.693943 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:10.693980 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:10.694569 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 12:57:11.584224 (+889655us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:11.584287 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.584388 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:11.961308 (+376920us) raft_consensus.cc:1685] finished
0709 12:57:11.961325 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:11.961447 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1377,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":889091,"threads_started":1,"wal-append.queue_time_us":953722}
I20260709 12:57:12.036725 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.038009 13688 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:11.991737 15894 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:12.026417 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.056674 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:11.991482 15884 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:57:11.978422 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:11.979509 13535 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:11.811456 15872 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 12:57:11.849349 15989 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:11.982846 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.071327 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:11.982908 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:11.983673 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:12.072926 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.073630 13691 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:57:12.075623 13689 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:57:12.076633 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1215) took 2967 ms. Trace:
I20260709 12:57:11.986196 13693 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:57:12.076987 13687 rpcz_store.cc:276] 0709 12:57:09.109324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.172501 (+1063177us) service_pool.cc:224] Handling call
0709 12:57:10.840694 (+668193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:11.100254 (+259560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.076452 (+976198us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54002986d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:12.076578 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535742720}
I20260709 12:57:11.847312 15887 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:11.988420 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 68601.6us
W20260709 12:57:12.080155 13584 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 048528e413ac4c67a1cfd3dfe70673c7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:57:11.991271 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:11.991829 13618 pending_rounds.cc:107] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:57:12.084271 13618 raft_consensus.cc:2889] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:57:12.084340 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 93165.3us
I20260709 12:57:12.084467 15977 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:12.084734 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:12.085080 15977 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:12.085759 15887 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:12.085698 15977 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:12.086616 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:11.992935 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.089437 15663 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:12.089845 15663 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:12.090153 15963 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:11.925840 15988 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.092257 15761 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.090210 15663 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:12.099251 13613 raft_consensus.cc:2410] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:57:12.100698 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:12.101460 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:12.101887 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:11.916774 13615 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:11.934579 15861 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.000118 13537 raft_consensus.cc:2410] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:57:11.880059 15991 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:12.079110 15980 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:11.987076 15729 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.002218 15983 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:57:12.015833 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:12.043218 15846 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:11.965878 15957 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:11.923619 16010 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:12.063158 13688 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:12.047112 15893 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:12.065495 13540 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:12.106125 13615 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:12.085973 14212 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:12.191546 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 105654us
I20260709 12:57:12.191691 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:11.971494 13614 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:57:12.157138 15853 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:12.120000 13685 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:11.965543 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:12.108655 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.161194 15814 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:12.160483 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:12.112066 15995 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.164162 15888 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:12.167097 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:12.204950 13537 raft_consensus.cc:2376] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5. Re-sending same reply.
I20260709 12:57:12.206103 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.161257 16010 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:12.169554 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:12.159144 14244 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:12.175905 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 206837us
I20260709 12:57:12.177309 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 435) took 1080 ms. Trace:
I20260709 12:57:12.177846 15846 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:12.194260 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 371) took 1029 ms. Trace:
I20260709 12:57:12.194948 15893 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:12.192380 13615 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:12.219128 13543 rpcz_store.cc:276] 0709 12:57:11.097541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.119434 (+ 21893us) service_pool.cc:224] Handling call
0709 12:57:11.119713 (+   279us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:11.284669 (+164956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:11.284700 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:11.416625 (+131925us) log.cc:844] Serialized 52 byte log entry
0709 12:57:11.801544 (+384919us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:12.160183 (+358639us) log.cc:844] Serialized 14 byte log entry
0709 12:57:12.176862 (+ 16679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.176959 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:12.177015 (+    56us) raft_consensus.cc:1685] finished
0709 12:57:12.177033 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:12.177224 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72028,"prepare.queue_time_us":55079,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":196379,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":83093,"thread_start_us":607832,"threads_started":2,"wal-append.queue_time_us":516157}
I20260709 12:57:12.225430 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:12.198469 14243 raft_consensus.cc:2376] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6. Re-sending same reply.
W20260709 12:57:12.196116 13658 connection.cc:256] RPC call timeout handler was delayed by -1.0423820560008608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
I20260709 12:57:12.198558 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.227146 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:12.227182 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:12.228067 14243 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:12.228358 13658 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:12.229206 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:12.230067 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:12.174167 14240 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:12.220181 13614 rpcz_store.cc:276] 0709 12:57:11.164891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.970680 (+805789us) service_pool.cc:224] Handling call
0709 12:57:11.971159 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:12.193785 (+222626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.194177 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:12.022675 13612 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:12.220944 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:12.226745 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 57365.3us
I20260709 12:57:12.234014 13537 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:12.234176 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:12.234884 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.202509 15963 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:12.238338 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:12.237376 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.230563 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:12.233174 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:12.239445 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.239689 13614 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:12.227617 13540 raft_consensus.cc:2410] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:12.240427 13543 raft_consensus.cc:2410] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:12.115038 15989 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.243031 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:12.243453 13584 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:12.122335 15729 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:12.238144 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:12.244922 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:12.250460 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:12.250986 13614 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:12.249738 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:12.249686 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.269130 13538 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:12.269510 13538 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:12.248957 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:12.272361 13614 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:12.273515 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.201136 15872 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:57:12.248256 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.276683 13684 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:12.236312 15893 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:12.279861 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:12.210595 15988 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.281749 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:12.233385 15761 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:12.279235 15861 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:12.306207 15729 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:12.307863 15977 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:12.308733 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 368) took 1186 ms. Trace:
I20260709 12:57:12.309170 13610 rpcz_store.cc:276] 0709 12:57:11.122275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.960496 (+838221us) service_pool.cc:224] Handling call
0709 12:57:11.960801 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:11.961175 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:11.961202 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:11.969894 (+  8692us) log.cc:844] Serialized 52 byte log entry
0709 12:57:12.173707 (+203813us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:12.173749 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.173815 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:12.308481 (+134666us) raft_consensus.cc:1685] finished
0709 12:57:12.308511 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:12.308666 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8936,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":21280,"thread_start_us":161404,"threads_started":1,"wal-append.queue_time_us":222946}
I20260709 12:57:12.312412 15884 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.270483 13538 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:12.314352 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.314873 13538 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:12.283015 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:12.315376 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 81255.1us
I20260709 12:57:12.201759 15951 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:57:12.316257 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.311748 13610 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:57:12.237958 15991 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:12.238566 15764 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:12.322074 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 77224.2us
I20260709 12:57:12.209960 15995 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.327816 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.346544 13684 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
W20260709 12:57:12.327509 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45830.1us
I20260709 12:57:12.349575 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:12.347754 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.352367 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.355285 13684 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:12.357352 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.231791 15928 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:57:12.268275 15655 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:12.337569 13582 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:12.249339 15846 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.359138 13684 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:12.384791 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:12.238658 15963 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:12.368059 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:12.237504 15937 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:57:12.241631 15531 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:57:12.387149 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:12.316704 15926 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:57:12.395293 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 57812.2us
I20260709 12:57:12.305419 15980 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:12.362299 15999 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:57:12.321257 15789 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:12.248744 15945 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 12:57:12.341640 15988 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:12.445490 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:12.340018 15847 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:12.331708 15991 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:12.268257 16016 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.188s
I20260709 12:57:12.458881 15988 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:12.445765 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 58732.8us
I20260709 12:57:12.459967 15988 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:12.460995 15991 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:12.459239 15847 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:12.463352 15847 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:12.416219 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.466313 13540 raft_consensus.cc:2410] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:12.306429 15861 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:12.472220 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:12.343649 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:12.313292 15884 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.358700 15764 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:12.486812 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.472618 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:12.504555 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 59172.5us
I20260709 12:57:12.511166 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.513135 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.514302 13684 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:12.467653 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.515487 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.362739 15627 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:12.471704 16003 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:12.539963 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.544852 13684 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:12.248054 16021 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.211s
W20260709 12:57:12.560472 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 168524us
I20260709 12:57:12.403187 15963 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:12.400739 15976 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:12.559109 14246 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:12.562021 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:12.507521 15776 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.569439 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:12.570240 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.518076 15764 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:12.487342 15986 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:12.554960 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:12.571233 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 226002us
I20260709 12:57:12.536966 15983 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:12.572578 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.573561 13689 raft_consensus.cc:2376] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5. Re-sending same reply.
I20260709 12:57:12.548806 15853 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:12.472879 15791 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 12:57:12.487754 15453 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 12:57:12.554800 15903 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:12.538888 15627 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:12.362578 16019 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.240s
I20260709 12:57:12.574033 15926 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:12.571888 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:12.575825 15928 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.587194 15627 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:12.582736 15957 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:12.588102 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:12.401496 15846 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.581168 14240 raft_consensus.cc:2410] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:57:12.621558 16030 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:57:12.574239 15776 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.485872 16017 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 12:57:12.553925 16003 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:12.607479 16020 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.396s	user 0.001s	sys 0.000s
W20260709 12:57:12.607805 13569 heartbeater.cc:646] Failed to heartbeat to 127.12.30.126:40047 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.30.126:40047 timed out after 15.000s (SENT)
I20260709 12:57:12.583592 15986 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:12.365157 15888 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.191s
I20260709 12:57:12.583621 13538 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:57:12.724421 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.789398 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:12.792642 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1157) took 2748 ms. Trace:
W20260709 12:57:12.627365 16034 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:57:12.588580 15853 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:12.703122 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.751106 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:12.750392 15791 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:12.755119 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:12.758390 13612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1372) took 1208 ms (client timeout 1522 ms). Trace:
I20260709 12:57:12.760032 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2441) took 2010 ms. Trace:
I20260709 12:57:12.845593 16044 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.845767 13613 rpcz_store.cc:276] 0709 12:57:10.749432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.750818 (+  1386us) service_pool.cc:224] Handling call
0709 12:57:12.759755 (+2008937us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b540026eb54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:12.759961 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2291867520}
I20260709 12:57:12.762342 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:12.847456 13684 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:12.760641 15995 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.614020 15928 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:12.854187 16042 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:57:12.706094 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:12.750484 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:12.769048 15994 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.928s	user 0.005s	sys 0.006s
W20260709 12:57:12.630287 16031 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:57:12.793507 13538 rpcz_store.cc:276] 0709 12:57:10.043907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.484246 (+440339us) service_pool.cc:224] Handling call
0709 12:57:10.484574 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:12.269035 (+1784461us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b540002b0a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 12:57:12.792199 (+523164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.792565 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111045888}
I20260709 12:57:12.839447 16021 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:12.835145 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:12.822968 15816 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:57:12.842358 15791 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:12.598493 15926 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:12.837568 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:12.852159 13441 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49880 (request call id 327) took 16453 ms (client timeout 15000 ms). Trace:
I20260709 12:57:12.852515 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:12.856611 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 150614us
W20260709 12:57:12.888386 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.890172 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:12.856969 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 105976us
I20260709 12:57:12.857446 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:12.894093 13613 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:12.895807 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:12.902464 15995 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:12.875247 15893 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.001s
I20260709 12:57:12.933415 15846 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.933144 15893 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.936210 15986 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.940420 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:12.941486 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:12.942107 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 440) took 1215 ms. Trace:
I20260709 12:57:12.942615 13542 rpcz_store.cc:276] 0709 12:57:11.727011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.727124 (+   113us) service_pool.cc:224] Handling call
0709 12:57:11.727458 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:11.733956 (+  6498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:11.734002 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:11.741435 (+  7433us) log.cc:844] Serialized 26 byte log entry
0709 12:57:11.857680 (+116245us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:11.857753 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.857856 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:11.954298 (+ 96442us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b4800002dd8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:57:12.887210 (+932912us) raft_consensus.cc:1685] finished
0709 12:57:12.887231 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:12.941986 (+ 54755us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10682,"raft.run_cpu_time_us":3691,"raft.run_wall_time_us":212881,"spinlock_wait_cycles":15648000,"thread_start_us":67804,"threads_started":1,"wal-append.queue_time_us":98131}
I20260709 12:57:12.857158 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:12.860424 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:12.833356 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:12.865065 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.048409 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2598) took 1065 ms. Trace:
I20260709 12:57:12.857854 15928 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:12.859028 16017 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.049731 13686 rpcz_store.cc:276] 0709 12:57:11.983007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.983372 (+   365us) service_pool.cc:224] Handling call
0709 12:57:12.003561 (+ 20189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:12.004060 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:12.004105 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:12.004741 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 12:57:12.235880 (+231139us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:12.235939 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.236005 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.048071 (+812066us) raft_consensus.cc:1685] finished
0709 12:57:13.048101 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.048318 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13236,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":210603,"threads_started":1,"wal-append.queue_time_us":358676}
I20260709 12:57:12.858685 15531 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:12.846184 16044 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:12.822966 15903 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:12.901583 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:13.085340 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 183803us
I20260709 12:57:13.086484 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:12.891048 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:13.090801 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 199792us
I20260709 12:57:12.910408 15764 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:12.845584 13612 rpcz_store.cc:269] 0709 12:57:11.550438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.619534 (+ 69096us) service_pool.cc:224] Handling call
0709 12:57:12.758136 (+1138602us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540013a6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:12.758308 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413506432}
I20260709 12:57:12.913497 15776 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:12.866801 16049 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:12.923228 15663 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.827s	user 0.005s	sys 0.005s
W20260709 12:57:12.930100 15894 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.874s	user 0.003s	sys 0.007s
I20260709 12:57:12.883643 15791 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:12.931552 15984 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:13.135358 15984 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:13.136047 15984 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:12.932195 15861 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.010596 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:12.987267 16010 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.008121 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.143121 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:12.991411 13659 connection.cc:256] RPC call timeout handler was delayed by -1.1068732319999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
I20260709 12:57:13.001103 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:13.144326 13659 connection.cc:256] RPC call timeout handler was delayed by -1.0551667100007762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
I20260709 12:57:13.144665 13617 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:13.145678 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:13.146452 13617 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:13.006413 15777 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:12.988109 15846 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.931001 15988 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.159569 15903 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.162062 15988 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:12.946208 15893 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:12.919130 15991 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:12.981739 15888 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.871136 15655 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:12.996084 15761 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:12.853106 15884 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:13.052974 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 195909us
I20260709 12:57:12.887384 15926 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:12.865437 15994 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.091866 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:13.174549 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:13.124536 13685 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.093s	sys 0.140s
I20260709 12:57:13.042435 15470 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:13.177043 15470 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:13.133500 16038 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.174s	user 0.000s	sys 0.005s
I20260709 12:57:13.134012 15894 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.072904 15789 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:13.138175 15663 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.136920 15976 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:12.904547 16003 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:13.112344 15958 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:13.159459 16047 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.411s	user 0.002s	sys 0.000s
I20260709 12:57:13.120828 16017 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:13.174764 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 83014.5us
I20260709 12:57:13.055052 15937 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.150368 15777 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:13.178897 13685 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.093s	sys 0.140s
I20260709 12:57:13.162429 15959 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:57:13.149833 16046 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 12:57:13.184844 15789 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:13.178608 15893 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:13.174227 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:13.181694 15969 consensus_meta.cc:287] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.378s	user 0.012s	sys 0.007s
W20260709 12:57:13.182376 15866 consensus_meta.cc:287] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.000s
I20260709 12:57:13.167953 15888 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.211217 15958 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:13.174479 15761 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.238276 13685 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:57:13.225754 15789 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:13.231002 14238 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:13.172693 15846 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.231205 15972 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:13.226281 15937 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:13.288323 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 487) took 1184 ms. Trace:
I20260709 12:57:13.231003 15959 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:13.298049 16051 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:57:13.237119 15969 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.238849 15958 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:13.301489 15764 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.231376 15777 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.217108 15976 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:13.302189 15960 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 2.508s	user 0.016s	sys 0.003s
I20260709 12:57:13.310065 15959 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:13.331153 16044 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.337829 15991 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.310282 15980 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:13.299890 15994 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.345162 15980 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:13.332062 16054 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.348393 15969 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.348150 16017 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.349550 15980 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.352571 15976 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:12.982663 13437 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 2 to 3, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 3 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:13.391255 13437 catalog_manager.cc:5697] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } }
W20260709 12:57:12.906262 13441 rpcz_store.cc:269] 0709 12:56:56.380365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:56:56.381821 (+  1456us) service_pool.cc:224] Handling call
0709 12:57:12.796701 (+16414880us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:57:04.304148 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:57:04.457873 (+153725us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:57:04.515058 (+ 57185us) write_op.cc:435] Acquired schema lock
0709 12:57:04.515087 (+    29us) tablet.cc:662] Decoding operations
0709 12:57:04.760367 (+245280us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:57:04.773113 (+ 12746us) write_op.cc:641] Partition lock acquired for write op
0709 12:57:04.773162 (+    49us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:57:04.774212 (+  1050us) tablet.cc:701] Row locks acquired
0709 12:57:04.774277 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:57:04.774491 (+   214us) write_op.cc:270] Start()
0709 12:57:04.774654 (+   163us) write_op.cc:276] Timestamp: P: 1783601824774412 usec, L: 0
0709 12:57:04.774692 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:57:04.775524 (+   832us) log.cc:844] Serialized 2449 byte log entry
0709 12:57:05.331614 (+556090us) op_driver.cc:464] REPLICATION: finished
0709 12:57:05.667727 (+336113us) write_op.cc:301] APPLY: starting
0709 12:57:05.680641 (+ 12914us) tablet.cc:1367] starting BulkCheckPresence
0709 12:57:05.743978 (+ 63337us) tablet.cc:1370] finished BulkCheckPresence
0709 12:57:05.744143 (+   165us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:57:05.806354 (+ 62211us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:57:05.885308 (+ 78954us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:57:05.885381 (+    73us) write_op.cc:312] APPLY: finished
0709 12:57:05.885726 (+   345us) log.cc:844] Serialized 53 byte log entry
0709 12:57:05.885926 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:05.886034 (+   108us) write_op.cc:454] Released schema lock
0709 12:57:05.886099 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:57:05.886693 (+   594us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:05.886734 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6972,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":279521,"apply.run_cpu_time_us":4632,"apply.run_wall_time_us":246230,"num_ops":6,"prepare.queue_time_us":607397,"prepare.run_cpu_time_us":24655,"prepare.run_wall_time_us":784849,"raft.queue_time_us":135984,"raft.run_cpu_time_us":41469,"raft.run_wall_time_us":70247,"replication_time_us":556718,"thread_start_us":507484,"threads_started":4,"wal-append.queue_time_us":236706,"wal-append.run_cpu_time_us":53814,"wal-append.run_wall_time_us":1966900}]]}
I20260709 12:57:13.395713 13437 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:13.398777 13437 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 2 to 4, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 4 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:13.401916 13437 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:13.404372 13437 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 3, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 3 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:13.407156 13437 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:13.048103 13535 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:13.044164 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.140378 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.456796 15846 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.457710 15777 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:13.148279 13543 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 3: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:13.171074 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:13.461757 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1170) took 1370 ms. Trace:
I20260709 12:57:13.462289 13543 rpcz_store.cc:276] 0709 12:57:12.091447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.142553 (+1051106us) service_pool.cc:224] Handling call
0709 12:57:13.461458 (+318905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.043697 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:13.465737 15894 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.466555 13542 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:57:13.466709 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 422969us
I20260709 12:57:13.467535 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1164) took 1377 ms. Trace:
I20260709 12:57:13.139951 13616 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:13.508322 15453 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:13.509203 15453 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:13.509845 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:13.420470 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 333915us
I20260709 12:57:13.292244 14240 rpcz_store.cc:276] 0709 12:57:12.101079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.130101 (+1029022us) service_pool.cc:224] Handling call
0709 12:57:13.285553 (+155452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.441316 13692 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:13.446077 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1261) took 1379 ms. Trace:
W20260709 12:57:13.450934 15887 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.356s	user 0.013s	sys 0.005s
W20260709 12:57:13.344990 15963 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.763s	user 0.013s	sys 0.000s
W20260709 12:57:13.311017 15627 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.704s	user 0.014s	sys 0.000s
W20260709 12:57:13.105425 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:13.522238 15627 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:13.523488 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:13.524446 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:13.526093 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:13.422921 13615 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:13.489961 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 344287us
I20260709 12:57:13.494719 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 485) took 1395 ms. Trace:
I20260709 12:57:13.484835 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2612) took 1239 ms. Trace:
I20260709 12:57:13.459793 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 436) took 1802 ms. Trace:
I20260709 12:57:13.468408 15866 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59164 (request call id 763) took 5951 ms. Trace:
I20260709 12:57:13.489521 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.489234 15470 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:13.471192 13613 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.000s
I20260709 12:57:13.530961 13613 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:13.530254 15866 rpcz_store.cc:276] 0709 12:57:07.540865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.619804 (+ 78939us) service_pool.cc:224] Handling call
0709 12:57:08.005146 (+385342us) ts_tablet_manager.cc:934] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
0709 12:57:10.833111 (+2827965us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:57:12.577523 (+1744412us) tablet_metadata.cc:665] Metadata flushed
0709 12:57:12.585140 (+  7617us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:57:13.268105 (+682965us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:57:13.328392 (+ 60287us) tablet_replica.cc:206] Creating consensus instance
0709 12:57:13.467854 (+139462us) ts_tablet_manager.cc:989] Replica registered.
0709 12:57:13.468056 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":41148,"tablet-copy.queue_time_us":205875,"thread_start_us":214395,"threads_started":2}
I20260709 12:57:13.493721 15761 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.491358 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1357) took 3880 ms. Trace:
I20260709 12:57:13.532392 15866 tablet_copy_client.cc:806] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:57:13.532918 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:13.533370 13538 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
W20260709 12:57:13.474458 15847 consensus_meta.cc:287] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.010s	user 0.019s	sys 0.000s
I20260709 12:57:13.231644 15655 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.302254 16057 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.293s
I20260709 12:57:13.484380 15976 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.469394 15903 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:57:13.540868 15903 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:13.540477 15976 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.541397 15903 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.541926 15903 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:13.542510 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:13.495754 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 781) took 1590 ms. Trace:
I20260709 12:57:13.543730 15969 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:13.544032 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:13.494153 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 319533us
I20260709 12:57:13.544713 15969 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.545948 16057 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:13.527845 13691 rpcz_store.cc:276] 0709 12:57:12.245737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.245949 (+   212us) service_pool.cc:224] Handling call
0709 12:57:12.246256 (+   307us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:12.287663 (+ 41407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:12.287696 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:12.337644 (+ 49948us) log.cc:844] Serialized 52 byte log entry
0709 12:57:12.578524 (+240880us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:12.599555 (+ 21031us) log.cc:844] Serialized 14 byte log entry
0709 12:57:12.606170 (+  6615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.606264 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.432267 (+826003us) raft_consensus.cc:1685] finished
0709 12:57:13.445645 (+ 13378us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.484597 (+ 38952us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31302,"prepare.queue_time_us":134160,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":39981,"raft.run_cpu_time_us":4585,"raft.run_wall_time_us":185096,"thread_start_us":91033,"threads_started":1,"wal-append.queue_time_us":190179}
I20260709 12:57:13.487515 15663 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.333775 15853 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.472486 15937 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.548908 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2447) took 1503 ms. Trace:
I20260709 12:57:13.544075 14244 rpcz_store.cc:276] 0709 12:57:11.905141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.909315 (+  4174us) service_pool.cc:224] Handling call
0709 12:57:13.495231 (+1585916us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400681fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:13.495532 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319431424}
I20260709 12:57:13.507710 15888 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.441215 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 303) took 1157 ms. Trace:
I20260709 12:57:13.487111 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 293) took 2795 ms. Trace:
I20260709 12:57:13.498487 13542 rpcz_store.cc:276] 0709 12:57:12.089880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.465083 (+1375203us) service_pool.cc:224] Handling call
0709 12:57:13.465819 (+   736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.467145 (+  1326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.467454 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.550819 13685 rpcz_store.cc:276] 0709 12:57:12.283525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.229610 (+946085us) service_pool.cc:224] Handling call
0709 12:57:13.233868 (+  4258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.239234 (+  5366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.440824 (+201590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.550910 13690 rpcz_store.cc:276] 0709 12:57:10.691684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.691879 (+   195us) service_pool.cc:224] Handling call
0709 12:57:10.692192 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:11.100538 (+408346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.486380 (+2385842us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540034e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:13.486841 (+   461us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375717248}
I20260709 12:57:13.552304 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.552502 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:13.552994 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:13.553988 13690 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:13.554517 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:13.527287 14246 rpcz_store.cc:276] 0709 12:57:12.100276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.279496 (+179220us) service_pool.cc:224] Handling call
0709 12:57:13.494250 (+1214754us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54004c1dd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:13.494458 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555024128}
I20260709 12:57:13.528486 13539 rpcz_store.cc:276] 0709 12:57:11.656854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.457691 (+1800837us) service_pool.cc:224] Handling call
0709 12:57:13.458608 (+   917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.459094 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.459156 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:13.459225 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:13.459270 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.459372 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.459519 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.536456 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2608) took 1454 ms. Trace:
W20260709 12:57:13.544768 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:13.515466 13534 rpcz_store.cc:276] 0709 12:57:12.039790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.086889 (+ 47099us) service_pool.cc:224] Handling call
0709 12:57:12.087249 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:12.087710 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:12.087745 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:12.121473 (+ 33728us) log.cc:844] Serialized 26 byte log entry
0709 12:57:12.489122 (+367649us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:12.505246 (+ 16124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.536948 (+ 31702us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.418297 (+881349us) raft_consensus.cc:1685] finished
0709 12:57:13.418406 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.418863 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2694,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":164924,"threads_started":1,"wal-append.queue_time_us":368778}
I20260709 12:57:13.538434 15963 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.547693 15969 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.537106 16054 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.520287 13616 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:13.549582 13610 rpcz_store.cc:276] 0709 12:57:12.045649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.318530 (+272881us) service_pool.cc:224] Handling call
0709 12:57:12.318825 (+   295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:12.338427 (+ 19602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:12.338463 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:12.339075 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 12:57:12.865032 (+525957us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:12.865083 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.865161 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.454055 (+588894us) raft_consensus.cc:1685] finished
0709 12:57:13.501085 (+ 47030us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.548656 (+ 47571us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17980,"raft.queue_time_us":18996,"spinlock_wait_cycles":9504640,"thread_start_us":378322,"threads_started":1,"wal-append.queue_time_us":709044}
I20260709 12:57:13.543545 15976 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.531810 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2460) took 1019 ms. Trace:
I20260709 12:57:13.556734 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:13.141184 16050 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.559s
I20260709 12:57:13.533469 15866 tablet_copy_client.cc:670] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:57:13.565645 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.556741 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:13.579931 13543 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:57:13.571978 13613 rpcz_store.cc:276] 0709 12:57:12.512395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.846901 (+334506us) service_pool.cc:224] Handling call
0709 12:57:13.531687 (+684786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.580241 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:13.532529 13611 rpcz_store.cc:276] 0709 12:57:09.610315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:09.638825 (+ 28510us) service_pool.cc:224] Handling call
0709 12:57:09.714892 (+ 76067us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:09.715125 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:09.715150 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:09.715614 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:57:10.032929 (+317315us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:10.032983 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:10.033286 (+   303us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.292599 (+3259313us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:13.490607 (+198008us) raft_consensus.cc:1685] finished
0709 12:57:13.490664 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.491057 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6987008,"thread_start_us":269713,"threads_started":1,"wal-append.queue_time_us":983213}
I20260709 12:57:13.544629 16060 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.557423 13693 rpcz_store.cc:276] 0709 12:57:11.990026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.077972 (+ 87946us) service_pool.cc:224] Handling call
0709 12:57:12.078261 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:12.136544 (+ 58283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:12.136599 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:12.173246 (+ 36647us) log.cc:844] Serialized 26 byte log entry
0709 12:57:12.579010 (+405764us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:12.579058 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:12.579130 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.422798 (+843668us) raft_consensus.cc:1685] finished
0709 12:57:13.422837 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.444154 (+ 21317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34425,"raft.queue_time_us":40212,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4017,"thread_start_us":338079,"threads_started":1,"wal-append.queue_time_us":378018}
I20260709 12:57:13.573838 15846 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:13.590359 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 48226.8us
I20260709 12:57:13.579063 15453 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.591526 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1161) took 1498 ms. Trace:
I20260709 12:57:13.599260 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:13.600507 13543 rpcz_store.cc:276] 0709 12:57:12.089045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.505536 (+1416491us) service_pool.cc:224] Handling call
0709 12:57:13.579479 (+ 73943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.585703 (+  6224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.587337 (+  1634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.601461 13616 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:13.601739 15453 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:13.602237 13543 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: []
I20260709 12:57:13.603030 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1263) took 1487 ms. Trace:
I20260709 12:57:13.579524 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:13.574045 15627 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.579803 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:13.603467 13543 rpcz_store.cc:276] 0709 12:57:12.115539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.601490 (+1485951us) service_pool.cc:224] Handling call
0709 12:57:13.601998 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.602719 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.602942 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.607717 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:13.618485 15949 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.613795 13539 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:13.620154 15949 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.623222 15949 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.584342 15887 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:13.636480 13612 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:13.568303 15847 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.642184 15847 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.648316 13691 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:13.650565 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 304) took 1343 ms. Trace:
I20260709 12:57:13.652817 13685 rpcz_store.cc:276] 0709 12:57:12.283771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.555670 (+1271899us) service_pool.cc:224] Handling call
0709 12:57:13.565519 (+  9849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.614335 (+ 48816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.614396 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:13.627062 (+ 12666us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:13.627139 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.627257 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.627444 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.664767 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:13.655471 14240 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 488) took 1548 ms (client timeout 1930 ms). Trace:
I20260709 12:57:13.678987 13690 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:13.658093 15850 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.661415 15627 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.679592 15761 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:13.706463 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:13.711813 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:13.659605 15846 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:13.677347 14240 rpcz_store.cc:269] 0709 12:57:12.101357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.556144 (+1454787us) service_pool.cc:224] Handling call
0709 12:57:13.629530 (+ 73386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.717811 13543 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:13.661984 16060 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.724969 15816 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.725862 15816 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:13.727744 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:13.728287 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:13.730304 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 116684us
I20260709 12:57:13.738382 16053 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:13.739293 15970 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 3.248s	user 0.000s	sys 0.017s
W20260709 12:57:13.740200 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:13.744160 15627 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.745606 15846 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.746088 15989 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.746222 15846 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.746758 15627 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.747464 15663 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.748289 15663 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:13.749926 16032 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.345s	user 0.000s	sys 0.002s
I20260709 12:57:13.751989 15846 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.739392 15850 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.755182 16060 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.756880 15963 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.757201 15717 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:13.709206 13535 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a689534693d4469befa622d6b4917e3"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:13.758183 13535 raft_consensus.cc:606] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:13.758612 13535 raft_consensus.cc:614] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:57:13.759074 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32778 (request call id 26) took 1601 ms. Trace:
I20260709 12:57:13.759792 13542 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:13.762957 13685 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:13.763852 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:13.765036 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:13.765476 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:13.769150 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:13.771368 13685 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:13.772612 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:13.773932 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:13.774472 13690 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:13.777020 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:13.778055 13685 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:13.779066 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:13.779871 13685 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:13.781201 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:13.782086 13685 raft_consensus.cc:2410] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:57:13.781950 16044 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:13.782850 16044 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.783365 15963 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:13.785060 16060 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:13.720911 15976 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:13.789805 14239 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 472) took 3631 ms (client timeout 1913 ms). Trace:
I20260709 12:57:13.790120 15976 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.790683 15816 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:13.790414 14239 rpcz_store.cc:269] 0709 12:57:10.158408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.171591 (+ 13183us) service_pool.cc:224] Handling call
0709 12:57:13.789309 (+3617718us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54005d0a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:13.789577 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2918584064}
I20260709 12:57:13.792373 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.792748 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:13.793462 13541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1258) took 1820 ms (client timeout 1933 ms). Trace:
I20260709 12:57:13.793879 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 787) took 1407 ms. Trace:
I20260709 12:57:13.794435 14239 rpcz_store.cc:276] 0709 12:57:12.386528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.792374 (+1405846us) service_pool.cc:224] Handling call
0709 12:57:13.793784 (+  1410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.795881 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:13.796802 13619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2442) took 3045 ms (client timeout 1577 ms). Trace:
W20260709 12:57:13.797326 13619 rpcz_store.cc:269] 0709 12:57:10.751167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.751701 (+   534us) service_pool.cc:224] Handling call
0709 12:57:13.796306 (+3044605us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54002b5554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:13.796512 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1344498432}
I20260709 12:57:13.801987 13618 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:57:13.804940 14241 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:13.805260 16054 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.806609 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:13.807549 13617 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:13.807613 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:13.809455 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:13.811626 16060 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:13.793902 13541 rpcz_store.cc:269] 0709 12:57:11.972922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.973142 (+   220us) service_pool.cc:224] Handling call
0709 12:57:13.792011 (+1818869us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:13.793356 (+  1345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64500096}
I20260709 12:57:13.817230 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 373) took 1714 ms. Trace:
I20260709 12:57:13.818650 15846 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.759418 13535 rpcz_store.cc:276] 0709 12:57:12.157409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.513551 (+1356142us) service_pool.cc:224] Handling call
0709 12:57:13.758973 (+245422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.819072 13619 rpcz_store.cc:276] 0709 12:57:12.102813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.799173 (+1696360us) service_pool.cc:224] Handling call
0709 12:57:13.815025 (+ 15852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.816565 (+  1540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.816595 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:13.816640 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:13.816676 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.816939 (+   263us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.817109 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.827550 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.828100 13691 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:13.831713 13543 raft_consensus.cc:2410] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:57:13.832860 16050 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:13.834836 15593 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:13.845314 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1262) took 1727 ms. Trace:
I20260709 12:57:13.848377 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2601) took 1864 ms. Trace:
I20260709 12:57:13.739384 16053 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.848922 13692 rpcz_store.cc:276] 0709 12:57:11.983900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.071048 (+ 87148us) service_pool.cc:224] Handling call
0709 12:57:13.844953 (+1773905us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400002154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:13.848291 (+  3338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114958592}
I20260709 12:57:13.857133 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:13.858151 13612 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:57:13.870602 15957 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:13.871245 15957 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:57:13.874513 15926 consensus_meta.cc:287] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.013s
I20260709 12:57:13.877579 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:13.841109 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 490) took 1716 ms. Trace:
I20260709 12:57:13.879513 14239 rpcz_store.cc:276] 0709 12:57:12.109350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.795518 (+1686168us) service_pool.cc:224] Handling call
0709 12:57:13.825794 (+ 30276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.881035 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:13.881438 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:13.881888 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 492) took 1772 ms. Trace:
I20260709 12:57:13.882309 14239 rpcz_store.cc:276] 0709 12:57:12.109939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.880597 (+1770658us) service_pool.cc:224] Handling call
0709 12:57:13.881832 (+  1235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.884904 16060 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:13.887353 13617 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:13.890027 16060 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.885113 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:13.890779 16060 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.892149 15534 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:13.892401 15958 consensus_meta.cc:287] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.008s
I20260709 12:57:13.892688 15534 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:13.892990 15958 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.894397 15947 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.895031 15949 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:13.875366 14209 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:13.846925 13538 rpcz_store.cc:276] 0709 12:57:12.111880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.586983 (+1475103us) service_pool.cc:224] Handling call
0709 12:57:13.598579 (+ 11596us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.599238 (+   659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.599268 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:13.628597 (+ 29329us) log.cc:844] Serialized 26 byte log entry
0709 12:57:13.710761 (+ 82164us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:13.710812 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.710897 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.838694 (+127797us) raft_consensus.cc:1685] finished
0709 12:57:13.838711 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.838850 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":918,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":182,"thread_start_us":55004,"threads_started":1,"wal-append.queue_time_us":205498}
I20260709 12:57:13.895022 15947 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.825706 15847 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.903997 15847 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.904484 15947 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:13.904673 15928 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:13.906123 15761 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:13.906872 15761 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:13.883769 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:13.893086 15534 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:13.916292 13690 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:13.945791 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:13.947443 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:13.948724 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:13.949226 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:13.907153 15847 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:13.954814 13685 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:13.949672 13538 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:13.875515 15926 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.951942 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:13.916376 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 376) took 1812 ms. Trace:
I20260709 12:57:13.893122 16060 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:13.962292 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 49124.3us
I20260709 12:57:13.918682 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:13.925983 15846 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:13.980027 15984 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.776s	user 0.004s	sys 0.017s
I20260709 12:57:13.993258 15984 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:13.981765 14237 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:13.997262 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 471) took 3839 ms. Trace:
I20260709 12:57:13.982980 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1360) took 2436 ms. Trace:
I20260709 12:57:13.998405 14245 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 12:57:13.998886 14245 pending_rounds.cc:85] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 4
W20260709 12:57:13.983908 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 143450us
I20260709 12:57:13.999284 14245 pending_rounds.cc:107] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 12:57:13.999620 14245 raft_consensus.cc:2889] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:13.984486 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:14.000801 15789 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.734s	user 0.016s	sys 0.003s
I20260709 12:57:14.001274 15789 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:13.985163 16003 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:14.002568 13691 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:13.985760 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 301) took 1678 ms. Trace:
I20260709 12:57:13.986238 16011 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:13.955885 15989 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:14.005735 15989 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.006117 15989 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:14.006274 16016 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:14.008980 15977 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:14.003580 13686 rpcz_store.cc:276] 0709 12:57:12.283000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.082185 (+799185us) service_pool.cc:224] Handling call
0709 12:57:13.082832 (+   647us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:13.150662 (+ 67830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.150710 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:13.178840 (+ 28130us) log.cc:844] Serialized 52 byte log entry
0709 12:57:13.644207 (+465367us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:13.644811 (+   604us) log.cc:844] Serialized 14 byte log entry
0709 12:57:13.645157 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.645275 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.955851 (+310576us) raft_consensus.cc:1685] finished
0709 12:57:13.955880 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.961588 (+  5708us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1003,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":20186,"raft.run_cpu_time_us":528,"raft.run_wall_time_us":16925,"thread_start_us":160668,"threads_started":2,"wal-append.queue_time_us":310313}
I20260709 12:57:14.010572 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.011327 13686 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:14.013334 15980 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:14.018170 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:13.990850 13535 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:14.018971 13542 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:14.019760 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1184) took 1413 ms. Trace:
I20260709 12:57:14.020066 15850 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.020409 13535 rpcz_store.cc:276] 0709 12:57:12.606666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.840213 (+1233547us) service_pool.cc:224] Handling call
0709 12:57:13.952923 (+112710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.019346 (+ 66423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.019576 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:14.021730 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:14.023088 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:13.995775 15984 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.998801 13618 rpcz_store.cc:276] 0709 12:57:11.546898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.085653 (+538755us) service_pool.cc:224] Handling call
0709 12:57:13.800783 (+1715130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.802796 (+  2013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.982702 (+179906us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54002b5550. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:13.982897 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3942182784}
I20260709 12:57:14.028263 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 422) took 2053 ms. Trace:
I20260709 12:57:14.028812 14239 rpcz_store.cc:276] 0709 12:57:11.974928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.883353 (+1908425us) service_pool.cc:224] Handling call
0709 12:57:14.028144 (+144791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.029229 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.030625 13686 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:14.031132 15720 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:14.030447 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.031757 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:13.959685 13617 rpcz_store.cc:276] 0709 12:57:12.103834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.892664 (+1788830us) service_pool.cc:224] Handling call
0709 12:57:13.893087 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.895452 (+  2365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.895483 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:13.895551 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:13.895612 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.895673 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.895798 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:13.958405 15453 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:13.990819 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:14.034572 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 247129us
I20260709 12:57:14.030449 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2451) took 1917 ms. Trace:
I20260709 12:57:14.038985 13618 rpcz_store.cc:276] 0709 12:57:12.113483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.028815 (+1915332us) service_pool.cc:224] Handling call
0709 12:57:14.029245 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.029520 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.029545 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:14.029605 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:14.029655 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.029705 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.030135 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.032680 14239 raft_consensus.cc:2368] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
W20260709 12:57:14.032788 16069 log.cc:927] Time spent T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.264s	user 0.002s	sys 0.003s
I20260709 12:57:14.052978 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:14.055907 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 111544us
I20260709 12:57:14.058372 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.058765 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.043701 15847 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:57:14.064136 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 189578us
W20260709 12:57:14.066155 15959 consensus_meta.cc:287] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.523s	user 0.003s	sys 0.013s
I20260709 12:57:14.045596 15789 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:14.060382 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:14.060755 13692 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:14.066670 15959 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.049048 15947 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:14.088768 13617 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:14.087515 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 45166.3us
I20260709 12:57:14.097026 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2452) took 1980 ms. Trace:
I20260709 12:57:14.097533 13618 rpcz_store.cc:276] 0709 12:57:12.116928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.054590 (+1937662us) service_pool.cc:224] Handling call
0709 12:57:14.096426 (+ 41836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.096659 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.096685 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:14.096729 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:14.096763 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.096815 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.096925 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.031816 15720 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.099560 13618 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
W20260709 12:57:14.099891 15777 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.641s	user 0.013s	sys 0.005s
I20260709 12:57:14.102762 15720 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:14.101267 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:14.103396 15814 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:14.088789 15947 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.104477 15947 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:14.104727 15969 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:14.105436 15969 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:14.105839 15969 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:14.087034 15453 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.102069 15958 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:14.096916 16053 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:13.997929 14237 rpcz_store.cc:276] 0709 12:57:10.158145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.171505 (+ 13360us) service_pool.cc:224] Handling call
0709 12:57:11.627256 (+1455751us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:11.644993 (+ 17737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.981335 (+2336342us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b54004b49a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 12:57:13.996645 (+ 15310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.997189 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160889600}
I20260709 12:57:14.123615 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:14.114414 15534 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.115196 15957 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.115708 13534 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:57:14.101559 15877 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:57:14.116231 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2454) took 1995 ms. Trace:
W20260709 12:57:14.126547 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 66260.9us
I20260709 12:57:14.126685 13618 rpcz_store.cc:276] 0709 12:57:12.120855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.098605 (+1977750us) service_pool.cc:224] Handling call
0709 12:57:14.099013 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.111994 (+ 12981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.116176 (+  4182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.116109 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.117199 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 378) took 1972 ms. Trace:
I20260709 12:57:14.119520 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.065860 15846 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:14.128204 15846 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:14.128324 13692 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:14.128957 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:14.129359 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.129896 14237 raft_consensus.cc:2368] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:14.130556 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.130857 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.115895 15969 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.132349 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:14.133740 15969 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.133869 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.130764 15957 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.134652 14209 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:14.135355 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:14.135870 15958 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:14.158656 15958 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:14.160231 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 55755us
I20260709 12:57:14.160341 15958 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.161615 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.162886 15957 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:14.163057 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.163352 15957 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:14.136121 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1160) took 2047 ms. Trace:
I20260709 12:57:14.137140 15928 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:14.163821 15957 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:14.164690 15972 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:14.165166 15972 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:14.165601 15972 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:14.165663 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.165894 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:14.166724 13690 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:14.167383 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.139793 15926 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.141821 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1226) took 1811 ms. Trace:
I20260709 12:57:14.131742 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.175676 13685 rpcz_store.cc:276] 0709 12:57:12.330778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.789262 (+1458484us) service_pool.cc:224] Handling call
0709 12:57:13.945630 (+156368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.955460 (+  9830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.141609 (+186149us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540031e050. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:14.141735 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358597504}
I20260709 12:57:14.127961 13617 rpcz_store.cc:276] 0709 12:57:12.117494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.052362 (+1934868us) service_pool.cc:224] Handling call
0709 12:57:14.084568 (+ 32206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.089299 (+  4731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.089497 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.123809 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1371) took 2398 ms. Trace:
I20260709 12:57:14.087061 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 775) took 3434 ms. Trace:
W20260709 12:57:14.169477 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.087446 14240 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:14.167619 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.179937 13690 raft_consensus.cc:2410] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:57:14.084959 16060 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:57:14.139289 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 412) took 3972 ms. Trace:
I20260709 12:57:14.087237 15949 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
W20260709 12:57:14.176026 15903 consensus_meta.cc:287] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.002s
I20260709 12:57:14.192272 15903 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.194650 15947 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.196123 13612 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:14.197125 15453 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:14.196007 14240 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 420) took 2251 ms (client timeout 2520 ms). Trace:
I20260709 12:57:14.186981 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.163972 13540 rpcz_store.cc:276] 0709 12:57:12.088777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.458133 (+1369356us) service_pool.cc:224] Handling call
0709 12:57:13.458905 (+   772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.562135 (+103230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.562179 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:13.588363 (+ 26184us) log.cc:844] Serialized 26 byte log entry
0709 12:57:13.676534 (+ 88171us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:13.676666 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.677028 (+   362us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:14.031676 (+354648us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:14.125403 (+ 93727us) raft_consensus.cc:1685] finished
0709 12:57:14.125426 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.135778 (+ 10352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38617,"raft.queue_time_us":67772,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":257,"spinlock_wait_cycles":14315264,"thread_start_us":66911,"threads_started":1,"wal-append.queue_time_us":164200}
I20260709 12:57:14.201242 13686 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:14.177184 13614 rpcz_store.cc:276] 0709 12:57:11.550104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.275519 (+725415us) service_pool.cc:224] Handling call
0709 12:57:12.276011 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:12.331744 (+ 55733us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:57:13.473089 (+1141345us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b180034c840. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 12:57:13.609483 (+136394us) log.cc:844] Serialized 14 byte log entry
0709 12:57:13.721990 (+112507us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:13.869637 (+147647us) log.cc:844] Serialized 26 byte log entry
0709 12:57:13.870160 (+   523us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:13.875693 (+  5533us) log.cc:844] Serialized 14 byte log entry
0709 12:57:13.903506 (+ 27813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.908853 (+  5347us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.923371 (+ 14518us) raft_consensus.cc:1685] finished
0709 12:57:13.923407 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.948411 (+ 25004us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93627,"prepare.queue_time_us":1402897,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":93689,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":38912,"spinlock_wait_cycles":25652480,"thread_start_us":169490,"threads_started":2,"wal-append.queue_time_us":255605}
I20260709 12:57:14.203130 13506 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:14.195734 15903 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.229045 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:14.231678 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49676.1us
I20260709 12:57:14.233203 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:14.200081 15877 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:14.197976 15976 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.239542 13685 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:57:14.244621 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1239) took 1633 ms. Trace:
I20260709 12:57:14.244957 13685 rpcz_store.cc:276] 0709 12:57:12.607112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.179653 (+1572541us) service_pool.cc:224] Handling call
0709 12:57:14.223605 (+ 43952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.239963 (+ 16358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.240169 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.205158 15928 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:57:14.247663 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 47064.9us
W20260709 12:57:14.198835 14240 rpcz_store.cc:269] 0709 12:57:11.933124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.760655 (+1827531us) service_pool.cc:224] Handling call
0709 12:57:14.181363 (+420708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.201789 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:14.230479 16053 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:14.253016 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 108141us
I20260709 12:57:14.238750 15926 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:14.254136 13614 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:14.239074 15877 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.243673 15814 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:57:14.267020 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.255182 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2457) took 2092 ms. Trace:
I20260709 12:57:14.281005 13614 rpcz_store.cc:276] 0709 12:57:12.162601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.223947 (+2061346us) service_pool.cc:224] Handling call
0709 12:57:14.251589 (+ 27642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.254973 (+  3384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.255136 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.282302 14246 raft_consensus.cc:2410] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:57:14.285000 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.185047 14241 rpcz_store.cc:276] 0709 12:57:10.166902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.438776 (+271874us) service_pool.cc:224] Handling call
0709 12:57:13.804397 (+3365621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.805617 (+  1220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.139076 (+333459us) spinlock_profiling.cc:243] Waited 3.36 s on lock 0x7b54005d0a50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:14.139225 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7739238784}
I20260709 12:57:14.235692 15761 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.260229 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:14.250051 15928 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:14.292515 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.293547 14243 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:14.253109 16053 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.257966 15926 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.298650 13685 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:14.299468 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2611) took 1995 ms. Trace:
I20260709 12:57:14.300215 16053 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:14.300243 15926 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:14.301148 15791 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.120s	user 0.005s	sys 0.016s
I20260709 12:57:14.301949 15791 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.303897 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.304145 13539 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:14.304672 14241 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:14.304788 15761 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.305958 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.264277 15877 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:14.312222 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 60445.2us
W20260709 12:57:14.318173 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.284895 15814 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:14.298403 15928 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:14.341428 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:14.342577 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.342705 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:14.343405 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 57406.3us
I20260709 12:57:14.344051 16050 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:14.346279 15814 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.314416 15959 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:14.347462 15534 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.300035 13688 rpcz_store.cc:276] 0709 12:57:12.245400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.246344 (+   944us) service_pool.cc:224] Handling call
0709 12:57:12.246754 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:12.312105 (+ 65351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:12.312314 (+   209us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:12.313111 (+   797us) log.cc:844] Serialized 52 byte log entry
0709 12:57:13.619118 (+1306007us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b4800212958. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:57:13.818356 (+199238us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b18002f4760. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1
0709 12:57:13.896646 (+ 78290us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:13.971087 (+ 74441us) log.cc:844] Serialized 14 byte log entry
0709 12:57:14.203141 (+232054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.231363 (+ 28222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:14.237534 (+  6171us) raft_consensus.cc:1685] finished
0709 12:57:14.237561 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.240737 (+  3176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35125,"prepare.queue_time_us":41882,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":93,"raft.queue_time_us":176432,"raft.run_cpu_time_us":15939,"raft.run_wall_time_us":1317275,"spinlock_wait_cycles":67528704,"thread_start_us":228244,"threads_started":2,"wal-append.queue_time_us":548805,"wal-append.run_cpu_time_us":2450,"wal-append.run_wall_time_us":1115623}
W20260709 12:57:14.349088 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.350440 15976 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:14.314416 15791 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:14.325013 15761 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:14.352461 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.311779 15847 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:14.354863 13508 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:14.307868 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.366816 13614 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:14.368196 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:14.370785 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.382122 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2458) took 2210 ms. Trace:
I20260709 12:57:14.382647 13614 rpcz_store.cc:276] 0709 12:57:12.162888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.290657 (+2127769us) service_pool.cc:224] Handling call
0709 12:57:14.366561 (+ 75904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.370901 (+  4340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.373598 (+  2697us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.358897 16050 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.386405 15859 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:14.386868 15859 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.387213 15859 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:14.365124 15453 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:14.389533 15453 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.389923 15453 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:14.390498 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.399504 13614 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:14.400213 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2459) took 2237 ms. Trace:
I20260709 12:57:14.400506 13614 rpcz_store.cc:276] 0709 12:57:12.163142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.398847 (+2235705us) service_pool.cc:224] Handling call
0709 12:57:14.399265 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.399971 (+   706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.400152 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.383950 15761 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.370689 15847 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.372942 15903 raft_consensus.cc:2764] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:57:14.409642 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:14.410068 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.392248 15534 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:14.384827 15976 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.468609 13685 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:14.529208 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1182) took 1923 ms. Trace:
W20260709 12:57:14.503620 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 144568us
W20260709 12:57:14.453994 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 61015us
W20260709 12:57:14.536358 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.535216 13539 rpcz_store.cc:276] 0709 12:57:12.606073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.621305 (+1015232us) service_pool.cc:224] Handling call
0709 12:57:13.621929 (+   624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.305089 (+683160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.528849 (+223760us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b5400003054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:14.529115 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568437120}
I20260709 12:57:14.532876 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 374) took 2429 ms. Trace:
I20260709 12:57:14.499382 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 380) took 2343 ms. Trace:
W20260709 12:57:14.446461 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 49303.8us
I20260709 12:57:14.485837 13688 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:57:14.539006 13619 rpcz_store.cc:276] 0709 12:57:12.103081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.857426 (+1754345us) service_pool.cc:224] Handling call
0709 12:57:13.870605 (+ 13179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.057967 (+187362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.058008 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:14.063373 (+  5365us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:14.063443 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.532548 (+469105us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540013a6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:14.532606 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.532766 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377674496}
I20260709 12:57:14.392967 16050 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:14.539242 13617 rpcz_store.cc:276] 0709 12:57:12.156501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.183756 (+2027255us) service_pool.cc:224] Handling call
0709 12:57:14.236426 (+ 52670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.293284 (+ 56858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.293319 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:14.309228 (+ 15909us) log.cc:844] Serialized 26 byte log entry
0709 12:57:14.366218 (+ 56990us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:14.366258 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.366321 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:14.499048 (+132727us) raft_consensus.cc:1685] finished
0709 12:57:14.499072 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.499267 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12721,"raft.queue_time_us":12469,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":14905,"thread_start_us":56536,"threads_started":1,"wal-append.queue_time_us":186239}
I20260709 12:57:14.494606 15761 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:14.489928 15969 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:14.550264 15969 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.407879 15847 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:14.553740 16044 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:14.440912 14241 raft_consensus.cc:2393] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:14.494867 16068 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.709s	user 0.004s	sys 0.000s
I20260709 12:57:14.455665 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.573751 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:14.573096 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:14.446010 15470 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:14.532112 14238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 782) took 2553 ms (client timeout 2520 ms). Trace:
I20260709 12:57:14.583151 13685 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:57:14.583585 13539 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:14.567632 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.589134 15627 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:14.440691 15989 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.575651 15969 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:14.454345 15865 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.135s
I20260709 12:57:14.605904 15470 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.612577 15789 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:14.612907 15627 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:14.588327 14238 rpcz_store.cc:269] 0709 12:57:11.905515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.906618 (+  1103us) service_pool.cc:224] Handling call
0709 12:57:14.370293 (+2463675us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54007201d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:14.457385 (+ 87092us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2799944448}
I20260709 12:57:14.625057 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:14.178797 14242 rpcz_store.cc:276] 0709 12:57:10.422492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.588220 (+165728us) service_pool.cc:224] Handling call
0709 12:57:10.670152 (+ 81932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:10.728722 (+ 58570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:10.746320 (+ 17598us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:10.865099 (+118779us) log.cc:844] Serialized 26 byte log entry
0709 12:57:11.818728 (+953629us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:11.843635 (+ 24907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:11.850026 (+  6391us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:12.007413 (+157387us) raft_consensus.cc:1685] finished
0709 12:57:12.007436 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:13.821552 (+1814116us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b5400497ad0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:13.856884 (+ 35332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66101,"raft.queue_time_us":34014,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":7226,"spinlock_wait_cycles":176991232,"thread_start_us":937136,"threads_started":1,"wal-append.queue_time_us":978879,"wal-append.run_cpu_time_us":2625,"wal-append.run_wall_time_us":1160207}
I20260709 12:57:14.636497 13614 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:57:14.630966 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:14.652302 13539 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:14.618260 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.657146 15833 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:14.668612 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 77333.2us
I20260709 12:57:14.673226 15789 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.691248 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:14.694271 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:14.696542 16059 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.998s	user 0.000s	sys 0.005s
I20260709 12:57:14.699092 15453 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.701951 15453 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:14.703128 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.704677 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:14.705211 13658 leader_election.cc:400] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:14.705814 13536 raft_consensus.cc:2410] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:14.705698 13658 leader_election.cc:403] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:57:14.669114 13508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 2.520s (SENT) [suppressed 22 similar messages]
I20260709 12:57:14.721329 13535 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:14.722975 13536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1259) took 2729 ms (client timeout 3100 ms). Trace:
W20260709 12:57:14.723637 13536 rpcz_store.cc:269] 0709 12:57:11.991055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:11.991287 (+   232us) service_pool.cc:224] Handling call
0709 12:57:14.704282 (+2712995us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:14.720760 (+ 16478us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33180544}
I20260709 12:57:14.627636 15846 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.728585 13658 leader_election.cc:400] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:14.729574 13658 leader_election.cc:403] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:57:14.729382 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 311) took 1182 ms. Trace:
I20260709 12:57:14.730293 13690 rpcz_store.cc:276] 0709 12:57:13.546953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.198913 (+651960us) service_pool.cc:224] Handling call
0709 12:57:14.238409 (+ 39496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.291152 (+ 52743us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:14.340308 (+ 49156us) log.cc:844] Serialized 14 byte log entry
0709 12:57:14.729005 (+388697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:14.729059 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:14.729108 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.729195 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.729325 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6213,"prepare.queue_time_us":59972,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":316055,"threads_started":1}
I20260709 12:57:14.731413 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 300) took 2426 ms. Trace:
I20260709 12:57:14.728688 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 309) took 2441 ms. Trace:
I20260709 12:57:14.723340 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:14.735641 14242 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:14.745661 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:14.747751 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:14.748624 13539 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:14.726823 15976 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.732427 13693 rpcz_store.cc:276] 0709 12:57:12.287452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.594052 (+1306600us) service_pool.cc:224] Handling call
0709 12:57:13.594564 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:13.595268 (+   704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.595309 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:13.605963 (+ 10654us) log.cc:844] Serialized 26 byte log entry
0709 12:57:13.674865 (+ 68902us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:13.674924 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.675024 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:14.455873 (+780849us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:14.724016 (+268143us) raft_consensus.cc:1685] finished
0709 12:57:14.724039 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.728590 (+  4551us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1594,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":10396,"spinlock_wait_cycles":61494656,"thread_start_us":66809,"threads_started":1,"wal-append.queue_time_us":178238}
I20260709 12:57:14.754750 13690 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:57:14.755708 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2620) took 1109 ms. Trace:
I20260709 12:57:14.756140 13690 rpcz_store.cc:276] 0709 12:57:13.646338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.743325 (+1096987us) service_pool.cc:224] Handling call
0709 12:57:14.753412 (+ 10087us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.755358 (+  1946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.755639 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.758157 13690 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:14.732017 13684 rpcz_store.cc:276] 0709 12:57:12.282526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.954750 (+672224us) service_pool.cc:224] Handling call
0709 12:57:13.045475 (+ 90725us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:13.046013 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:13.046056 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:13.062269 (+ 16213us) log.cc:844] Serialized 52 byte log entry
0709 12:57:13.242398 (+180129us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:13.248843 (+  6445us) log.cc:844] Serialized 14 byte log entry
0709 12:57:13.472163 (+223320us) log.cc:844] Serialized 14 byte log entry
0709 12:57:13.472802 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:13.473020 (+   218us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:13.476263 (+  3243us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b180045e9e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:14.675437 (+1199174us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:14.708866 (+ 33429us) raft_consensus.cc:1685] finished
0709 12:57:14.708898 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.709161 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74487,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":7072,"raft.queue_time_us":1095,"raft.run_cpu_time_us":748,"raft.run_wall_time_us":18488,"spinlock_wait_cycles":454645760,"thread_start_us":93941,"threads_started":1,"wal-append.queue_time_us":462704}
I20260709 12:57:14.763224 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:14.764094 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:14.743530 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:14.770581 13539 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:14.772081 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.769845 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2621) took 1123 ms. Trace:
I20260709 12:57:14.776193 13693 rpcz_store.cc:276] 0709 12:57:13.646634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.752440 (+1105806us) service_pool.cc:224] Handling call
0709 12:57:14.765950 (+ 13510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.769427 (+  3477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.769455 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:14.769502 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:14.769546 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.769611 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.769781 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.775964 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1227) took 2431 ms. Trace:
I20260709 12:57:14.782482 13691 rpcz_store.cc:276] 0709 12:57:12.344356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.838232 (+1493876us) service_pool.cc:224] Handling call
0709 12:57:13.849061 (+ 10829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.002996 (+153935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.775604 (+772608us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54002ee5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:14.775858 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352564096}
W20260709 12:57:14.767640 15957 consensus_meta.cc:287] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.601s	user 0.014s	sys 0.000s
W20260709 12:57:14.769280 16088 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:57:14.760484 16084 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
W20260709 12:57:14.778931 16086 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.271s	user 0.002s	sys 0.000s
I20260709 12:57:14.745679 15926 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.795419 13691 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:14.797139 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:14.803364 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 789) took 1023 ms. Trace:
I20260709 12:57:14.803711 14242 rpcz_store.cc:276] 0709 12:57:13.780361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.749100 (+968739us) service_pool.cc:224] Handling call
0709 12:57:14.803306 (+ 54206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.786026 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:14.808557 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.810678 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2448) took 2764 ms. Trace:
I20260709 12:57:14.811313 15593 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:14.811239 13611 rpcz_store.cc:276] 0709 12:57:12.045968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.243970 (+2198002us) service_pool.cc:224] Handling call
0709 12:57:14.294507 (+ 50537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.328430 (+ 33923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.328480 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:14.339293 (+ 10813us) log.cc:844] Serialized 26 byte log entry
0709 12:57:14.717216 (+377923us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:14.717260 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.717332 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:14.810475 (+ 93143us) raft_consensus.cc:1685] finished
0709 12:57:14.810498 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.810616 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27965,"raft.queue_time_us":33677,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":7568,"thread_start_us":307758,"threads_started":1,"wal-append.queue_time_us":392054}
I20260709 12:57:14.811967 15593 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:14.813118 15593 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:14.813279 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.814226 14237 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:14.816475 15453 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:14.817094 15453 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:14.826167 13542 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:14.827083 13535 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:14.832309 15969 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:57:14.837252 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.840435 13611 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:57:14.841424 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 385) took 1881 ms. Trace:
I20260709 12:57:14.841748 13611 rpcz_store.cc:276] 0709 12:57:12.963377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.813326 (+1849949us) service_pool.cc:224] Handling call
0709 12:57:14.819516 (+  6190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.841136 (+ 21620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.841363 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:14.842562 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 92913.4us
W20260709 12:57:14.846755 16016 consensus_meta.cc:287] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.839s	user 0.014s	sys 0.002s
I20260709 12:57:14.847604 13610 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:14.847975 13535 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:14.848351 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2449) took 2802 ms. Trace:
I20260709 12:57:14.848732 13610 rpcz_store.cc:276] 0709 12:57:12.046295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.811788 (+2765493us) service_pool.cc:224] Handling call
0709 12:57:14.819043 (+  7255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.848079 (+ 29036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.848294 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:14.792538 15957 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:14.853935 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 219091us
W20260709 12:57:14.837021 13509 connection.cc:256] RPC call timeout handler was delayed by -1.3579447359998085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 12:57:14.856683 13539 raft_consensus.cc:2410] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:57:14.858366 13509 connection.cc:256] RPC call timeout handler was delayed by -1.1360833130001993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
W20260709 12:57:14.858808 13509 connection.cc:256] RPC call timeout handler was delayed by -1.1037505760004933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3294s for a response
I20260709 12:57:14.836678 15989 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:14.859259 13509 connection.cc:256] RPC call timeout handler was delayed by -1.0223268710005868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
I20260709 12:57:14.839727 15976 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:14.869741 13686 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:14.873659 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 105157us
I20260709 12:57:14.852725 15789 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:14.854372 16016 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:14.877537 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:14.880416 13540 raft_consensus.cc:2368] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:14.904415 13611 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:14.905576 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 387) took 1800 ms. Trace:
I20260709 12:57:14.906064 13611 rpcz_store.cc:276] 0709 12:57:13.104924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.842352 (+1737428us) service_pool.cc:224] Handling call
0709 12:57:14.879443 (+ 37091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.905235 (+ 25792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.905508 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:14.889079 14244 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 493) took 2778 ms (client timeout 2244 ms). Trace:
W20260709 12:57:14.917940 14244 rpcz_store.cc:269] 0709 12:57:12.110232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.888823 (+2778591us) service_pool.cc:211] Skipping call since client already timed out
0709 12:57:14.888977 (+   154us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:57:14.889724 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:14.876505 15969 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 12:57:14.889575 15972 consensus_meta.cc:287] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.006s
I20260709 12:57:14.919595 14212 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:14.933075 16097 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:57:14.934535 13543 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:14.935573 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2619) took 1310 ms. Trace:
I20260709 12:57:14.936338 13543 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:14.936259 13685 rpcz_store.cc:276] 0709 12:57:13.624794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.614627 (+989833us) service_pool.cc:224] Handling call
0709 12:57:14.615135 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.615672 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.615699 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:14.616688 (+   989us) log.cc:844] Serialized 26 byte log entry
0709 12:57:14.760891 (+144203us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:14.760949 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.761038 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:14.935216 (+174178us) raft_consensus.cc:1685] finished
0709 12:57:14.935239 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:14.935377 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":223,"thread_start_us":143280,"threads_started":1,"wal-append.queue_time_us":224088}
I20260709 12:57:14.938696 13685 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:14.940584 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:14.941105 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 381) took 2747 ms. Trace:
I20260709 12:57:14.941555 13614 rpcz_store.cc:276] 0709 12:57:12.193471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.401407 (+2207936us) service_pool.cc:224] Handling call
0709 12:57:14.402088 (+   681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.641964 (+239876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.940845 (+298881us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540026e154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:14.941049 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536937856}
I20260709 12:57:14.941200 15989 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:57:14.944140 15453 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:14.946230 16087 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:57:14.946588 15859 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.559s	user 0.015s	sys 0.000s
I20260709 12:57:14.886905 15976 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:14.944173 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:14.952044 13688 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:14.928613 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:14.952258 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 375) took 2848 ms. Trace:
I20260709 12:57:14.918020 14240 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:14.954030 13612 rpcz_store.cc:276] 0709 12:57:12.103489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.875854 (+1772365us) service_pool.cc:224] Handling call
0709 12:57:13.876678 (+   824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.196602 (+319924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.951707 (+755105us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54001293d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:14.952164 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734172544}
W20260709 12:57:14.957638 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 87585us
I20260709 12:57:14.931515 15470 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:14.965623 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 75527.5us
W20260709 12:57:14.985160 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 127744us
W20260709 12:57:14.986021 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 316993us
I20260709 12:57:14.953512 15761 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
I20260709 12:57:14.956007 15663 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:14.964993 15720 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:15.042490 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.050988 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1192) took 1274 ms. Trace:
I20260709 12:57:15.051724 13541 rpcz_store.cc:276] 0709 12:57:13.776390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.987616 (+1211226us) service_pool.cc:224] Handling call
0709 12:57:15.050917 (+ 63301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:15.053436 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:15.053429 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2633) took 1010 ms. Trace:
W20260709 12:57:15.066267 13584 connection.cc:256] RPC call timeout handler was delayed by -1.528250613999262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:57:15.055554 13691 rpcz_store.cc:276] 0709 12:57:14.043014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.799754 (+756740us) service_pool.cc:224] Handling call
0709 12:57:14.828031 (+ 28277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.867008 (+ 38977us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:14.919349 (+ 52341us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.052434 (+133085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:15.052545 (+   111us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:15.052613 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.052707 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.053360 (+   653us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23279,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"thread_start_us":118780,"threads_started":1}
I20260709 12:57:15.068158 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:14.990535 15453 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:14.951453 15972 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:14.956620 15859 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:15.080464 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 60410.1us
I20260709 12:57:15.081074 16104 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:15.081552 15957 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:15.042646 15887 consensus_queue.cc:1059] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:57:15.084509 15976 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:15.086061 13658 connection.cc:256] RPC call timeout handler was delayed by -1.2414494869999544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 12:57:15.213336 13658 connection.cc:256] RPC call timeout handler was delayed by -1.1262494869997681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 12:57:15.096411 14244 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:15.091375 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2445) took 3046 ms. Trace:
I20260709 12:57:15.043686 15926 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.110762 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.112423 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:15.113303 15947 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.113606 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:15.115490 15847 consensus_meta.cc:287] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.002s
I20260709 12:57:15.221108 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2630) took 1353 ms. Trace:
I20260709 12:57:15.116273 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:15.117975 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:15.146276 15928 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.003s
I20260709 12:57:15.100194 15720 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:15.153394 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2627) took 1375 ms. Trace:
I20260709 12:57:15.153869 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:15.161751 15958 consensus_queue.cc:1059] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:15.163038 15946 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:57:15.155236 16079 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:57:15.204388 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:15.247556 13690 rpcz_store.cc:276] 0709 12:57:13.778107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.759141 (+981034us) service_pool.cc:224] Handling call
0709 12:57:14.759749 (+   608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.760428 (+   679us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:14.781605 (+ 21177us) log.cc:844] Serialized 14 byte log entry
0709 12:57:14.987185 (+205580us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.152754 (+165569us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:15.153004 (+   250us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:15.153061 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.153158 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.153314 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36519,"prepare.queue_time_us":22162,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":92,"raft.queue_time_us":45138,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":47646,"spinlock_wait_cycles":56674560,"thread_start_us":328870,"threads_started":2}
I20260709 12:57:15.209084 13537 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:15.214414 13615 rpcz_store.cc:276] 0709 12:57:12.044924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.276622 (+231698us) service_pool.cc:224] Handling call
0709 12:57:12.338263 (+ 61641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:13.545183 (+1206920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.042152 (+1496969us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b540030a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:15.091294 (+ 49142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921885952}
I20260709 12:57:15.225414 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:15.251104 13614 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:57:15.194700 15453 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:15.230984 13685 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:15.229946 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:15.208560 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.265115 13540 raft_consensus.cc:2410] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:57:15.230314 13684 rpcz_store.cc:276] 0709 12:57:13.868171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.772215 (+904044us) service_pool.cc:224] Handling call
0709 12:57:14.785485 (+ 13270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.801360 (+ 15875us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:14.839830 (+ 38470us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.220626 (+380796us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:15.220696 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:15.220777 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.220868 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.221031 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29366,"prepare.queue_time_us":38158,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":48,"raft.queue_time_us":94337,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":15684,"thread_start_us":336405,"threads_started":2,"wal-append.queue_time_us":373047}
I20260709 12:57:15.214660 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:15.278234 13611 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:15.296490 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 66648.5us
I20260709 12:57:15.296600 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:15.303151 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.264173 15453 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:15.263439 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:15.267352 15947 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:15.317385 15947 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:15.320796 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:15.321384 15655 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:15.321894 15655 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:15.325157 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 267943us
I20260709 12:57:15.325634 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2614) took 1703 ms. Trace:
I20260709 12:57:15.325984 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1242) took 1430 ms. Trace:
I20260709 12:57:15.326587 13693 rpcz_store.cc:276] 0709 12:57:13.895437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.777889 (+882452us) service_pool.cc:224] Handling call
0709 12:57:14.793122 (+ 15233us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.819690 (+ 26568us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:14.858236 (+ 38546us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.186330 (+328094us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.186535 (+   205us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:15.227556 (+ 41021us) log.cc:844] Serialized 26 byte log entry
0709 12:57:15.227784 (+   228us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:15.227837 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.227954 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:15.325778 (+ 97824us) raft_consensus.cc:1685] finished
0709 12:57:15.325798 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.325924 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20719,"prepare.queue_time_us":38403,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":3467,"raft.queue_time_us":24083,"raft.run_cpu_time_us":471,"raft.run_wall_time_us":30214,"thread_start_us":256146,"threads_started":1,"wal-append.queue_time_us":423539}
I20260709 12:57:15.328636 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:15.329262 15859 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.329775 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 414) took 4886 ms. Trace:
I20260709 12:57:15.329927 15859 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:15.330474 15859 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:15.329285 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:15.260598 15928 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:15.332144 16092 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.780s	user 0.005s	sys 0.000s
I20260709 12:57:15.333395 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:15.333925 15928 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.330152 14245 rpcz_store.cc:276] 0709 12:57:10.443019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:10.688979 (+245960us) service_pool.cc:224] Handling call
0709 12:57:10.689539 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.030844 (+3341305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.329461 (+1298617us) spinlock_profiling.cc:243] Waited 3.31 s on lock 0x7b540011f154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:15.329678 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7607821056}
I20260709 12:57:15.335093 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.321940 13688 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:15.337819 15947 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.326115 13686 rpcz_store.cc:276] 0709 12:57:13.622581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.206329 (+583748us) service_pool.cc:224] Handling call
0709 12:57:14.235722 (+ 29393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:14.877180 (+641458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.325329 (+448149us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54002972d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:15.325566 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457524992}
I20260709 12:57:15.338637 15655 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:15.341791 16082 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.788s	user 0.005s	sys 0.001s
I20260709 12:57:15.342515 13616 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:15.303483 16091 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.754s	user 0.000s	sys 0.009s
I20260709 12:57:15.344484 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.349953 13614 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:15.242961 15847 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:15.296783 15976 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:15.329272 13693 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:57:15.354802 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 46541.4us
I20260709 12:57:15.357796 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:15.359724 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.362594 13684 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:15.367525 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:15.361747 15877 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.025s	user 0.017s	sys 0.000s
I20260709 12:57:15.375806 13619 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:57:15.377061 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 172411us
I20260709 12:57:15.377645 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.359581 15470 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:57:15.380736 13535 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:15.360359 15976 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:15.386585 14244 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:15.394351 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:15.395592 13584 connection.cc:256] RPC call timeout handler was delayed by -1.3113506139998208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 12:57:15.410820 13584 connection.cc:256] RPC call timeout handler was delayed by -1.2258506139996825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 12:57:15.411307 13584 connection.cc:256] RPC call timeout handler was delayed by -1.217487859999892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 12:57:15.411614 13584 connection.cc:256] RPC call timeout handler was delayed by -1.1979506139996374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
W20260709 12:57:15.411911 13584 connection.cc:256] RPC call timeout handler was delayed by -1.1826506139996127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
W20260709 12:57:15.412252 13584 connection.cc:256] RPC call timeout handler was delayed by -1.1790506139996069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
I20260709 12:57:15.372818 15663 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:57:15.375116 15877 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:15.368942 15926 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.114s
I20260709 12:57:15.378964 15720 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:15.683476 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:15.518015 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:15.686003 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:15.389866 16016 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:15.384413 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 431) took 1603 ms. Trace:
I20260709 12:57:15.465853 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:15.440482 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51894.5us
I20260709 12:57:15.686908 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:15.413462 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:15.378810 15655 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:15.675732 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:15.692836 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.548352 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:15.691207 13535 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:57:15.629978 13659 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.011s	sys 0.014s
W20260709 12:57:15.695094 13659 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.011s	sys 0.015s
W20260709 12:57:15.695552 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 604769us
I20260709 12:57:15.482486 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:15.696671 13614 raft_consensus.cc:2368] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6.
W20260709 12:57:15.704995 14243 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 486) took 3604 ms (client timeout 1787 ms). Trace:
I20260709 12:57:15.398981 15976 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:15.682672 15593 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.006s
W20260709 12:57:15.705734 14243 rpcz_store.cc:269] 0709 12:57:12.100652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.230289 (+129637us) service_pool.cc:224] Handling call
0709 12:57:15.687901 (+3457612us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b540005fd54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:15.704870 (+ 16969us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4401425280}
W20260709 12:57:15.548424 16113 log.cc:927] Time spent T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 12:57:15.716219 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:15.716970 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 337510us
W20260709 12:57:15.717240 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 336820us
I20260709 12:57:15.548784 16104 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:15.720237 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:15.685046 15887 raft_consensus.cc:3055] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:15.723798 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:15.442209 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:15.726090 13534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1187) took 2238 ms (client timeout 2246 ms). Trace:
W20260709 12:57:15.728338 16123 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
W20260709 12:57:15.788880 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 63274.1us
I20260709 12:57:15.827783 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:15.828002 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.730228 14243 raft_consensus.cc:2368] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:57:15.818933 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:15.731691 13692 raft_consensus.cc:2393] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:15.788322 15957 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:57:15.787395 13693 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:15.788002 16105 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.346s	user 0.000s	sys 0.001s
I20260709 12:57:15.719715 15593 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:15.833551 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 444) took 1447 ms. Trace:
I20260709 12:57:15.834945 15593 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:15.725571 13540 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
W20260709 12:57:15.715796 16050 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.164s	user 0.015s	sys 0.000s
I20260709 12:57:15.834156 13536 rpcz_store.cc:276] 0709 12:57:14.386115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.765319 (+379204us) service_pool.cc:224] Handling call
0709 12:57:14.780556 (+ 15237us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:57:14.790806 (+ 10250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.790835 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:57:14.803151 (+ 12316us) log.cc:844] Serialized 78 byte log entry
0709 12:57:15.108100 (+304949us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:15.391130 (+283030us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.620428 (+229298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.620508 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:15.833323 (+212815us) raft_consensus.cc:1685] finished
0709 12:57:15.833343 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.833492 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50338,"prepare.queue_time_us":81423,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":317416,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":15218,"thread_start_us":697613,"threads_started":3,"wal-append.queue_time_us":529089}
W20260709 12:57:15.857261 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:15.861070 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:15.860817 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70531.3us
W20260709 12:57:15.825074 16053 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.524s	user 0.014s	sys 0.003s
W20260709 12:57:15.863402 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 72558.9us
I20260709 12:57:15.710605 16016 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:15.774099 16103 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:57:15.728432 15887 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:15.817485 15859 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:15.825762 15847 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.860719 16050 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:15.956686 15627 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:15.883767 15951 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:15.960106 15877 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.962968 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2640) took 1607 ms. Trace:
I20260709 12:57:15.963416 13685 rpcz_store.cc:276] 0709 12:57:14.355214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.940777 (+585563us) service_pool.cc:224] Handling call
0709 12:57:14.941056 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:15.263059 (+322003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.962712 (+699653us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540042d1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:15.962894 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666325248}
I20260709 12:57:15.888125 16017 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:15.888947 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:15.970821 13614 raft_consensus.cc:2393] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:15.896077 16053 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:15.900151 16099 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:15.900413 15928 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:15.902598 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2450) took 3854 ms. Trace:
I20260709 12:57:15.770346 16104 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:15.905027 16142 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:15.910146 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 383) took 3036 ms. Trace:
I20260709 12:57:15.995451 13617 rpcz_store.cc:276] 0709 12:57:12.873963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.809148 (+1935185us) service_pool.cc:224] Handling call
0709 12:57:14.809634 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.962791 (+153157us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:15.273714 (+310923us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.685339 (+411625us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:15.792207 (+106868us) log.cc:844] Serialized 26 byte log entry
0709 12:57:15.792412 (+   205us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:15.794310 (+  1898us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.794606 (+   296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.794722 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:15.909928 (+115206us) raft_consensus.cc:1685] finished
0709 12:57:15.909947 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.910088 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19414,"prepare.queue_time_us":557429,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":134,"raft.queue_time_us":376223,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":16040,"spinlock_wait_cycles":12558720,"thread_start_us":647558,"threads_started":3,"wal-append.queue_time_us":565963}
I20260709 12:57:15.915098 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:15.838526 15972 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:15.969580 16017 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:15.929631 13684 consensus_meta.cc:287] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.003s
I20260709 12:57:16.001823 13684 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:16.002250 16017 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.003125 16104 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.003839 13688 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:15.696873 14239 rpcz_store.cc:276] 0709 12:57:13.775346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.197308 (+421962us) service_pool.cc:224] Handling call
0709 12:57:15.249775 (+1052467us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:15.378927 (+129152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3426048}
W20260709 12:57:16.004638 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 89724.7us
I20260709 12:57:16.005069 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 429) took 2256 ms. Trace:
I20260709 12:57:16.005592 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:16.006094 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:16.006924 13688 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:16.007128 15847 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:16.007500 15847 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:16.007982 15847 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:15.931870 13611 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.641s	user 0.004s	sys 0.004s
I20260709 12:57:16.010851 13611 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:15.999265 16119 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:16.012090 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:16.012292 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 622886us
I20260709 12:57:16.012544 16119 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.013640 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:16.013986 13686 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:57:15.728659 13534 rpcz_store.cc:269] 0709 12:57:13.483097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.579080 (+ 95983us) service_pool.cc:224] Handling call
0709 12:57:15.704513 (+2125433us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54003cb754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:15.719673 (+ 15160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163242752}
I20260709 12:57:16.014871 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.015692 16140 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:16.006928 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:16.016265 15887 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:16.016855 13692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1240) took 2122 ms (client timeout 2821 ms). Trace:
I20260709 12:57:16.017468 15903 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:16.017544 13692 rpcz_store.cc:269] 0709 12:57:13.894683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.131506 (+236823us) service_pool.cc:224] Handling call
0709 12:57:16.013606 (+1882100us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400159d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:16.016793 (+  3187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3556999936}
I20260709 12:57:16.017544 15887 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:16.019224 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:16.020265 13611 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.018178 15903 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:16.020750 15903 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:16.021311 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.022006 13611 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:15.933482 14241 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:16.016315 16140 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:15.838156 15946 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:16.022900 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.023049 15946 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:16.023401 16140 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.014957 16119 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:15.994801 13613 rpcz_store.cc:276] 0709 12:57:12.048128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.867976 (+2819848us) service_pool.cc:224] Handling call
0709 12:57:14.888918 (+ 20942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.928268 (+ 39350us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:57:14.928298 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:14.965318 (+ 37020us) log.cc:844] Serialized 26 byte log entry
0709 12:57:15.186050 (+220732us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:15.186105 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.186196 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:15.902134 (+715938us) raft_consensus.cc:1685] finished
0709 12:57:15.902162 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.902343 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":221645,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":134835,"threads_started":1,"wal-append.queue_time_us":303055}
I20260709 12:57:16.024056 15946 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.025300 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.025440 15946 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:16.007553 16053 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.026172 15928 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:15.837795 15470 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:16.027534 13615 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:15.999055 16142 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:16.028512 16142 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:16.029398 15989 consensus_meta.cc:287] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.086s	user 0.015s	sys 0.000s
I20260709 12:57:16.030459 16050 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.003659 16017 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:16.030723 15847 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.008916 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1249) took 1900 ms. Trace:
I20260709 12:57:16.032069 13689 rpcz_store.cc:276] 0709 12:57:14.108221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.808112 (+699891us) service_pool.cc:224] Handling call
0709 12:57:14.808677 (+   565us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:14.809431 (+   754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.809460 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:14.810690 (+  1230us) log.cc:844] Serialized 525 byte log entry
0709 12:57:15.147893 (+337203us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:15.147951 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.148016 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:16.008589 (+860573us) raft_consensus.cc:1685] finished
0709 12:57:16.008608 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:16.008741 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6781,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":90,"thread_start_us":336729,"threads_started":1,"wal-append.queue_time_us":531447}
I20260709 12:57:16.023823 13611 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:57:15.963891 13537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1165) took 3873 ms (client timeout 1898 ms). Trace:
I20260709 12:57:16.012147 15859 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:16.002995 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:16.065868 13684 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:16.042099 13659 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.528s (SENT)
W20260709 12:57:16.042526 13537 rpcz_store.cc:269] 0709 12:57:12.090147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.242809 (+152662us) service_pool.cc:224] Handling call
0709 12:57:15.963596 (+3720787us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b54000daa54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:15.963776 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6248060800}
I20260709 12:57:16.063762 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 382) took 3487 ms. Trace:
I20260709 12:57:16.067319 13619 rpcz_store.cc:276] 0709 12:57:12.576197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.568573 (+1992376us) service_pool.cc:224] Handling call
0709 12:57:14.628583 (+ 60010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:15.376318 (+747735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:16.063384 (+687066us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54003e12d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:16.063673 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715747200}
I20260709 12:57:15.964784 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:16.000511 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.005527 14242 rpcz_store.cc:276] 0709 12:57:13.743399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.824008 (+1080609us) service_pool.cc:224] Handling call
0709 12:57:14.855588 (+ 31580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.923756 (+ 68168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.923796 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:14.933115 (+  9319us) log.cc:844] Serialized 26 byte log entry
0709 12:57:15.088208 (+155093us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:15.088250 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.088552 (+   302us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:15.999729 (+911177us) raft_consensus.cc:1685] finished
0709 12:57:15.999751 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:15.999937 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36212,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":26455,"thread_start_us":154344,"threads_started":1,"wal-append.queue_time_us":327778}
I20260709 12:57:16.012833 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:15.960160 15951 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:16.089488 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 53573.7us
I20260709 12:57:16.091380 13659 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:16.035087 16125 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.353s	user 0.002s	sys 0.000s
I20260709 12:57:16.098515 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.099610 14245 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:57:16.100162 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 240914us
W20260709 12:57:16.078173 14246 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 491) took 3968 ms (client timeout 1934 ms). Trace:
W20260709 12:57:16.104441 14246 rpcz_store.cc:269] 0709 12:57:12.109666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:13.556333 (+1446667us) service_pool.cc:224] Handling call
0709 12:57:16.064109 (+2507776us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b5400601654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:16.078088 (+ 13979us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605550720}
I20260709 12:57:16.099926 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.107172 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:16.107556 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:16.110237 13611 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:16.110949 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.036623 16106 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:16.046064 15846 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.036118 16134 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:16.124529 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.125061 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:16.115984 15833 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.129174 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.129441 15833 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.130440 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.131145 13619 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.131489 15833 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:16.132213 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 96263.4us
I20260709 12:57:16.001087 16099 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.133177 15976 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.135464 16099 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.092448 15761 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:16.113176 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:16.038722 16016 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.132293 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:16.136044 13613 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:16.136183 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:16.168855 13611 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:16.136855 13543 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:16.169981 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:16.170570 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53498.9us
I20260709 12:57:16.170804 13536 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:16.171348 14212 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:16.158035 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61760.6us
I20260709 12:57:16.158754 13537 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:16.155949 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:16.163755 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65727.8us
I20260709 12:57:16.137637 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.210265 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.210621 13614 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.211022 13611 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.157649 13684 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:16.116638 16119 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:16.188910 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:16.217360 13614 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:16.214250 13619 raft_consensus.cc:2410] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:16.157405 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2609) took 4091 ms. Trace:
I20260709 12:57:16.169229 13539 raft_consensus.cc:2376] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5. Re-sending same reply.
I20260709 12:57:16.160372 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:16.222349 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:16.223033 13619 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:16.223486 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:16.156463 16135 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.224735 16135 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.227001 16135 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:16.228720 13508 leader_election.cc:336] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 1.515s (SENT)
I20260709 12:57:16.228886 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:16.229924 13541 raft_consensus.cc:2368] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5.
I20260709 12:57:16.230109 15663 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:16.218816 13687 rpcz_store.cc:276] 0709 12:57:12.001823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:12.078174 (+ 76351us) service_pool.cc:224] Handling call
0709 12:57:12.078423 (+   249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.944553 (+2866130us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b18003421c0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7aabfb 00007fa02e7b564b
0709 12:57:14.944785 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.944822 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:14.945458 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 12:57:14.999520 (+ 54062us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:15.000803 (+  1283us) log.cc:844] Serialized 14 byte log entry
0709 12:57:15.361219 (+360416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.958241 (+597022us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4800212958. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:57:16.064936 (+106695us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540015ead4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:16.064998 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:16.065061 (+    63us) raft_consensus.cc:1685] finished
0709 12:57:16.065080 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:16.093346 (+ 28266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51426,"prepare.queue_time_us":71465,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":141693,"raft.run_cpu_time_us":11815,"raft.run_wall_time_us":885029,"spinlock_wait_cycles":575444864,"thread_start_us":444502,"threads_started":4,"wal-append.queue_time_us":132469,"wal-append.run_cpu_time_us":2215,"wal-append.run_wall_time_us":1002230}
I20260709 12:57:16.231681 13534 raft_consensus.cc:2410] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:57:16.231873 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 372) took 4129 ms. Trace:
I20260709 12:57:16.232488 15627 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:16.140329 14239 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:16.234889 15789 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:16.143813 15951 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.236639 15926 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:16.237422 15947 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:57:16.238510 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 391) took 1123 ms. Trace:
W20260709 12:57:16.240000 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 68735.9us
I20260709 12:57:16.256274 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:16.233381 16148 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:57:16.145753 15453 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.788s	user 0.012s	sys 0.000s
I20260709 12:57:16.112761 15928 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.142657 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 427) took 2399 ms. Trace:
I20260709 12:57:16.288141 14241 rpcz_store.cc:276] 0709 12:57:13.742833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.765174 (+1022341us) service_pool.cc:224] Handling call
0709 12:57:14.831957 (+ 66783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:16.022678 (+1190721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:16.142278 (+119600us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54004c1dd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:16.142590 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2310277376}
I20260709 12:57:16.242100 13616 rpcz_store.cc:276] 0709 12:57:12.102509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:15.142733 (+3040224us) service_pool.cc:224] Handling call
0709 12:57:15.175959 (+ 33226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:15.343025 (+167066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:16.231596 (+888571us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002fc6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:16.231800 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382397824}
W20260709 12:57:16.292335 14209 connection.cc:256] RPC call timeout handler was delayed by -1.4567159839998567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
W20260709 12:57:16.293072 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.246603 13613 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
W20260709 12:57:16.293542 14209 connection.cc:256] RPC call timeout handler was delayed by -1.3106194009997125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260709 12:57:16.141615 15847 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.152285 16108 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59162 (request call id 495) took 2346 ms. Trace:
I20260709 12:57:16.139662 15761 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.154639 16127 raft_consensus.cc:3055] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:57:16.267269 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.251796 15789 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:16.280544 15859 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.283708 13582 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:16.260371 16135 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:57:16.284606 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 188547us
I20260709 12:57:16.284935 15928 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:16.285485 16053 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:16.285574 16119 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:16.359900 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:16.360376 16121 raft_consensus.cc:3055] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:57:16.360319 16119 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.360940 16121 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:16.291455 15453 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.362618 16119 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:16.363409 16146 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:16.363935 15859 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.364107 16146 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.295300 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:16.366339 16146 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:16.141765 16134 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.284606 16148 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:16.367627 16148 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:16.367435 15789 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.368753 15789 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:16.368624 16148 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.369283 13508 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:16.367311 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:16.251791 15833 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 12:57:16.340507 15968 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:16.371735 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:16.348688 16085 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.796s	user 0.005s	sys 0.000s
I20260709 12:57:16.349699 15761 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:16.352257 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 68656.7us
I20260709 12:57:16.353014 15846 raft_consensus.cc:2764] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:57:16.330077 16127 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:16.320878 16108 rpcz_store.cc:276] 0709 12:57:13.805623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.814872 (+1009249us) service_pool.cc:224] Handling call
0709 12:57:16.151520 (+1336648us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68e1a2 00007fa0345cfbb9 00007fa033700d88 00007fa0337c4a07 00007fa0337f6f4e 00007fa0337dbfdd 00007fa0337dbf89 00007fa0337dbf19 00007fa0337dbee1
0709 12:57:16.152063 (+   543us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89298176,"tablet-copy.queue_time_us":566481,"thread_start_us":286975,"threads_started":1}
I20260709 12:57:16.356617 13691 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:57:16.369154 16134 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.358230 15959 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:16.366354 15859 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.363555 15453 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.370496 16145 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:16.297997 15946 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.135888 15976 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.426438 15946 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.426910 16140 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:57:16.346117 15903 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.372557 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:16.208630 16142 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.357323 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:16.412106 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:16.439369 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 329565us
I20260709 12:57:16.362115 16121 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:16.421867 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 191018us
W20260709 12:57:16.427286 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52772us
I20260709 12:57:16.419883 15663 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:16.373502 15968 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:16.407552 15958 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:16.381659 16119 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:16.437754 13684 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:16.410204 15761 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:16.404968 16135 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:16.428977 15866 tablet_copy_client.cc:538] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:57:16.456281 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.438999 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:16.493628 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:16.493713 15761 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:16.466234 15859 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:16.460280 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:16.421164 15926 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:16.420397 15959 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:16.463333 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 51358.1us
I20260709 12:57:16.487834 16121 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:57:16.468479 16145 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:16.467145 13617 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.461616 15720 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 12:57:16.506986 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 66916.8us
I20260709 12:57:16.511078 15946 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.523674 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:16.527518 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:16.529775 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 441) took 2153 ms. Trace:
I20260709 12:57:16.530331 13542 rpcz_store.cc:276] 0709 12:57:14.370057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.374227 (+  4170us) service_pool.cc:224] Handling call
0709 12:57:14.389629 (+ 15402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:14.829515 (+439886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:16.523043 (+1693528us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54000ab254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:16.523245 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002423168}
I20260709 12:57:16.512148 16016 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:16.491596 16146 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:57:16.508260 15926 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:16.545181 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 379) took 4424 ms. Trace:
I20260709 12:57:16.493477 16119 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.499696 16134 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:57:16.518577 16136 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.559058 16155 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:16.560081 15989 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:16.561081 15846 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:16.573051 13542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c5bef52669b468c8765408698f5884d"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:16.542797 16016 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:16.611253 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.579947 13684 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:16.576252 13542 raft_consensus.cc:606] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:16.614859 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 244308us
I20260709 12:57:16.568430 15928 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:16.593061 16153 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:57:16.594161 16156 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:57:16.603096 15947 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:16.603556 15959 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.254447 13610 rpcz_store.cc:276] 0709 12:57:15.114727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:15.139173 (+ 24446us) service_pool.cc:224] Handling call
0709 12:57:15.152432 (+ 13259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:15.156499 (+  4067us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:15.450611 (+294112us) log.cc:844] Serialized 14 byte log entry
0709 12:57:16.055582 (+604971us) log.cc:844] Serialized 14 byte log entry
0709 12:57:16.238063 (+182481us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b180034daa0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 12:57:16.238214 (+   151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:16.238260 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:16.238300 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:16.238359 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:16.238460 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149886,"prepare.queue_time_us":392618,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":207383,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":11599,"spinlock_wait_cycles":80004736,"thread_start_us":699508,"threads_started":2}
I20260709 12:57:16.490932 16127 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.612021 13615 raft_consensus.cc:2376] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 3. Re-sending same reply.
I20260709 12:57:16.615945 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:16.564981 15859 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:16.621879 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 158473us
I20260709 12:57:16.546906 13618 rpcz_store.cc:276] 0709 12:57:12.121177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:14.168973 (+2047796us) service_pool.cc:224] Handling call
0709 12:57:14.189201 (+ 20228us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:14.189745 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:14.189773 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:14.190441 (+   668us) log.cc:844] Serialized 52 byte log entry
0709 12:57:14.279548 (+ 89107us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:14.297988 (+ 18440us) log.cc:844] Serialized 14 byte log entry
0709 12:57:14.328696 (+ 30708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:14.328767 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:16.262561 (+1933794us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:16.535396 (+272835us) raft_consensus.cc:1685] finished
0709 12:57:16.544550 (+  9154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:16.545094 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30107,"prepare.queue_time_us":17832,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":35920,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":5857,"spinlock_wait_cycles":14058752,"thread_start_us":65140,"threads_started":1,"wal-append.queue_time_us":361775}
W20260709 12:57:16.645602 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.614486 16016 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:16.523654 15968 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.531972 16145 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.617193 15928 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:16.608425 16157 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:16.652300 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.660632 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.631227 15947 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:16.662876 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.671268 13618 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.667402 13619 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:16.670953 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.576416 16136 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.696125 13614 raft_consensus.cc:2368] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8.
W20260709 12:57:16.735340 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 206680us
I20260709 12:57:16.736428 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.737531 13614 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.737650 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:16.736040 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 277445us
W20260709 12:57:16.710940 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 276337us
W20260709 12:57:16.739938 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 242148us
W20260709 12:57:16.740584 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 95003.7us
I20260709 12:57:16.658830 16120 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.746997 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.570183 16135 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.745216 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.668134 15928 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:16.746735 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.743858 13540 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:57:16.743782 15947 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.761247 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.762007 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:16.750015 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 110002us
I20260709 12:57:16.763000 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:16.696033 16153 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:57:16.751335 13617 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.746294 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.772765 13618 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:16.745131 16165 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:16.780066 13535 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:16.608592 15761 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.648990 16053 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 12:57:16.787492 13684 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: [5.7-6.8]
I20260709 12:57:16.742064 16142 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:16.768507 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:16.769080 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.734210 15972 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:16.772835 13509 leader_election.cc:336] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.577s (SENT)
I20260709 12:57:16.790581 13509 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:16.790597 15972 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.739041 13536 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:16.795070 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:16.795821 13536 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:16.782133 13584 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:16.734298 15903 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.779366 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.787375 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:16.759418 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:16.811074 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.811746 13536 raft_consensus.cc:2410] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:57:16.632833 15926 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:16.814584 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:16.817533 13537 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
I20260709 12:57:16.812525 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:16.818475 13539 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:57:16.818888 13539 pending_rounds.cc:85] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 4
I20260709 12:57:16.819126 13539 pending_rounds.cc:107] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:57:16.819357 13539 raft_consensus.cc:2889] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:16.820559 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:16.821259 13535 raft_consensus.cc:2368] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:57:16.785341 16084 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:57:16.827747 15972 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:16.828613 16160 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:57:16.773211 15958 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:16.829591 15972 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.829883 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1391) took 1037 ms. Trace:
I20260709 12:57:16.830240 15972 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.830499 15958 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.830276 13611 rpcz_store.cc:276] 0709 12:57:15.792973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.239824 (+446851us) service_pool.cc:224] Handling call
0709 12:57:16.240145 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:16.281158 (+ 41013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:16.281192 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:16.281713 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:57:16.505484 (+223771us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:57:16.505545 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:16.505666 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:16.829635 (+323969us) raft_consensus.cc:1685] finished
0709 12:57:16.829664 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:16.829819 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40614,"raft.queue_time_us":42022,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":223137,"threads_started":1,"wal-append.queue_time_us":363207}
I20260709 12:57:16.833951 15972 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.844604 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:16.845386 13686 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:16.859007 13685 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:57:16.845029 15968 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:16.868161 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.758195 16120 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.873503 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:16.874351 13690 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:16.875470 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 63915us
W20260709 12:57:16.890066 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 77610.5us
I20260709 12:57:16.869279 13611 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:16.854915 15846 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:16.853298 16142 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:16.867803 13536 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:57:16.860852 15947 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:16.789613 16165 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:16.917528 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:16.918704 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:16.915887 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:16.918917 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:16.877161 16173 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:57:16.927671 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 151286us
I20260709 12:57:16.923946 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:16.908749 16157 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:16.935079 16167 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:57:16.919795 15926 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:16.915637 16138 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.941504 13584 leader_election.cc:400] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:57:16.921053 15846 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:16.945004 13584 leader_election.cc:403] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:16.946051 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:16.949815 16142 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:16.928167 15957 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:16.949541 13690 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:16.936203 16182 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:16.962380 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:16.952351 16176 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:57:16.966998 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:16.967341 16157 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.969730 16165 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.982511 16119 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:16.982623 16120 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:16.984758 14239 consensus_meta.cc:287] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.704s	user 0.012s	sys 0.000s
I20260709 12:57:16.985222 14239 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:16.963812 16142 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:16.995177 16136 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:16.964105 16101 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:57:16.998212 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:16.989509 16142 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:16.960453 16084 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:17.012480 16101 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:17.017338 13611 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:57:17.019501 13611 pending_rounds.cc:85] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 2
I20260709 12:57:17.019775 13611 pending_rounds.cc:107] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:57:17.020107 13611 raft_consensus.cc:2889] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:57:17.006706 15789 consensus_meta.cc:287] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.006s
I20260709 12:57:16.962148 16182 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:16.960299 15957 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:16.953298 16138 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:17.044675 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 127731us
I20260709 12:57:17.013041 16084 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:17.022179 16161 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:57:17.013362 16176 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.067129 13611 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
W20260709 12:57:17.068627 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.050413 16182 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:17.077323 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 79248.6us
I20260709 12:57:17.062659 16084 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:17.082772 13691 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:17.083333 15865 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:17.083740 15865 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:17.084584 15865 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:17.086018 15865 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:17.092427 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.094087 13693 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:17.096308 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.070472 16145 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:17.100481 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:17.105849 13582 connection.cc:256] RPC call timeout handler was delayed by -1.0088428310000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2821s for a response
W20260709 12:57:17.109699 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.111150 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.111150 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.112540 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:17.110311 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.113451 13693 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:17.060137 16169 consensus_queue.cc:1059] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:57:17.110117 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:17.116760 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.117533 13538 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:17.116184 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.120406 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.121202 13688 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:17.121984 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:17.126932 13684 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:57:17.127367 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.131846 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.133806 13537 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:17.135300 13659 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:17.139705 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.143163 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:17.143949 13539 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:17.144802 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:17.145529 13535 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:17.145666 13541 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:17.136498 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.151789 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.152346 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.152956 13684 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:17.152963 13692 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 3.
W20260709 12:57:17.156569 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 46578.7us
I20260709 12:57:17.157424 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:17.158524 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.120942 16101 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:17.165257 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.166100 13688 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:17.116520 15968 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.170907 15968 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:17.173137 15968 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:17.174369 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:17.174672 15470 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:17.175024 15470 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:17.168795 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.169598 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.175081 13535 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:17.203998 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 147428us
W20260709 12:57:17.206637 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:17.166962 13584 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 2.246s (SENT)
I20260709 12:57:17.144918 16157 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:17.200304 13684 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:17.200500 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 65300.9us
I20260709 12:57:17.202338 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:17.070250 15846 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:17.159749 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:17.071962 16176 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:17.196233 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 83755.5us
I20260709 12:57:17.277050 13541 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:57:17.277365 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:17.229619 13540 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:17.137295 16171 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:17.282560 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.281144 13537 raft_consensus.cc:2376] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5. Re-sending same reply.
I20260709 12:57:17.305251 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:17.352129 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:17.302028 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 137338us
W20260709 12:57:17.353760 13584 connection.cc:256] RPC call timeout handler was delayed by -1.8400740939996467s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 12:57:17.283599 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52995.8us
W20260709 12:57:17.303830 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 146474us
W20260709 12:57:17.355710 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.356377 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:17.303045 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:17.357308 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.359189 13692 raft_consensus.cc:2410] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:57:17.360567 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 57645.8us
I20260709 12:57:17.360867 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.165369 15789 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:17.199086 16121 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:17.361954 13688 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
W20260709 12:57:17.281617 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 78924.1us
I20260709 12:57:17.365759 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.368345 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:17.357753 13611 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:17.201478 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:17.364562 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 82157.2us
I20260709 12:57:17.371112 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.363381 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:17.368655 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:17.199496 16169 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.170s
I20260709 12:57:17.369457 13537 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:57:17.370224 13536 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:17.375280 13582 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:17.379145 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.279299 15976 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:17.365695 13506 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:17.389070 13535 raft_consensus.cc:2410] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:17.389501 16172 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:57:17.393144 16175 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:57:17.394374 16129 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.556s	user 0.008s	sys 0.000s
I20260709 12:57:17.395407 16176 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:17.395711 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 388) took 2295 ms. Trace:
I20260709 12:57:17.396083 13612 rpcz_store.cc:276] 0709 12:57:15.113778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:15.114858 (+  1080us) service_pool.cc:224] Handling call
0709 12:57:15.115378 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:15.115890 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:15.116452 (+   562us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:15.152893 (+ 36441us) log.cc:844] Serialized 26 byte log entry
0709 12:57:15.368106 (+215213us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:15.368317 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:15.368409 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:17.353203 (+1984794us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:17.395416 (+ 42213us) raft_consensus.cc:1685] finished
0709 12:57:17.395443 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:17.395654 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1082,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":14902,"spinlock_wait_cycles":10979072,"thread_start_us":190146,"threads_started":1,"wal-append.queue_time_us":458492}
I20260709 12:57:17.398084 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.398782 13692 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:17.200711 16136 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.196s
I20260709 12:57:17.403133 13537 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:17.398084 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.412679 13612 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:17.404919 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.408921 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.354053 16183 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:17.376106 16171 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:17.413978 13535 raft_consensus.cc:2410] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:57:17.385984 16166 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:17.379098 16165 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:17.416087 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 50492.4us
I20260709 12:57:17.165807 15957 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:17.432698 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.439309 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.385787 16155 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:57:17.357669 16181 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:57:17.421485 16166 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:17.421453 16176 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:17.384594 16121 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:57:17.456600 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 66183us
I20260709 12:57:17.469751 16166 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:17.436476 16175 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:57:17.469491 16176 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:17.478765 16172 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:57:17.491349 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:17.493683 13618 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:17.494752 15761 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.504503 15761 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:17.506690 15761 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:17.509420 15928 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.001s
I20260709 12:57:17.509927 15928 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:17.512188 16203 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.513157 16203 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:17.516794 16189 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:57:17.518318 16199 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 12:57:17.520279 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.521183 13618 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:17.521288 16016 consensus_meta.cc:287] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.860s	user 0.010s	sys 0.002s
I20260709 12:57:17.521900 16016 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:17.524679 13658 leader_election.cc:400] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5.
I20260709 12:57:17.525046 13658 leader_election.cc:403] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:57:17.526408 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:17.528384 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:17.487743 16176 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:17.531982 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.544180 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.544453 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.541375 16173 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:17.545588 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.546403 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 800) took 1211 ms. Trace:
I20260709 12:57:17.546729 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.546801 14241 rpcz_store.cc:276] 0709 12:57:16.335085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.344052 (+  8967us) service_pool.cc:224] Handling call
0709 12:57:17.545590 (+1201538us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:17.545791 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:17.546022 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:17.546053 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:17.546100 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:17.546144 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:17.546203 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:17.546337 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8399744}
I20260709 12:57:17.548230 16173 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:17.548408 15926 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.001s
I20260709 12:57:17.530809 15789 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5.
I20260709 12:57:17.550489 15959 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:17.464761 15866 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:57:17.412446 16120 consensus_queue.cc:1059] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:57:17.465782 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:17.458272 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:17.553025 16210 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.556066 16210 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:17.557986 13692 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
W20260709 12:57:17.560107 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 102023us
I20260709 12:57:17.560509 13539 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:17.554448 13612 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:17.469959 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.562215 13688 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:17.360275 15470 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:17.432192 16171 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:17.547842 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:17.492311 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.530400 16181 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:17.568992 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:17.599797 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 224557us
W20260709 12:57:17.600358 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 46570.6us
I20260709 12:57:17.601665 13535 raft_consensus.cc:2410] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:17.601907 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:17.602229 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.602764 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:17.600449 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.603525 13537 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:17.605087 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.606226 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.607309 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:17.610939 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.611773 13615 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:57:17.612023 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 50168.6us
W20260709 12:57:17.553129 16194 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:57:17.613888 13584 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:17.615046 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:17.660408 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.615638 16181 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:17.616302 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.664886 13537 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
W20260709 12:57:17.666419 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:17.666988 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.576185 15926 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:17.576782 16145 raft_consensus.cc:3055] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:17.675945 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:17.672781 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.677740 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.677886 13535 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:17.678504 13537 raft_consensus.cc:2393] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:17.714179 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:17.659415 16016 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:17.715042 13687 raft_consensus.cc:2376] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4. Re-sending same reply.
W20260709 12:57:17.717799 13687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1254) took 1367 ms (client timeout 1756 ms). Trace:
I20260709 12:57:17.718645 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:17.677474 16182 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
W20260709 12:57:17.718400 13687 rpcz_store.cc:269] 0709 12:57:16.350055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.679469 (+1329414us) service_pool.cc:224] Handling call
0709 12:57:17.717733 (+ 38264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:17.680384 15761 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.865585 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:17.912987 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 122007us
W20260709 12:57:17.912274 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 298547us
I20260709 12:57:17.675822 16145 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:17.671486 16169 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:17.719007 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.953734 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:17.951234 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 343996us
W20260709 12:57:17.938776 16084 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.836s	user 0.006s	sys 0.005s
W20260709 12:57:17.793099 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.912397 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:17.913564 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.806672 16220 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:17.941936 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 381088us
I20260709 12:57:17.869114 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.912516 13612 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:17.865875 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:17.868122 13613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1407) took 1953 ms (client timeout 2159 ms). Trace:
I20260709 12:57:17.867295 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:17.953744 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.758407 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:17.962626 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 97129.2us
W20260709 12:57:17.883828 13659 leader_election.cc:336] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.576s (SENT)
I20260709 12:57:17.849023 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:17.963495 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:17.992799 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:17.992792 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.994004 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:17.988196 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:17.967907 16169 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:17.972342 16173 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:17.998411 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:17.999221 14242 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:17.999104 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:17.964826 14246 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 805) took 1600 ms (client timeout 1802 ms). Trace:
I20260709 12:57:18.000530 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:18.000754 14246 rpcz_store.cc:269] 0709 12:57:16.364493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.366179 (+  1686us) service_pool.cc:224] Handling call
0709 12:57:17.953579 (+1587400us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:17.964753 (+ 11174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8973568}
I20260709 12:57:17.713726 15928 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:17.972502 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:17.953384 15947 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.036s	user 0.015s	sys 0.000s
I20260709 12:57:18.008422 14240 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:18.008893 13691 raft_consensus.cc:2410] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:57:17.864104 16207 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:57:17.723886 16168 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.723634 16213 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:17.849345 16157 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:17.677145 16203 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:17.960831 15865 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:17.787747 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:17.817324 16121 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:17.984681 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:17.982919 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 99161.6us
W20260709 12:57:17.980644 13613 rpcz_store.cc:269] 0709 12:57:15.914451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.025019 (+110568us) service_pool.cc:224] Handling call
0709 12:57:17.867879 (+1842860us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540013a1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:17.868022 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238932224}
I20260709 12:57:17.910914 16210 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:17.723289 16215 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:17.956121 15926 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:18.009198 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 808) took 1642 ms. Trace:
I20260709 12:57:17.811985 16221 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:18.010001 16225 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:18.033679 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.019122 16213 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.020238 16220 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:57:17.808667 16205 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 12:57:17.965018 16199 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:17.924327 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:17.894152 15951 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:18.003067 13543 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.047001 16184 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.001173 13687 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:17.960757 15903 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:57:18.003933 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:18.013244 14243 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
W20260709 12:57:18.050758 14243 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 807) took 1685 ms (client timeout 1929 ms). Trace:
W20260709 12:57:18.050765 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46966.2us
W20260709 12:57:18.051187 14243 rpcz_store.cc:269] 0709 12:57:16.365122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.366429 (+  1307us) service_pool.cc:224] Handling call
0709 12:57:17.867132 (+1500703us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:18.050700 (+183568us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8145664}
I20260709 12:57:18.010406 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.979964 16218 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:18.052245 15903 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:18.052341 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.052841 15903 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:18.052561 16218 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:17.987470 16209 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:17.955142 16182 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.032341 14240 rpcz_store.cc:276] 0709 12:57:16.366877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.370871 (+  3994us) service_pool.cc:224] Handling call
0709 12:57:18.008029 (+1637158us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:18.008210 (+   181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.008919 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.009139 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24263040}
I20260709 12:57:18.053368 15903 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:18.054667 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:17.999095 16169 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:18.055644 16182 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:18.056586 13693 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:17.977789 15761 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:18.027070 13659 leader_election.cc:336] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.646s (SENT)
I20260709 12:57:18.028203 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:17.975857 16084 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.066607 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1398) took 2166 ms. Trace:
I20260709 12:57:18.067070 13619 rpcz_store.cc:276] 0709 12:57:15.899757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.223936 (+324179us) service_pool.cc:224] Handling call
0709 12:57:16.667131 (+443195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:16.698505 (+ 31374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.060563 (+1362058us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54002fb7d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:18.066534 (+  5971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018539776}
I20260709 12:57:17.983608 13688 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:57:18.010681 16173 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:17.986192 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:17.983884 13538 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:18.014956 16217 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.018652 16229 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.018184 16168 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.030632 15627 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:18.046522 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:17.848609 16171 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:18.044328 16219 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:57:18.051796 13584 leader_election.cc:400] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5.
I20260709 12:57:18.053095 16127 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.060679 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.061601 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.065577 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.022634 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.141877 13689 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
W20260709 12:57:18.147549 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 95904.7us
I20260709 12:57:18.148775 13693 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:18.107738 13535 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.137650 13613 raft_consensus.cc:2368] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8.
I20260709 12:57:18.157866 13535 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:18.165371 13535 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:18.106017 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2482) took 1023 ms. Trace:
I20260709 12:57:18.038909 16134 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:57:18.069535 14242 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:18.070945 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:18.070061 13619 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.170289 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:18.172508 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.173164 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.210152 13613 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.210738 14243 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:18.070676 15761 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:18.141553 16166 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.000s
I20260709 12:57:18.207144 13686 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.066211 16203 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:18.208582 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 181546us
W20260709 12:57:18.208348 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 166560us
I20260709 12:57:18.042546 16213 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.169754 13617 rpcz_store.cc:276] 0709 12:57:17.082931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.086101 (+  3170us) service_pool.cc:224] Handling call
0709 12:57:17.086479 (+   378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:17.086965 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:17.308230 (+221265us) log.cc:844] Serialized 14 byte log entry
0709 12:57:17.606119 (+297889us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:17.606987 (+   868us) log.cc:844] Serialized 52 byte log entry
0709 12:57:17.607250 (+   263us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:17.607870 (+   620us) log.cc:844] Serialized 14 byte log entry
0709 12:57:17.676138 (+ 68268us) log.cc:844] Serialized 14 byte log entry
0709 12:57:17.771440 (+ 95302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:17.771561 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:18.105770 (+334209us) raft_consensus.cc:1685] finished
0709 12:57:18.105791 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:18.105942 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66277,"prepare.queue_time_us":337041,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":133,"raft.queue_time_us":432292,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":336,"thread_start_us":522882,"threads_started":4,"wal-append.queue_time_us":236809}
W20260709 12:57:18.140683 16176 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.596s	user 0.005s	sys 0.005s
W20260709 12:57:18.206399 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 159120us
I20260709 12:57:18.211632 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.217146 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:18.048033 15951 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:18.212400 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 144910us
I20260709 12:57:18.036170 16145 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.035581 16215 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.068399 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:18.214123 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 61561.3us
I20260709 12:57:18.216183 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.139865 16218 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.142195 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.220500 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.221815 13535 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:18.221815 14243 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:18.215660 16230 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:18.206614 15453 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:18.244793 15453 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:18.245891 15453 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.246670 16231 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:18.247309 15453 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.247668 16231 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:18.248225 16231 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:18.248853 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.249047 16231 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.250028 16138 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:18.250579 16231 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:18.138392 15627 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.251415 15627 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.252456 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.224534 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 395) took 1103 ms. Trace:
I20260709 12:57:18.253296 13619 rpcz_store.cc:276] 0709 12:57:17.113792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.067962 (+954170us) service_pool.cc:224] Handling call
0709 12:57:18.069896 (+  1934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.214781 (+144885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.215534 (+   753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:18.250476 16184 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:57:18.255308 16218 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:18.255797 16220 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.257238 16181 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.258502 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.242165 14246 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:18.260926 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.261584 14246 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:18.262015 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32778 (request call id 27) took 1705 ms. Trace:
I20260709 12:57:18.262466 13542 rpcz_store.cc:276] 0709 12:57:16.556405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.557533 (+  1128us) service_pool.cc:224] Handling call
0709 12:57:18.261585 (+1704052us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54002a8620. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e72f520 00007fa02e7ac652 00007fa03377a60a 00007fa02cd007d4 00007fa02cd00762 00007fa02cd00691 00007fa02cd0060c 00007fa02ccff8c2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:18.261926 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477286272}
I20260709 12:57:18.264842 16145 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.264657 15951 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.265848 15951 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:18.210947 16182 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.268536 16182 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:18.211938 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 397) took 1082 ms. Trace:
I20260709 12:57:18.061069 16199 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.037984 16221 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.229180 13534 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:18.230809 13617 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:18.224933 16203 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.104337 15926 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:18.276829 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55026.9us
I20260709 12:57:18.277534 13693 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:18.233515 16084 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:18.221037 16229 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:18.236690 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:18.236377 16173 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.238154 16191 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:18.238955 15593 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:18.249162 16226 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.250442 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.274490 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.250612 16227 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:57:18.300025 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:18.301247 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:18.301970 16144 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:18.302153 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.219156 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.225343 16176 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.303869 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.305315 16203 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.305405 16176 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.305984 16181 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.306399 13688 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:18.306502 16181 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.306793 16226 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.308576 16120 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:18.212224 16207 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:57:18.274809 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.275179 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 399) took 1144 ms. Trace:
I20260709 12:57:18.313331 13617 rpcz_store.cc:276] 0709 12:57:17.130352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.230111 (+1099759us) service_pool.cc:224] Handling call
0709 12:57:18.230459 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.274910 (+ 44451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.275119 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:18.224392 16166 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.275250 13613 rpcz_store.cc:276] 0709 12:57:17.129645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.173840 (+1044195us) service_pool.cc:224] Handling call
0709 12:57:18.209962 (+ 36122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.211642 (+  1680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.211846 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:18.248862 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.286290 14243 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:18.097123 15946 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.177s
I20260709 12:57:18.053750 16209 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.290482 14209 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:18.294582 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.295267 13610 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:57:18.292363 15928 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.290495 16229 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.297501 16168 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.298045 16215 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.298120 16202 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:57:18.274696 16230 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.277020 15663 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.308861 16181 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.284291 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2473) took 1471 ms. Trace:
I20260709 12:57:18.337643 13538 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:18.336552 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.263763 16225 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:18.317303 14243 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:18.337944 13614 rpcz_store.cc:276] 0709 12:57:16.813019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.236428 (+1423409us) service_pool.cc:224] Handling call
0709 12:57:18.284205 (+ 47777us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:18.319383 13613 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:18.320446 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 45715.5us
I20260709 12:57:18.308929 16226 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:18.320989 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 100453us
I20260709 12:57:18.339756 13614 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:18.342892 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:18.343547 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.329327 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.292430 15593 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:18.332716 13617 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:57:18.315069 16084 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:18.290194 16191 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:18.312487 16120 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:18.340063 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:18.326717 16203 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.352406 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:18.350750 13659 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.959s (SENT)
I20260709 12:57:18.359760 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.359951 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.274914 16169 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.335132 15720 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:18.374051 13584 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:18.376310 13618 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:18.391038 13537 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.393796 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:18.350936 16084 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:18.374109 15959 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.816s	user 0.008s	sys 0.003s
I20260709 12:57:18.337498 16230 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:18.400643 14244 raft_consensus.cc:2368] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7.
I20260709 12:57:18.349016 15947 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.370188 16226 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:18.331058 15928 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.411372 15928 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.358642 16120 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.406656 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:18.417944 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.369836 15946 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:18.418917 15946 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.419528 15946 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:18.420583 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.399164 15846 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:18.426724 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 75925.7us
I20260709 12:57:18.379318 15720 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:18.349969 15663 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.436452 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:18.404189 16084 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.399499 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.394173 16168 consensus_queue.cc:1059] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:57:18.427253 16203 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:18.407757 16215 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:18.429204 16121 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.432338 16225 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:18.451588 13613 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:18.441102 15720 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.455140 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.440414 16226 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:18.453135 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 80977us
I20260709 12:57:18.392940 15903 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.454010 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.403267 16176 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:57:18.457238 13686 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:18.407052 16016 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:57:18.460362 16199 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.460834 16229 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.461519 16209 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:18.462417 16101 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.263s	user 0.008s	sys 0.003s
W20260709 12:57:18.470939 14238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 803) took 2107 ms (client timeout 2210 ms). Trace:
I20260709 12:57:18.473925 16144 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:18.474546 15946 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.467684 13619 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:18.489091 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:18.489909 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 141606us
I20260709 12:57:18.486291 15789 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.479086 16225 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:18.496054 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.506244 13617 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:18.492475 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.458860 16203 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:18.505282 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.441329 15593 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.506660 14243 raft_consensus.cc:2376] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5. Re-sending same reply.
I20260709 12:57:18.479102 15663 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.512684 14240 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:18.513216 14240 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:18.514212 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.514066 14240 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.514776 13617 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:18.508517 13508 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:18.515694 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:18.517632 13659 leader_election.cc:336] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.767s (SENT)
I20260709 12:57:18.517697 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.518199 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.518594 14243 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:18.520085 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.521555 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.522804 13686 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:18.523307 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:18.512785 16225 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.515219 16120 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.524039 13617 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:18.528645 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.502420 16101 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.534512 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.535130 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.535611 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.536553 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.534009 16017 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:57:18.524987 15663 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:18.524698 16231 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.541301 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.542872 13611 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.535707 16144 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.528160 16181 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:18.543051 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.544524 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.528856 16247 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:18.552876 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:18.538792 16199 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:18.548818 13617 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:57:18.536023 16229 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.555795 13615 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:18.547370 13686 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 4.
I20260709 12:57:18.539250 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.541867 16017 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:18.500669 14238 rpcz_store.cc:269] 0709 12:57:16.363755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.370494 (+  6739us) service_pool.cc:224] Handling call
0709 12:57:18.378391 (+2007897us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:18.470872 (+ 92481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7186176}
I20260709 12:57:18.557139 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:18.524814 16209 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:57:18.570046 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:18.570987 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.545581 16166 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.578800 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.556131 15663 raft_consensus.cc:697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:18.578868 16017 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:18.579767 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.582468 13686 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:18.559463 16215 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:57:18.583352 13659 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.161s (SENT)
I20260709 12:57:18.558182 16134 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.552521 16181 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.558451 16101 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:18.600051 16134 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.553085 16169 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:18.585716 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:18.584321 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.595469 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:18.595975 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.542312 16226 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.598110 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:18.599027 16202 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:18.599401 15946 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.608909 13619 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:18.608814 13686 raft_consensus.cc:2376] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7. Re-sending same reply.
I20260709 12:57:18.612497 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:18.552312 16231 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.620905 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:18.612955 13658 leader_election.cc:336] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.532s (SENT)
I20260709 12:57:18.614408 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.614230 15663 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.592489 16247 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:18.619572 16202 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:18.621070 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:18.626552 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:18.583774 16203 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.628420 16148 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.606086 16181 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.630893 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.631511 13686 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:18.619822 16017 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:18.633446 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.634428 13619 raft_consensus.cc:2410] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:57:18.635223 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.636368 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:18.638629 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.641149 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.642805 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:18.644229 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.644784 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:18.645772 14245 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:57:18.583105 16229 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.659556 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:18.662846 13686 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: [5.7-6.8]
I20260709 12:57:18.622903 16247 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.624929 16144 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.660697 13542 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:18.670238 13584 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:18.638156 16171 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:57:18.672483 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:18.674167 13659 leader_election.cc:336] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.202s (SENT)
I20260709 12:57:18.674878 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.674654 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:18.677270 13686 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:18.679785 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:18.624444 16202 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.682358 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:18.680943 13686 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.669687 16247 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:18.684947 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.674640 16171 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.673975 16225 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:18.676986 16145 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:18.678087 16120 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.698093 13619 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:57:18.679081 15968 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:57:18.691493 16171 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:18.680626 15627 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.691509 16225 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:18.738708 16225 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:18.694342 16202 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.734594 16229 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.735581 16182 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:18.756714 13686 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:18.775218 13688 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:18.775906 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:18.776270 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:18.705360 16213 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.777546 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:18.759975 15453 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.512s	user 0.014s	sys 0.000s
I20260709 12:57:18.780481 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:18.784129 13612 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 3. Current term is 5. Ops: []
I20260709 12:57:18.787812 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:18.788743 13535 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:57:18.789665 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2485) took 1705 ms. Trace:
I20260709 12:57:18.790040 13612 rpcz_store.cc:276] 0709 12:57:17.083875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.055106 (+971231us) service_pool.cc:224] Handling call
0709 12:57:18.055386 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.784665 (+729279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.789434 (+  4769us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b540026fa54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:18.789610 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1675496832}
I20260709 12:57:18.793053 16230 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.799377 13542 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:18.778841 15859 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.778595 16229 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:18.784880 16136 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:18.782120 16182 raft_consensus.cc:2764] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:57:18.817211 16231 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.811062 16136 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:18.793835 16144 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:18.813100 16221 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.819314 13612 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:18.820338 16144 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:18.825976 16144 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:18.828291 14244 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 804) took 2464 ms (client timeout 1722 ms). Trace:
W20260709 12:57:18.828789 14244 rpcz_store.cc:269] 0709 12:57:16.364126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.365897 (+  1771us) service_pool.cc:224] Handling call
0709 12:57:17.865702 (+1499805us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:18.828096 (+962394us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400601654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:18.828226 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974466176}
I20260709 12:57:18.832424 16136 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.833895 15957 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.834115 16136 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:18.838735 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.833141 16213 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:18.839401 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.845902 16230 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:18.846213 15593 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:18.847430 16181 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.848498 15976 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:18.849140 15976 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.850620 13541 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:18.846329 16084 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:18.865289 13659 leader_election.cc:336] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.310s (SENT)
I20260709 12:57:18.865985 13659 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.869982 16084 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.872078 16180 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:18.872759 16180 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:18.878757 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 392) took 2834 ms. Trace:
I20260709 12:57:18.881326 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1257) took 1784 ms. Trace:
I20260709 12:57:18.881673 13689 rpcz_store.cc:276] 0709 12:57:17.096963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.097107 (+   144us) service_pool.cc:224] Handling call
0709 12:57:17.129301 (+ 32194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.170777 (+1041476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.881127 (+710350us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400101b54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:18.881280 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328172288}
W20260709 12:57:18.884045 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:18.888823 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1273) took 1282 ms. Trace:
I20260709 12:57:18.889379 13684 rpcz_store.cc:276] 0709 12:57:17.606664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.607228 (+   564us) service_pool.cc:224] Handling call
0709 12:57:17.607612 (+   384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:17.679127 (+ 71515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:17.679164 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:17.723103 (+ 43939us) log.cc:844] Serialized 52 byte log entry
0709 12:57:18.640456 (+917353us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b1800427a80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:18.677091 (+ 36635us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:18.874197 (+197106us) log.cc:844] Serialized 14 byte log entry
0709 12:57:18.874487 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.874585 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:18.874642 (+    57us) raft_consensus.cc:1685] finished
0709 12:57:18.887988 (+ 13346us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:18.888755 (+   767us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58191,"raft.queue_time_us":70224,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":41574,"spinlock_wait_cycles":8251136,"thread_start_us":214876,"threads_started":2,"wal-append.queue_time_us":1001004}
I20260709 12:57:18.892791 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:18.894977 13689 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.879171 13616 rpcz_store.cc:276] 0709 12:57:16.044570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:16.336736 (+292166us) service_pool.cc:224] Handling call
0709 12:57:16.340809 (+  4073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:16.371391 (+ 30582us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:16.394104 (+ 22713us) log.cc:844] Serialized 14 byte log entry
0709 12:57:16.643039 (+248935us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:16.643117 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:16.643185 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:18.878506 (+2235321us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b540013c9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:18.878555 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:18.878697 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30121,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":10175,"raft.queue_time_us":1679,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":7660,"spinlock_wait_cycles":69805952,"thread_start_us":248141,"threads_started":1,"wal-append.queue_time_us":387083,"wal-append.run_cpu_time_us":2240,"wal-append.run_wall_time_us":1144052}
I20260709 12:57:18.869252 15720 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.902704 15720 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:18.904646 16259 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:57:18.864289 16181 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:18.873023 16247 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:18.907745 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:18.916301 16190 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.634s	user 0.004s	sys 0.000s
I20260709 12:57:18.876269 16182 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.925911 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:18.926666 13689 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:18.929244 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:18.930054 13684 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:18.931716 13684 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:18.932874 16222 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.039s	user 0.000s	sys 0.002s
I20260709 12:57:18.933861 13684 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:18.935451 13684 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:18.940747 16247 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:18.941926 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 1.56328e+06us
I20260709 12:57:18.943186 13509 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.945816 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.948804 13509 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.951967 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.954566 13684 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:57:18.954667 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:18.955835 13535 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:57:18.903762 16225 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:18.964954 13659 leader_election.cc:336] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.342s (SENT)
I20260709 12:57:18.965595 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.949983 16016 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.968516 16158 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:18.971232 13619 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:18.954442 16180 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.954250 16181 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.971058 16119 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:18.972162 13509 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.957999 16230 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:18.964543 16135 raft_consensus.cc:3055] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:18.980644 13535 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:18.981871 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.983444 16135 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:18.985896 16135 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:18.989784 13509 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:18.989306 16230 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:18.991674 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:18.992309 15663 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:18.994154 15663 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:18.994724 15663 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:18.995944 15663 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:18.995368 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:18.999543 13509 leader_election.cc:400] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8.
I20260709 12:57:18.999881 13509 leader_election.cc:403] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:19.001430 16158 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:19.001919 16158 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:19.002851 16134 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:18.983824 16180 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:19.002935 16158 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.009403 16230 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8.
W20260709 12:57:19.023459 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:19.025673 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1208) took 1918 ms. Trace:
I20260709 12:57:19.026165 13536 rpcz_store.cc:276] 0709 12:57:17.107494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.116480 (+  8986us) service_pool.cc:224] Handling call
0709 12:57:17.142986 (+ 26506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:17.377987 (+235001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:19.025312 (+1647325us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540006ead4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:19.025585 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522068736}
I20260709 12:57:19.015774 16225 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:19.031494 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 403) took 1062 ms. Trace:
I20260709 12:57:19.031876 13618 rpcz_store.cc:276] 0709 12:57:17.969343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.255745 (+286402us) service_pool.cc:224] Handling call
0709 12:57:18.270688 (+ 14943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.392857 (+122169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:19.031253 (+638396us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54000762d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:19.031423 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242459008}
I20260709 12:57:19.033320 13692 raft_consensus.cc:2376] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4. Re-sending same reply.
I20260709 12:57:19.027694 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:19.035908 13536 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 5: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:19.037393 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:19.038138 13616 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:19.038697 13687 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:57:19.039165 13687 pending_rounds.cc:85] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 2
I20260709 12:57:19.039453 13687 pending_rounds.cc:107] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:57:19.010346 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:19.039727 13687 raft_consensus.cc:2889] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:19.010916 16135 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:19.012075 13610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 400) took 1873 ms (client timeout 2177 ms). Trace:
W20260709 12:57:19.040655 13610 rpcz_store.cc:269] 0709 12:57:17.138978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.995376 (+856398us) service_pool.cc:224] Handling call
0709 12:57:19.011876 (+1016500us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54002b3c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:19.012020 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558402944}
I20260709 12:57:19.012387 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:19.013154 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:19.044977 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:19.013440 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:19.047338 14209 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:19.048010 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:18.983525 16181 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:19.040112 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:19.050231 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:19.050426 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:19.050925 13612 raft_consensus.cc:2376] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6. Re-sending same reply.
I20260709 12:57:19.051121 14244 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:19.054386 14238 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:19.054674 16181 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:19.056363 16247 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.057042 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:19.057725 13535 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:19.059116 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:19.061707 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:19.062407 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:19.062638 13616 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:19.057900 13612 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:19.064539 16213 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:19.031173 16144 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.056504 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:19.066627 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 62310.3us
I20260709 12:57:19.068819 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:19.064561 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:19.072633 13616 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:19.074365 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:19.076483 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:19.057243 16247 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:18.983070 16171 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:19.088521 13684 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:19.077245 13610 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:19.080860 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:19.081826 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:19.083369 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:19.083837 13536 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:19.085003 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:19.113180 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:19.078437 16134 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:19.078893 16221 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:19.078176 16229 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:19.081707 15994 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:19.142731 16215 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:57:19.217170 13659 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.292s (SENT)
I20260709 12:57:19.147238 15720 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:19.151289 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1276) took 1011 ms. Trace:
I20260709 12:57:19.225178 13693 rpcz_store.cc:276] 0709 12:57:18.140400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.208431 (+ 68031us) service_pool.cc:224] Handling call
0709 12:57:19.151047 (+942616us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b54001eebd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:19.151217 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72170752}
I20260709 12:57:19.154088 13686 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:19.161327 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 478) took 1132 ms. Trace:
I20260709 12:57:19.226943 13538 rpcz_store.cc:276] 0709 12:57:18.029369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.143858 (+114489us) service_pool.cc:224] Handling call
0709 12:57:18.144224 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.346202 (+201978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:19.161071 (+814869us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540028bc54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:19.161261 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444366720}
I20260709 12:57:19.089375 16213 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:19.164047 16134 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:19.092272 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:19.164692 16229 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:19.166047 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:19.166297 16171 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:19.168540 15976 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:19.172914 16225 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.175623 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:19.178222 15833 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:19.244328 15833 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:19.240610 15663 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:19.245152 16266 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:57:19.245707 15833 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:19.237762 16134 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:57:19.179502 13541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1294) took 1648 ms (client timeout 1532 ms). Trace:
I20260709 12:57:19.246699 15663 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:19.246932 13541 rpcz_store.cc:269] 0709 12:57:17.531129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.531289 (+   160us) service_pool.cc:224] Handling call
0709 12:57:19.179170 (+1647881us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540028c654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:19.179406 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3002905984}
I20260709 12:57:19.248507 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:19.249374 13541 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:19.179992 15593 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:57:19.181149 16212 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:19.181576 16184 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:19.182163 16216 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:19.184618 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:19.265863 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 81309.3us
I20260709 12:57:19.261714 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:19.186736 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:19.266960 13658 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:19.196341 15946 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:19.198312 16144 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:19.200156 13684 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:57:19.144121 15972 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:57:19.118295 16197 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:57:19.119123 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:19.272781 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:19.146097 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:19.118901 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:19.275162 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 156312us
I20260709 12:57:19.276279 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:19.220434 13536 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:57:19.229915 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 442) took 1286 ms. Trace:
I20260709 12:57:19.228173 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:19.118712 16183 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:19.229053 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:19.238021 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:19.240597 13618 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:19.225215 16119 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.119437 16235 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 12:57:19.266789 14211 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:19.185546 16136 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:19.234457 16213 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:19.238569 16229 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:19.268424 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:19.267628 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:19.267400 13693 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:19.285391 13541 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:19.278414 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:19.246762 15833 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:19.290004 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:19.290140 13658 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:19.290994 13538 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 3.
I20260709 12:57:19.295351 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:19.279040 14246 rpcz_store.cc:276] 0709 12:57:17.943874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.262249 (+318375us) service_pool.cc:224] Handling call
0709 12:57:18.262593 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:18.286176 (+ 23583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:18.286222 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:18.286729 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:57:18.998876 (+712147us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b1800392860. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:19.097265 (+ 98389us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:19.163299 (+ 66034us) log.cc:844] Serialized 14 byte log entry
0709 12:57:19.163779 (+   480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:19.163856 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:19.229534 (+ 65678us) raft_consensus.cc:1685] finished
0709 12:57:19.229644 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:19.229812 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30349,"prepare.queue_time_us":49175,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":24734,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":4036,"spinlock_wait_cycles":3523456,"thread_start_us":97954,"threads_started":1,"wal-append.queue_time_us":907926}
W20260709 12:57:19.313030 13509 leader_election.cc:336] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.963s (SENT)
I20260709 12:57:19.313606 13509 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:19.253383 16221 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:19.270089 16171 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:19.284626 14237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 811) took 1562 ms (client timeout 1770 ms). Trace:
I20260709 12:57:19.317190 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:19.313660 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:19.253129 16225 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:19.283098 16183 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:19.285874 16215 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:19.386283 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:19.441749 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:19.530125 13693 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:19.529695 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 216644us
I20260709 12:57:19.289031 16213 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:19.311440 16144 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:19.307312 16155 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.314868 16187 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:19.310520 16185 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:19.315181 16016 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.310560 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:19.317900 15994 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:19.382202 16199 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:19.315543 16140 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.310850 16197 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:19.318912 16166 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:19.516065 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:19.580557 13618 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:19.384166 16184 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:19.295507 16119 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.546463 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:19.613525 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:19.614217 13684 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:19.376678 15663 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.144s
W20260709 12:57:19.517947 15866 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:57:19.536423 16202 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.786s	user 0.011s	sys 0.004s
I20260709 12:57:19.534539 16230 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.550127 16155 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.534998 15593 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:19.566629 16247 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:19.567806 15994 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.570439 16209 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:19.688174 16209 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:19.689183 16209 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.691208 16209 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:19.520025 16171 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:19.692248 16171 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:19.692992 16135 consensus_meta.cc:287] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.002s
I20260709 12:57:19.501051 13541 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:19.694487 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:19.695533 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:19.537668 16183 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:19.501614 15720 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:57:19.537382 16215 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:19.580797 16166 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:19.582471 16185 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:19.587446 16228 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:19.588378 15946 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:19.589314 16180 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.004s
I20260709 12:57:19.549243 16179 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:19.582991 16140 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:19.529474 16216 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
W20260709 12:57:19.519168 14237 rpcz_store.cc:269] 0709 12:57:17.718608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.724826 (+  6218us) service_pool.cc:224] Handling call
0709 12:57:19.165759 (+1440933us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:19.280529 (+114770us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60837120}
I20260709 12:57:19.861656 15865 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:19.755578 16215 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:19.531955 16272 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:19.804399 16228 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:19.863590 15865 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:19.504359 13659 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.505s (SENT)
I20260709 12:57:19.883036 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:19.699108 16155 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:19.772025 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 489237us
W20260709 12:57:19.885772 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:19.617113 15957 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.573184 16016 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.890820 13538 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:19.838922 16247 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:19.879709 16272 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:19.592734 16144 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:19.841513 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.005s	sys 0.005s
W20260709 12:57:19.907382 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.005s	sys 0.005s
W20260709 12:57:19.584688 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 274196us
I20260709 12:57:19.598536 16221 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.783324 16166 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:19.872145 13684 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:57:19.783037 15972 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.884744 16271 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:19.678900 16134 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.735073 16230 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.896111 16202 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.891639 15720 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:57:19.901250 13693 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:19.933739 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:19.904464 15865 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:19.577561 16226 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.202s
I20260709 12:57:19.908687 13541 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:19.601666 16225 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:19.915328 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:19.924857 13692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1271) took 2302 ms (client timeout 2025 ms). Trace:
W20260709 12:57:19.902837 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.021s	sys 0.026s
W20260709 12:57:19.940148 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.021s	sys 0.027s
W20260709 12:57:19.940068 13692 rpcz_store.cc:269] 0709 12:57:17.604705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.604851 (+   146us) service_pool.cc:224] Handling call
0709 12:57:19.907353 (+2302502us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002977d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:19.907458 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3283152128}
W20260709 12:57:19.940531 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 648500us
I20260709 12:57:19.934619 13693 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:19.941639 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 795562us
I20260709 12:57:19.907816 16272 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:19.894096 15957 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:19.955075 13542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1223) took 1446 ms (client timeout 1733 ms). Trace:
W20260709 12:57:19.958099 13542 rpcz_store.cc:269] 0709 12:57:18.508973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.669321 (+160348us) service_pool.cc:224] Handling call
0709 12:57:19.954890 (+1285569us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400088ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:19.955015 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281600896}
I20260709 12:57:19.934291 16144 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:19.959197 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:19.970691 13542 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:19.906423 16247 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:19.973519 13684 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:19.973538 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 323) took 2896 ms. Trace:
I20260709 12:57:19.941756 16199 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.883107 16197 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:19.974663 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:19.955591 16235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.007s	sys 0.022s
I20260709 12:57:19.926429 15972 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:19.985548 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:19.959275 15833 consensus_meta.cc:287] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.001s
I20260709 12:57:19.949833 15946 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:19.980890 13692 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:57:19.931512 16271 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:19.983443 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:19.984454 13691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 325) took 2906 ms (client timeout 1882 ms). Trace:
I20260709 12:57:19.985447 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:19.978861 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 665217us
I20260709 12:57:19.974140 13690 rpcz_store.cc:276] 0709 12:57:17.077272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.077439 (+   167us) service_pool.cc:224] Handling call
0709 12:57:17.077865 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:17.078139 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:17.080307 (+  2168us) log.cc:844] Serialized 14 byte log entry
0709 12:57:19.160534 (+2080227us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b1800430840. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a 00007fa02e7bca2a
0709 12:57:19.972955 (+812421us) log.cc:844] Serialized 14 byte log entry
0709 12:57:19.973126 (+   171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:19.973172 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:19.973224 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:19.973358 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:19.973481 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10660,"prepare.queue_time_us":2176,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"spinlock_wait_cycles":13472128,"thread_start_us":800985,"threads_started":2,"wal-append.queue_time_us":2779004}
I20260709 12:57:19.981993 13693 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:19.996130 13691 rpcz_store.cc:269] 0709 12:57:17.078211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.101223 (+ 23012us) service_pool.cc:224] Handling call
0709 12:57:19.984251 (+2883028us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54000483d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:19.984383 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049468800}
I20260709 12:57:19.997452 13693 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:19.941596 16202 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:19.991824 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:19.984783 16235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.008s	sys 0.022s
I20260709 12:57:19.975551 16146 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:19.930763 16185 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.005990 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.024890 13542 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:20.014855 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 73751.8us
W20260709 12:57:20.002748 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:20.024025 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1427) took 1575 ms. Trace:
I20260709 12:57:20.026762 13619 rpcz_store.cc:276] 0709 12:57:18.448831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.731539 (+282708us) service_pool.cc:224] Handling call
0709 12:57:18.759631 (+ 28092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.972771 (+213140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:20.023747 (+1050976us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54001298d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:20.023962 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486218624}
I20260709 12:57:20.026669 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:20.003754 16119 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:20.033313 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 529009us
I20260709 12:57:20.035388 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:20.041891 15972 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.043238 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.043944 13618 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:20.044512 16016 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.046373 16271 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:19.982705 16199 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.048396 16140 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.049995 16199 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:19.976204 16228 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.050892 16199 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:20.019531 15470 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.052734 16230 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.053841 16282 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:20.054809 16134 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.055620 16134 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.056217 16134 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.059703 16202 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.029409 16146 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.043977 15865 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.041496 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.076792 13617 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:20.047209 16271 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:20.019250 16144 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:20.001026 15946 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.016044 16278 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:20.067451 13538 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:20.087743 15865 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.104068 16144 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:20.105448 16144 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.105759 15865 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:20.108263 15865 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:57:20.035322 16277 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:20.112214 16277 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.112591 16277 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:20.113122 16247 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:20.114111 16270 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:57:20.117192 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.117489 15957 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:20.118041 14243 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 519) took 1595 ms (client timeout 1629 ms). Trace:
W20260709 12:57:20.118407 14243 rpcz_store.cc:269] 0709 12:57:18.522350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.537789 (+ 15439us) service_pool.cc:224] Handling call
0709 12:57:20.117043 (+1579254us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:20.117980 (+   937us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3004288}
I20260709 12:57:20.124042 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:20.077404 16119 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:20.125375 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 86531us
I20260709 12:57:20.132128 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:20.124643 16271 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:20.128281 15789 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:20.143183 13542 raft_consensus.cc:2410] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:57:20.136677 16229 consensus_meta.cc:287] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.821s	user 0.007s	sys 0.006s
W20260709 12:57:20.139475 13584 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 1.629s (SENT) [suppressed 58 similar messages]
I20260709 12:57:20.136348 16134 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.140168 16286 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:20.150532 15833 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:20.192731 16285 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:57:20.192565 15833 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.192974 16218 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:20.193477 16218 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:20.194329 16218 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.195441 16218 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.139475 16271 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:20.196442 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.197045 13542 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:20.196966 16271 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.197574 16212 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:20.198046 16271 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.198122 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.199354 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.200052 15866 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:57:20.200003 16229 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.200757 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:20.201211 15866 tablet_bootstrap.cc:492] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:57:20.202025 15866 ts_tablet_manager.cc:1403] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 2.738s	user 0.045s	sys 0.035s
I20260709 12:57:20.202898 13540 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:20.123294 16171 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:20.140451 16288 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.224531 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.222887 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 404) took 2246 ms. Trace:
I20260709 12:57:20.223421 15946 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.221558 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:20.225627 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 86136.7us
I20260709 12:57:20.226013 13611 rpcz_store.cc:276] 0709 12:57:17.976402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.269313 (+292911us) service_pool.cc:224] Handling call
0709 12:57:18.269668 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.547356 (+277688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:20.222614 (+1675258us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b540020ec54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:20.222824 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627891456}
I20260709 12:57:20.225143 13684 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:20.202299 16229 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.129624 16119 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.195721 15833 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.198007 16212 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:20.231463 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:20.232632 16247 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:20.233047 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:20.216413 15972 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.237659 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.249997 13542 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:20.216296 16148 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:20.220917 16213 consensus_meta.cc:287] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.668s	user 0.004s	sys 0.007s
I20260709 12:57:20.216805 16289 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.253325 16289 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.255893 16119 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:20.256969 13614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1409) took 1856 ms (client timeout 1576 ms). Trace:
W20260709 12:57:20.257427 13614 rpcz_store.cc:269] 0709 12:57:18.400072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:20.256729 (+1856657us) service_pool.cc:211] Skipping call since client already timed out
0709 12:57:20.256859 (+   130us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:57:20.219704 16275 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:20.258319 16275 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.258703 16275 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:20.264104 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.264514 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2661) took 2199 ms. Trace:
I20260709 12:57:20.264657 16148 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.264876 13687 rpcz_store.cc:276] 0709 12:57:18.065518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.067684 (+  2166us) service_pool.cc:224] Handling call
0709 12:57:18.100445 (+ 32761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:19.040337 (+939892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:20.264285 (+1223948us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b540002c454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:20.264444 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152428416}
I20260709 12:57:20.265291 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.265972 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:20.266319 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:20.266491 13687 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.266855 13684 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:20.230846 16288 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.268764 16247 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:20.268488 16212 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.269594 16212 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:20.271080 13543 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:20.258627 13534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1210) took 3147 ms (client timeout 1687 ms). Trace:
I20260709 12:57:20.271489 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:20.252707 15957 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:20.274600 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 479) took 2236 ms. Trace:
I20260709 12:57:20.243423 16166 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.275893 13537 rpcz_store.cc:276] 0709 12:57:18.038525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.145109 (+106584us) service_pool.cc:224] Handling call
0709 12:57:18.172385 (+ 27276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.391655 (+219270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:20.274390 (+1882735us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54000db954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:20.274547 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502307072}
I20260709 12:57:20.280992 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.284093 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.284382 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.284891 13614 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:20.288156 13684 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:20.289496 14244 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:20.249148 16226 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:20.294970 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.295223 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:20.295945 14245 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.296190 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.296730 14238 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:20.297322 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.297935 13618 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.298502 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.297673 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.300545 14237 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:20.301326 16168 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:20.290791 15865 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.301934 16168 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:20.302452 15865 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.302898 16168 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:20.276000 16184 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.639s	user 0.011s	sys 0.000s
I20260709 12:57:20.295764 13614 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:20.299841 16209 consensus_meta.cc:287] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.001s
I20260709 12:57:20.275513 15972 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.259546 16272 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.295168 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.310901 13618 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:20.329504 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.341917 13537 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:20.246063 15789 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:57:20.331549 13687 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:20.306602 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:20.333364 13582 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:20.343089 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.332875 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:20.343727 13687 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:20.344281 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:20.336236 14245 raft_consensus.cc:2410] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:20.305429 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.345377 13614 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:20.336992 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.347189 13542 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:20.341202 13619 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:20.328603 16184 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:20.350339 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.351027 14244 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:20.351581 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:20.269768 16247 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.357720 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:20.359014 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:20.274197 13534 rpcz_store.cc:269] 0709 12:57:17.111614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.116106 (+  4492us) service_pool.cc:224] Handling call
0709 12:57:20.258435 (+3142329us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54002841d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:20.258558 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507282176}
I20260709 12:57:20.299798 16275 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.360307 15859 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.360495 16158 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:20.273746 16215 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.343127 16277 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.264134 16213 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.372189 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.369517 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:20.368777 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:20.385049 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:20.329850 15951 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.377729 13611 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:20.380492 13614 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:20.368880 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.360951 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.389724 13618 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
W20260709 12:57:20.389910 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:20.369786 14237 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:20.368852 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.391115 13687 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:20.392789 14245 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:20.397162 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:20.396459 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 124076us
I20260709 12:57:20.398958 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:20.400148 15833 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:20.402958 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:20.376468 16158 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:20.283572 16289 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.422737 16289 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:20.423128 16158 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.424325 16289 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:20.424757 16289 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.425122 16289 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.432796 13612 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:20.437074 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.437868 13687 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:20.411365 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:20.368469 15470 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:20.442399 13693 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:20.421209 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.448143 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.449100 13542 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
W20260709 12:57:20.401523 16183 consensus_meta.cc:287] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.616s	user 0.006s	sys 0.005s
I20260709 12:57:20.361932 16148 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.376134 16281 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:20.456244 16144 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.457549 15470 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:20.477406 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.478235 13542 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:20.458563 16277 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.460407 16272 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.460649 15865 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.385721 16119 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:20.464089 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.450340 16275 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.360791 16296 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:20.397423 16295 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.377899 16288 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.420719 15951 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.381567 16228 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:57:20.463110 14241 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:20.490715 13687 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:20.511248 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:20.402181 16142 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:20.461463 16302 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.372427 15859 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.375900 15866 raft_consensus.cc:359] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:20.516170 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.474308 15957 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:20.494462 16296 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:20.497509 13613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1416) took 2095 ms (client timeout 1572 ms). Trace:
I20260709 12:57:20.488098 16281 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:20.512578 13687 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:20.468665 16183 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:20.495841 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:20.516782 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 96180.5us
I20260709 12:57:20.517584 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1324) took 1303 ms. Trace:
W20260709 12:57:20.518208 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 115318us
W20260709 12:57:20.519305 13613 rpcz_store.cc:269] 0709 12:57:18.404414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.417584 (+ 13170us) service_pool.cc:224] Handling call
0709 12:57:20.497268 (+2079684us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b540013c4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:20.497414 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75529600}
I20260709 12:57:20.523542 13584 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:20.487469 16144 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.500844 16171 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.521872 13542 rpcz_store.cc:276] 0709 12:57:19.213946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:20.500005 (+1286059us) service_pool.cc:224] Handling call
0709 12:57:20.517526 (+ 17521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:20.522395 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:20.513456 16142 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.512427 16288 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.512040 15951 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.511816 16215 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.530411 13687 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:20.530841 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.533047 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.562955 13613 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:20.508102 16295 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.533938 13582 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:20.529175 16306 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.540140 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.553567 16303 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:20.521075 16296 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:20.555073 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 322074us
I20260709 12:57:20.560057 16166 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.531466 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:20.563359 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.518296 15866 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:57:20.554107 16138 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.578066 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.578233 13613 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:20.534698 15865 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:20.570272 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:20.517843 16302 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.573356 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:20.568472 16309 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:20.567472 16303 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.534926 16142 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.583161 13687 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:20.580351 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:20.584914 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:20.554486 15789 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:57:20.612301 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.531150 16226 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.580789 16272 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.602263 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:20.571202 16277 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:57:20.622327 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.582783 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.581672 15859 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.581836 16215 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:20.582124 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.562480 15470 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.599098 16305 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.627244 13542 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:20.584339 16171 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.581759 16308 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:57:20.614589 16168 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:20.629997 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.660759 13617 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:20.608913 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.601694 16213 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.563817 16288 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:20.640955 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:20.578433 15972 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.570694 16306 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.630223 16215 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.618932 16303 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:20.586151 16138 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.663105 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:20.682699 13687 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.624528 16226 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.625885 16272 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.664609 16277 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:20.655397 14241 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:20.616128 15957 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.694609 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2668) took 2593 ms. Trace:
I20260709 12:57:20.674412 16311 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:20.631480 16286 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:57:20.678331 16213 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.680485 16215 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.695117 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.702524 13688 rpcz_store.cc:276] 0709 12:57:18.101244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.472185 (+370941us) service_pool.cc:224] Handling call
0709 12:57:18.472481 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.775773 (+303292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:20.694371 (+1918598us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540015ead4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:20.694553 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695738624}
I20260709 12:57:20.684257 15972 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:20.622447 15866 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:20.706020 16288 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.710126 13688 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:20.711158 16138 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.711375 13688 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:20.713250 15972 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:20.714156 15972 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.716341 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.716950 13618 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
W20260709 12:57:20.718828 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 133998us
I20260709 12:57:20.720216 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.721122 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:20.721899 14244 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 5: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:20.717437 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:20.723505 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:20.726202 13687 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:20.674000 16305 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:20.721697 16271 consensus_meta.cc:287] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.002s
I20260709 12:57:20.704289 16311 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:20.729559 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 127397us
I20260709 12:57:20.624363 16281 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.759258 13619 raft_consensus.cc:2410] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:57:20.698697 16183 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.730237 16144 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.704586 15859 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:20.721809 16307 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.166s	user 0.000s	sys 0.003s
I20260709 12:57:20.731348 15663 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:20.630878 16289 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.731271 16166 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:20.726342 13542 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:20.765493 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.750344 16309 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:20.760051 16218 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.000s
I20260709 12:57:20.722019 16213 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.765036 16295 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.774312 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.773762 13617 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:20.758919 16311 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.787642 16309 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.789764 15866 ts_tablet_manager.cc:1434] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.587s	user 0.014s	sys 0.001s
I20260709 12:57:20.800453 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.804457 13542 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:20.777264 16166 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.805878 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.806684 16306 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.806938 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.811383 13618 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:20.812610 13542 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.785589 16183 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.779383 15663 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:20.798343 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:20.797910 16302 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.779011 16308 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:57:20.804020 16295 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.829881 16135 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:57:20.804116 15866 ts_tablet_manager.cc:1467] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Tablet startup took 601ms
I20260709 12:57:20.833091 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:20.833761 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 73354.2us
I20260709 12:57:20.771152 15859 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.827903 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.841827 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:20.842510 13617 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.810730 14240 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:20.811937 16166 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.831370 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.856052 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.856722 16305 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.856940 13534 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:57:20.857828 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.812044 15903 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:20.858987 16316 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:57:20.841818 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:20.847945 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:20.848951 15833 consensus_queue.cc:1059] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:20.846997 16172 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:20.846829 16158 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:20.840544 16135 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:20.850044 15859 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.854702 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.854514 16296 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.877270 13542 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:20.841461 14238 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:20.833427 16183 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.893851 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.860387 16148 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:57:20.886437 16272 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.874953 16172 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.859037 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.878450 16158 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.843752 16295 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.902068 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:20.887773 16144 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:20.908768 13537 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:20.874584 16157 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:20.909933 16191 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:20.911899 13615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2494) took 3825 ms (client timeout 1963 ms). Trace:
W20260709 12:57:20.930830 13615 rpcz_store.cc:269] 0709 12:57:17.086651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.661517 (+574866us) service_pool.cc:224] Handling call
0709 12:57:20.911697 (+3250180us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b540012b1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:20.911821 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1683367296}
I20260709 12:57:20.913182 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.932106 13611 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:20.933197 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.933919 13611 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:20.934937 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.913997 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.936210 13610 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:20.860509 16247 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:20.902323 15957 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.900979 16308 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:20.919673 16172 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:20.922317 16226 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:20.952186 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.927634 13542 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:20.940209 13611 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:57:20.929309 16144 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:20.956758 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:20.957571 13611 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.956033 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:20.959533 13537 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:20.899454 15663 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:20.939590 16247 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:20.840809 15866 ts_tablet_manager.cc:1469] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Trace:
0709 12:57:07.540865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:07.619804 (+ 78939us) service_pool.cc:224] Handling call
0709 12:57:08.005146 (+385342us) ts_tablet_manager.cc:934] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
0709 12:57:10.833111 (+2827965us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:57:12.577523 (+1744412us) tablet_metadata.cc:665] Metadata flushed
0709 12:57:12.585140 (+  7617us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:57:13.268105 (+682965us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:57:13.328392 (+ 60287us) tablet_replica.cc:206] Creating consensus instance
0709 12:57:13.467854 (+139462us) ts_tablet_manager.cc:989] Replica registered.
0709 12:57:13.468056 (+   202us) inbound_call.cc:177] Queueing success response
0709 12:57:17.464169 (+3996113us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:57:20.202347 (+2738178us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:57:20.373296 (+170949us) tablet_replica.cc:261] Starting instrumentation
0709 12:57:20.373889 (+   593us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":41148,"raft.queue_time_us":13712,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":20321,"tablet-copy.queue_time_us":205875,"tablet-download-pool.queue_time_us":22881,"tablet-download-pool.run_cpu_time_us":2426,"tablet-download-pool.run_wall_time_us":2801956,"thread_start_us":214395,"threads_started":2}
I20260709 12:57:20.926826 16158 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:20.950806 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.951026 16289 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:20.970331 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:20.955673 16191 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:20.967725 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.981798 13610 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:20.982944 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:20.983245 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:20.929702 16142 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.965319 16144 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:20.996837 16296 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:20.951365 16308 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:20.971362 13615 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:20.970863 15833 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:21.000808 15833 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:21.001230 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.001859 13615 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:21.002228 16295 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.002691 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:21.003659 15972 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:21.004185 15972 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:21.004566 15972 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:21.005934 16142 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.001940 15833 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:21.008349 16321 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:57:21.023559 16236 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:21.024142 16236 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.026208 16236 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.026774 16323 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:21.009061 16247 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.027310 16323 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.032223 16247 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:21.032518 13684 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:57:21.032881 16311 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.033739 13684 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:21.033969 13687 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:21.035023 16311 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:21.036237 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:21.037313 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:21.038300 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:21.038816 13536 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:21.039522 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:21.028399 14238 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:21.041290 16213 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.042260 16183 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.042609 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 329) took 2571 ms. Trace:
W20260709 12:57:21.043196 16269 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.535s	user 0.009s	sys 0.004s
I20260709 12:57:21.044708 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.044469 16303 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.047852 16303 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.048220 15627 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.042989 13686 rpcz_store.cc:276] 0709 12:57:18.475534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.757683 (+282149us) service_pool.cc:224] Handling call
0709 12:57:19.153767 (+396084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:19.226614 (+ 72847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.042420 (+1815806us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54001eebd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:21.042549 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909952256}
I20260709 12:57:21.045899 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.054683 13612 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:21.056022 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.057102 13612 raft_consensus.cc:2410] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:57:21.058125 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.059077 13612 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:21.060701 16273 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:21.061373 16273 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:21.045027 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1225) took 2534 ms. Trace:
I20260709 12:57:21.067886 13539 rpcz_store.cc:276] 0709 12:57:18.510914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:19.062565 (+551651us) service_pool.cc:224] Handling call
0709 12:57:19.062907 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:19.063261 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:19.063290 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:19.066455 (+  3165us) log.cc:844] Serialized 52 byte log entry
0709 12:57:19.235986 (+169531us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:19.244929 (+  8943us) log.cc:844] Serialized 14 byte log entry
0709 12:57:19.249971 (+  5042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:19.250047 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:20.967123 (+1717076us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:21.044753 (+ 77630us) raft_consensus.cc:1685] finished
0709 12:57:21.044839 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.044970 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4726,"prepare.queue_time_us":13790,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":7322,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":17896,"spinlock_wait_cycles":17133312,"thread_start_us":79373,"threads_started":1,"wal-append.queue_time_us":166834}
I20260709 12:57:21.069088 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:21.072480 13536 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:21.073733 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:21.074316 13684 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:21.075033 16229 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:21.075793 16229 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:21.076205 16323 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.076709 16229 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.077548 13584 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:21.078585 16289 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:21.079068 16289 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:21.003718 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.079958 16289 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:21.080781 13538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1322) took 1867 ms (client timeout 2083 ms). Trace:
I20260709 12:57:21.081429 13684 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:21.012208 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:21.082614 13542 raft_consensus.cc:2410] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:20.964231 16203 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:20.925571 16135 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:20.983547 13610 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:21.095851 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:21.096915 13539 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:21.097316 13610 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:57:20.940725 13617 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:21.100116 13611 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:21.101594 13542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:21.102347 13542 raft_consensus.cc:606] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:21.102779 13542 raft_consensus.cc:614] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:57:21.020766 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:21.000990 16308 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:20.991496 14240 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:57:20.948989 15957 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.075129 13691 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:21.010313 16144 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.080814 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.083806 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.121691 13615 raft_consensus.cc:2410] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:21.083554 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:21.085208 14246 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 4 LEARNER]: Advancing to term 5
I20260709 12:57:21.122139 13612 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
W20260709 12:57:21.081233 13538 rpcz_store.cc:269] 0709 12:57:19.213435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:19.317960 (+104525us) service_pool.cc:224] Handling call
0709 12:57:21.080618 (+1762658us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400002654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:21.080725 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828765056}
I20260709 12:57:21.091435 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:21.088492 13537 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
W20260709 12:57:21.104171 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 83510us
I20260709 12:57:21.107753 13539 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:21.123333 13538 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:21.123625 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1311) took 1912 ms. Trace:
I20260709 12:57:21.123910 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1315) took 1912 ms. Trace:
I20260709 12:57:21.109941 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32778 (request call id 28) took 1563 ms. Trace:
I20260709 12:57:21.124605 13542 rpcz_store.cc:276] 0709 12:57:19.540764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.095466 (+1554702us) service_pool.cc:224] Handling call
0709 12:57:21.103197 (+  7731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.124397 13538 rpcz_store.cc:276] 0709 12:57:19.211821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.122893 (+1911072us) service_pool.cc:224] Handling call
0709 12:57:21.123179 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.123685 (+   506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.123858 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.109748 14240 pending_rounds.cc:85] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 5
W20260709 12:57:21.146905 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55551.3us
W20260709 12:57:21.149408 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65936us
I20260709 12:57:21.106986 16171 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:21.155292 13615 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:57:21.154033 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.167276 13686 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:57:21.111598 16303 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:21.164055 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1318) took 1951 ms. Trace:
I20260709 12:57:21.168853 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.171551 13538 rpcz_store.cc:276] 0709 12:57:19.212477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.127260 (+1914783us) service_pool.cc:224] Handling call
0709 12:57:21.163337 (+ 36077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.163567 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.163605 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:21.163660 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:21.163704 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.163789 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.163966 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.126804 16135 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:21.155598 14240 pending_rounds.cc:107] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:57:21.101720 16166 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.148617 16142 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:21.156162 16228 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:21.156006 16296 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:21.181182 16226 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:21.182045 16148 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:21.062651 16273 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:21.183364 13540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1303) took 3629 ms (client timeout 1512 ms). Trace:
I20260709 12:57:21.184751 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.154716 15968 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:57:21.178395 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 84690.1us
W20260709 12:57:21.197274 13540 rpcz_store.cc:269] 0709 12:57:17.553578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:17.576379 (+ 22801us) service_pool.cc:224] Handling call
0709 12:57:21.183174 (+3606795us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b54002832d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:21.183285 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5977280896}
W20260709 12:57:21.197783 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.124017 13539 rpcz_store.cc:276] 0709 12:57:19.210971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.107161 (+1896190us) service_pool.cc:224] Handling call
0709 12:57:21.107530 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.123392 (+ 15862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.123560 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.199404 13539 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:21.187196 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1323) took 1973 ms. Trace:
I20260709 12:57:21.200170 13538 rpcz_store.cc:276] 0709 12:57:19.213733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.173070 (+1959337us) service_pool.cc:224] Handling call
0709 12:57:21.186577 (+ 13507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.186801 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.186834 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:21.186890 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:21.186949 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.187018 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.187134 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.121572 13611 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:21.115561 13536 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:21.172775 16213 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:21.191613 13615 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:21.173588 16303 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:21.109432 16203 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.141611 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.214381 16136 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:21.213627 13542 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:21.229785 16203 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.229628 16303 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:21.231225 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1325) took 1716 ms. Trace:
I20260709 12:57:21.229159 16136 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.231837 13538 rpcz_store.cc:276] 0709 12:57:19.514490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.202572 (+1688082us) service_pool.cc:224] Handling call
0709 12:57:21.203594 (+  1022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.229839 (+ 26245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.230110 (+   271us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:21.230400 (+   290us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:21.231018 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.231073 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.231168 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.183135 14240 raft_consensus.cc:2889] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:21.234333 13536 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 12:57:21.234966 13537 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:21.234344 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 492) took 1980 ms. Trace:
I20260709 12:57:21.239128 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 494) took 1252 ms. Trace:
I20260709 12:57:21.239089 13539 rpcz_store.cc:276] 0709 12:57:19.254087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.198852 (+1944765us) service_pool.cc:224] Handling call
0709 12:57:21.199187 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.199839 (+   652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.234288 (+ 34449us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:21.234521 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 49852us
W20260709 12:57:21.237727 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 96246.9us
I20260709 12:57:21.239584 13536 rpcz_store.cc:276] 0709 12:57:19.987032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.232036 (+1245004us) service_pool.cc:224] Handling call
0709 12:57:21.234106 (+  2070us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:21.238862 (+  4756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.239076 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.246287 13509 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:21.172003 16215 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.240926 16335 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:21.245265 13539 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:21.240500 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.193248 16157 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:21.234197 16343 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:57:21.190876 16226 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:21.271692 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.229082 16213 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:21.260505 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.260783 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1238) took 1170 ms. Trace:
I20260709 12:57:21.245070 16289 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:21.251096 16273 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:21.192512 16142 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.276055 13659 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.248355 15968 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:21.274852 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 122709us
I20260709 12:57:21.259454 15470 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:21.192824 16228 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.253212 16335 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:21.248980 16176 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:21.262060 16229 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:21.343137 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.307358 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.276309 13536 rpcz_store.cc:276] 0709 12:57:20.090199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.259675 (+1169476us) service_pool.cc:224] Handling call
0709 12:57:21.260052 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.260278 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.260314 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:21.260381 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:21.260433 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.260499 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.260660 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.273679 16226 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:21.373584 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 333) took 2896 ms. Trace:
W20260709 12:57:21.373970 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 97917.1us
W20260709 12:57:21.320300 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.268452 16288 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:21.306348 16289 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:21.521734 16289 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:21.522790 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2506) took 1235 ms. Trace:
I20260709 12:57:21.308772 16281 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.217s
I20260709 12:57:21.381259 16335 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:21.525604 16288 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:21.527298 16288 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.529131 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:21.529829 13536 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:21.530733 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
W20260709 12:57:21.531087 16158 consensus_meta.cc:287] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.000s
I20260709 12:57:21.531473 13536 raft_consensus.cc:606] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:21.555883 13536 raft_consensus.cc:614] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:57:21.500106 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.531957 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.532773 13689 rpcz_store.cc:276] 0709 12:57:18.478340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.895717 (+417377us) service_pool.cc:224] Handling call
0709 12:57:18.907127 (+ 11410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:18.927609 (+ 20482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.373230 (+2445621us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b5400238cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:21.373510 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44562688}
I20260709 12:57:21.523193 13610 rpcz_store.cc:276] 0709 12:57:20.286943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.098170 (+811227us) service_pool.cc:224] Handling call
0709 12:57:21.098529 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:21.098936 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.098978 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:21.099510 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:57:21.213348 (+113838us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:21.213402 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.213470 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:21.522574 (+309104us) raft_consensus.cc:1685] finished
0709 12:57:21.522596 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.522729 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2255,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":13108,"thread_start_us":50931,"threads_started":1,"wal-append.queue_time_us":149312}
I20260709 12:57:21.586570 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:21.587618 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 55756.2us
I20260709 12:57:21.297107 16213 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:21.369800 16217 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 12:57:21.539857 13616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 409) took 2366 ms (client timeout 1866 ms). Trace:
I20260709 12:57:21.756242 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1321) took 2543 ms. Trace:
W20260709 12:57:21.632716 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.709363 16229 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.305337 16311 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:21.764591 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2676) took 2545 ms. Trace:
I20260709 12:57:21.765707 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 496) took 1778 ms. Trace:
I20260709 12:57:21.765290 13692 rpcz_store.cc:276] 0709 12:57:19.219170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:19.948054 (+728884us) service_pool.cc:224] Handling call
0709 12:57:19.948388 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:19.991053 (+ 42665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.764266 (+1773213us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b540002ce54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:21.764503 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69123456}
W20260709 12:57:21.472888 16328 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:57:21.381901 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 820) took 2890 ms. Trace:
I20260709 12:57:21.710073 16136 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.520030 16142 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.710472 16228 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.711783 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:21.545456 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 238160us
W20260709 12:57:21.714483 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.800319 13610 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:21.710505 13536 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:21.756680 13537 rpcz_store.cc:276] 0709 12:57:19.213076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.128784 (+1915708us) service_pool.cc:224] Handling call
0709 12:57:21.129107 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.238844 (+109737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.756008 (+517164us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540028c654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:21.756180 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243011456}
W20260709 12:57:21.755291 13616 rpcz_store.cc:269] 0709 12:57:19.173794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:19.219708 (+ 45914us) service_pool.cc:224] Handling call
0709 12:57:21.539574 (+2319866us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54001293d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:21.539763 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1969342720}
I20260709 12:57:21.802520 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1240) took 1682 ms. Trace:
I20260709 12:57:21.803206 13537 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:57:21.761740 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1294) took 1227 ms. Trace:
I20260709 12:57:21.549382 16317 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:21.755604 16276 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:21.781183 16247 consensus_meta.cc:287] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.000s
I20260709 12:57:21.712802 16157 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:21.784806 16268 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 2.276s	user 0.009s	sys 0.007s
I20260709 12:57:21.785107 16215 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.785681 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:57:21.787529 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.517658 15968 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.789516 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2511) took 1499 ms. Trace:
I20260709 12:57:21.833868 13612 rpcz_store.cc:276] 0709 12:57:20.289685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.122886 (+833201us) service_pool.cc:224] Handling call
0709 12:57:21.155097 (+ 32211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.788489 (+633392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.788525 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:21.788572 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:21.788619 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.789267 (+   648us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540012a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:21.789316 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.789452 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456141696}
W20260709 12:57:21.435292 16341 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:57:21.373430 15957 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:21.766379 13539 rpcz_store.cc:276] 0709 12:57:19.987569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.240034 (+1252465us) service_pool.cc:224] Handling call
0709 12:57:21.240300 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.253607 (+ 13307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.755550 (+501943us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b54003c9e54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:21.765635 (+ 10085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10984320}
I20260709 12:57:21.803992 13685 rpcz_store.cc:276] 0709 12:57:20.533961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:20.910935 (+376974us) service_pool.cc:224] Handling call
0709 12:57:20.911258 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:20.999930 (+ 88672us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:21.000207 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:57:21.162638 (+162431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:21.162790 (+   152us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:21.162859 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.761433 (+598574us) spinlock_profiling.cc:243] Waited 88.4 ms on lock 0x7b54000f28d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:21.761494 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.761663 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":97,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":524,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6825,"spinlock_wait_cycles":231207552,"thread_start_us":161535,"threads_started":1,"wal-append.queue_time_us":259008}
I20260709 12:57:21.712412 16171 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:21.802878 13536 rpcz_store.cc:276] 0709 12:57:20.119888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.709811 (+1589923us) service_pool.cc:224] Handling call
0709 12:57:21.710161 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.802263 (+ 92102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.802467 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:21.806155 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:21.404058 16318 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:21.811681 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1245) took 1576 ms. Trace:
I20260709 12:57:21.808109 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:21.846652 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:21.847096 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:21.848259 13539 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:57:21.795951 16172 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.834s	user 0.010s	sys 0.000s
I20260709 12:57:21.755180 16296 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:21.822854 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:21.784806 16166 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.827780 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:21.785168 16308 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:21.764863 16311 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:21.810773 16136 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:21.887856 16136 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.890239 16136 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.890817 16136 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:21.891294 16136 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:21.819746 13538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.018s	sys 0.094s
I20260709 12:57:21.809725 16317 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:21.862186 13536 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:57:21.867697 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1254) took 1288 ms. Trace:
I20260709 12:57:21.929490 13539 rpcz_store.cc:276] 0709 12:57:20.579522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.866429 (+1286907us) service_pool.cc:224] Handling call
0709 12:57:21.866822 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.867247 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.867287 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:21.867346 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:21.867405 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.867481 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.867622 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5088,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":4121}
I20260709 12:57:21.754142 16158 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.847594 14237 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:21.846689 13537 rpcz_store.cc:276] 0709 12:57:20.235326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.802561 (+1567235us) service_pool.cc:224] Handling call
0709 12:57:21.802945 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:21.803688 (+   743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.811584 (+  7896us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.873432 13612 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:21.876813 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:21.877732 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:21.878161 16203 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:21.878986 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:21.941449 14242 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:21.941645 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:21.942335 13616 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:21.942370 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:21.943185 14242 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:21.869585 16215 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:21.886942 16335 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.835963 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:21.949743 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 113877us
I20260709 12:57:21.909264 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1224) took 3399 ms. Trace:
I20260709 12:57:21.909793 16311 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:21.842335 16171 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:21.963146 16171 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:21.963750 16158 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:21.965827 16335 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:21.916852 16217 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:21.871086 16172 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.918689 16304 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:57:21.919986 16191 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.891s	user 0.009s	sys 0.004s
W20260709 12:57:21.920912 13541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 493) took 2654 ms (client timeout 2088 ms). Trace:
W20260709 12:57:21.836915 16303 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.004s
W20260709 12:57:21.927628 13538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.019s	sys 0.094s
I20260709 12:57:21.929136 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1248) took 1651 ms. Trace:
I20260709 12:57:21.980743 13536 rpcz_store.cc:276] 0709 12:57:20.277500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.861522 (+1584022us) service_pool.cc:224] Handling call
0709 12:57:21.861844 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.928786 (+ 66942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.929059 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:21.981359 13616 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:21.820567 16276 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.870293 16221 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:21.936647 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 417) took 1483 ms. Trace:
I20260709 12:57:21.939030 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:21.875162 16318 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:21.791647 14240 rpcz_store.cc:276] 0709 12:57:18.491618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.502168 (+ 10550us) service_pool.cc:224] Handling call
0709 12:57:18.512498 (+ 10330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:20.810118 (+2297620us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b58007b05d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 12:57:21.288569 (+478451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.373068 (+ 84499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969035136}
I20260709 12:57:21.945751 14242 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:21.900328 16213 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:21.952119 13535 rpcz_store.cc:276] 0709 12:57:18.509786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:19.059164 (+549378us) service_pool.cc:224] Handling call
0709 12:57:19.086460 (+ 27296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:19.087207 (+   747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:19.087240 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:19.118888 (+ 31648us) log.cc:844] Serialized 26 byte log entry
0709 12:57:21.196518 (+2077630us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b180048aa80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:21.248696 (+ 52178us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:21.248734 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.248811 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:21.908934 (+660123us) raft_consensus.cc:1685] finished
0709 12:57:21.908959 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:21.909190 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12230,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":26128,"spinlock_wait_cycles":80724224,"thread_start_us":51991,"threads_started":1,"wal-append.queue_time_us":2708862}
I20260709 12:57:21.845003 16144 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:21.917536 15972 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.913s	user 0.010s	sys 0.000s
W20260709 12:57:21.947166 16273 consensus_meta.cc:287] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.000s
W20260709 12:57:21.973742 13541 rpcz_store.cc:269] 0709 12:57:19.266097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:19.695301 (+429204us) service_pool.cc:224] Handling call
0709 12:57:21.920707 (+2225406us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54000b81d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:21.920833 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376931712}
I20260709 12:57:21.920918 16308 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:21.979861 13508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.022s	sys 0.009s
I20260709 12:57:21.945047 16203 consensus_queue.cc:1243] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:57:22.018311 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.011022 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 418) took 1557 ms. Trace:
I20260709 12:57:22.030190 13616 rpcz_store.cc:276] 0709 12:57:20.453528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.010073 (+1556545us) service_pool.cc:224] Handling call
0709 12:57:22.010414 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.010666 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:22.010699 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:22.010741 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:22.010773 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.010833 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:22.010961 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.019697 14237 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:21.981752 16185 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.039119 14239 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:57:22.040490 13686 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:22.018833 13612 rpcz_store.cc:276] 0709 12:57:20.453048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.835107 (+1382059us) service_pool.cc:224] Handling call
0709 12:57:21.873250 (+ 38143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.936368 (+ 63118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.936589 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:22.027962 13508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.022s	sys 0.009s
I20260709 12:57:22.042829 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:21.973351 16304 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:22.044406 13685 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:22.045928 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:22.049156 13616 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:22.049803 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 422) took 1594 ms. Trace:
I20260709 12:57:22.018249 16221 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:22.050140 13508 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:22.010215 16191 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:22.050092 13616 rpcz_store.cc:276] 0709 12:57:20.454952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.031696 (+1576744us) service_pool.cc:224] Handling call
0709 12:57:22.048993 (+ 17297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.049561 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.049748 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.042981 16144 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.051424 15663 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:22.036576 16119 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.052359 16185 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:21.973152 16217 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.076833 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:21.982949 16303 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.080807 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 495) took 2093 ms. Trace:
I20260709 12:57:21.959419 16215 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.083324 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.038959 16286 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:22.071031 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.076706 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.076970 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.077571 13687 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:22.094481 13691 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:22.055982 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.079481 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.072181 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.095587 13692 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:22.095696 13684 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:22.096056 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.086441 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.041870 16199 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.078109 16119 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.102038 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.102134 15958 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:22.103817 16119 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.106447 13615 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:22.107089 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 416) took 1685 ms. Trace:
I20260709 12:57:22.107288 14241 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:22.107395 13615 rpcz_store.cc:276] 0709 12:57:20.421352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.232826 (+811474us) service_pool.cc:224] Handling call
0709 12:57:22.106049 (+873223us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:22.106231 (+   182us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:22.106828 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.107035 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2469120}
I20260709 12:57:22.108901 13615 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:22.109164 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.109308 16308 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.109450 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1446) took 1246 ms. Trace:
I20260709 12:57:22.109804 13615 rpcz_store.cc:276] 0709 12:57:20.863521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.108447 (+1244926us) service_pool.cc:224] Handling call
0709 12:57:22.108710 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.109227 (+   517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.109401 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.110288 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.109954 16318 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.110891 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.053580 16273 consensus_queue.cc:1059] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:57:22.114393 16332 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:22.114796 16332 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:22.072993 16344 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:22.085179 13616 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:22.117540 13691 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:22.118281 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 425) took 1662 ms. Trace:
I20260709 12:57:22.075168 13534 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:22.020779 13610 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:22.096895 13693 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:22.120919 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.081221 13538 rpcz_store.cc:276] 0709 12:57:19.987308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.236608 (+1249300us) service_pool.cc:224] Handling call
0709 12:57:21.257767 (+ 21159us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:21.258400 (+   633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.258455 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:21.272301 (+ 13846us) log.cc:844] Serialized 52 byte log entry
0709 12:57:21.980408 (+708107us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:21.980446 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:21.980516 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:22.080477 (+ 99961us) raft_consensus.cc:1685] finished
0709 12:57:22.080499 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:22.080636 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":157546,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":73780,"thread_start_us":682487,"threads_started":1,"wal-append.queue_time_us":718842}
I20260709 12:57:22.030957 15972 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.096662 13689 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:22.121959 13615 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:22.099807 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:21.973524 16317 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.135684 16317 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:22.145117 13692 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:22.073189 16221 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:22.113929 16304 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.086807 13612 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:22.147210 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 428) took 1425 ms. Trace:
I20260709 12:57:22.147688 13612 rpcz_store.cc:276] 0709 12:57:20.721501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.073546 (+1352045us) service_pool.cc:224] Handling call
0709 12:57:22.086484 (+ 12938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.146860 (+ 60376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.147136 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.118610 13616 rpcz_store.cc:276] 0709 12:57:20.455936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.072937 (+1617001us) service_pool.cc:224] Handling call
0709 12:57:22.084955 (+ 12018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.118062 (+ 33107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.118231 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.118846 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.131282 13687 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:22.133283 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1452) took 1268 ms. Trace:
I20260709 12:57:22.150359 13615 rpcz_store.cc:276] 0709 12:57:20.865177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.111780 (+1246603us) service_pool.cc:224] Handling call
0709 12:57:22.121731 (+  9951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.133004 (+ 11273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.133225 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.152016 13615 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:22.152771 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1453) took 1187 ms. Trace:
I20260709 12:57:22.134477 15972 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.153147 13615 rpcz_store.cc:276] 0709 12:57:20.965089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.151412 (+1186323us) service_pool.cc:224] Handling call
0709 12:57:22.151784 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.152461 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.152707 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.156075 13615 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:22.156764 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2520) took 1039 ms. Trace:
I20260709 12:57:22.156873 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.157258 13615 rpcz_store.cc:276] 0709 12:57:21.117250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.154551 (+1037301us) service_pool.cc:224] Handling call
0709 12:57:22.155027 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.156489 (+  1462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.156712 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:22.158414 13610 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:57:22.159209 14237 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:22.161515 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.162137 13687 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:22.162433 13685 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:22.075408 16144 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.164193 13540 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 7. Ops: []
I20260709 12:57:22.164674 16213 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.164937 16172 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.119715 16288 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.115574 16332 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.193424 13684 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:22.171262 16226 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.178172 15968 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.182379 13610 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 3. Current term is 4. Ops: []
I20260709 12:57:22.190150 16351 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.196887 15968 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.197564 16351 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.197907 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.198566 13610 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:22.199209 15968 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.178278 16336 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:22.200265 16336 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.202696 16336 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.203075 16171 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:22.194259 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.120072 16199 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.158586 16344 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.213390 16344 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.216949 16146 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.199815 13610 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:57:22.206938 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.208330 13685 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
W20260709 12:57:22.212895 16135 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.033s	user 0.011s	sys 0.000s
I20260709 12:57:22.101534 16286 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.190702 16304 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:22.191411 16276 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.199752 16351 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:22.231243 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.194448 16136 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:22.179243 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 155858us
I20260709 12:57:22.233805 13686 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:22.278759 13617 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:22.233968 13611 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:22.210919 16332 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:22.194648 16288 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.215804 16344 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.232751 16336 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.333985 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:22.335716 13616 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:22.304683 16350 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:57:22.304297 16276 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:57:22.340721 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 223162us
I20260709 12:57:22.351245 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:57:22.351639 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.232757 16296 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:22.364122 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:22.367014 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:22.367784 14242 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:22.352276 16289 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.830s	user 0.006s	sys 0.007s
I20260709 12:57:22.385690 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1317) took 3173 ms. Trace:
I20260709 12:57:22.386543 16199 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.387486 16226 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.336771 16336 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.389780 16336 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.390147 16185 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.382083 16354 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:22.391788 16354 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:22.392155 16354 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:22.396294 16345 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:22.396124 16289 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.360088 16351 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:57:22.398355 16289 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:22.398896 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.384891 16157 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:57:22.400159 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 431) took 1476 ms. Trace:
I20260709 12:57:22.400521 13616 rpcz_store.cc:276] 0709 12:57:20.923769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.149511 (+1225742us) service_pool.cc:224] Handling call
0709 12:57:22.162754 (+ 13243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.336147 (+173393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.399930 (+ 63783us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54002fc6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:22.400098 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397310848}
I20260709 12:57:22.356237 16135 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:57:22.347836 16276 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:22.386154 13542 rpcz_store.cc:276] 0709 12:57:19.212281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.125643 (+1913362us) service_pool.cc:224] Handling call
0709 12:57:21.127995 (+  2352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:21.229291 (+101296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.385439 (+1156148us) spinlock_profiling.cc:243] Waited 85.4 ms on lock 0x7b54002837d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:22.385628 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196397824}
I20260709 12:57:22.410871 14242 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:22.352788 16309 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:57:22.416180 13508 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:22.372678 16303 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.360350 16213 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:57:22.424472 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:22.425215 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.371218 16191 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:22.400444 16171 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:22.428853 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.384277 16215 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.373019 16344 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:57:22.435521 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.353634 16360 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:57:22.437256 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 523) took 2164 ms. Trace:
I20260709 12:57:22.357066 16353 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
W20260709 12:57:22.441222 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 230245us
I20260709 12:57:22.403088 16172 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:57:22.429692 16226 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:22.433348 16181 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:22.444438 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.427332 16357 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.477134 14242 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:22.488013 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:22.456560 16359 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:57:22.489951 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:22.449085 15663 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:22.418422 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.426165 16273 consensus_queue.cc:1243] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 6.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:57:22.496121 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 149081us
I20260709 12:57:22.458475 14246 rpcz_store.cc:276] 0709 12:57:20.273035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:20.273166 (+   131us) service_pool.cc:224] Handling call
0709 12:57:21.084736 (+811570us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b5800003c98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:21.084983 (+   247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:21.252295 (+167312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:21.252371 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:21.259000 (+  6629us) log.cc:844] Serialized 26 byte log entry
0709 12:57:21.832930 (+573930us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:22.223861 (+390931us) log.cc:844] Serialized 14 byte log entry
0709 12:57:22.426425 (+202564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.426505 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:22.426549 (+    44us) raft_consensus.cc:1685] finished
0709 12:57:22.437060 (+ 10511us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:22.437197 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14386,"prepare.queue_time_us":539792,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":79353,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":38622,"spinlock_wait_cycles":2911616,"thread_start_us":1003423,"threads_started":3,"wal-append.queue_time_us":750914}
I20260709 12:57:22.403854 16166 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:57:22.436650 16144 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.500540 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.501370 13690 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:22.493862 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:22.500753 14212 leader_election.cc:336] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.866s (SENT)
I20260709 12:57:22.498807 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.505036 13613 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
W20260709 12:57:22.501390 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 137579us
I20260709 12:57:22.510082 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.510598 16345 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:22.510767 13693 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:22.507617 14212 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:22.511453 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.476938 16171 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.519466 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.523639 13685 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:22.502817 16212 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:22.511458 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.531203 16345 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.577308 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.578028 16281 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:57:22.532627 16337 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.153s	user 0.010s	sys 0.000s
W20260709 12:57:22.533183 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.535018 13691 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:57:22.582387 16288 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.508639 16335 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:22.582902 16229 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.583246 16335 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:22.583673 16335 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.584064 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 490) took 3347 ms. Trace:
I20260709 12:57:22.584167 14242 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:22.584431 13540 rpcz_store.cc:276] 0709 12:57:19.237073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.198180 (+1961107us) service_pool.cc:224] Handling call
0709 12:57:21.241191 (+ 43011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.177048 (+935857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.583806 (+406758us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54000dbe54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:22.584007 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2122249472}
I20260709 12:57:22.588235 16344 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:22.487740 16276 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:22.589617 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.589897 16276 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.590047 16336 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:22.591464 16336 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:22.592339 16344 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:22.592772 16344 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:22.576026 16135 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:22.593743 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.593612 16344 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:22.593771 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.594717 13692 raft_consensus.cc:2410] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:57:22.594946 16344 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.596001 16155 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.576136 16212 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:22.596566 16155 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.597218 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2509) took 2309 ms. Trace:
I20260709 12:57:22.597000 16212 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.598022 16212 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:22.597641 13617 rpcz_store.cc:276] 0709 12:57:20.287606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.100036 (+812430us) service_pool.cc:224] Handling call
0709 12:57:21.100307 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.279865 (+1179558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.596984 (+317119us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002fcbd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:22.597146 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709512448}
I20260709 12:57:22.598759 16155 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.549937 14244 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:22.599443 16136 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:22.599890 16136 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:22.600090 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.589912 16371 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:57:22.600936 14244 raft_consensus.cc:2410] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:22.552914 15833 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:22.601006 16136 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.602310 16136 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.560178 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 476) took 4577 ms. Trace:
I20260709 12:57:22.602941 13543 rpcz_store.cc:276] 0709 12:57:17.983323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.048655 (+ 65332us) service_pool.cc:224] Handling call
0709 12:57:20.270887 (+2222232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:20.273901 (+  3014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.559939 (+2286038us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b54002841d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:22.560091 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5109886720}
I20260709 12:57:22.604743 14239 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:57:22.605094 16323 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:22.565201 13534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1235) took 2475 ms (client timeout 2980 ms). Trace:
I20260709 12:57:22.605602 16323 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:22.605738 13534 rpcz_store.cc:269] 0709 12:57:20.089467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:20.858765 (+769298us) service_pool.cc:224] Handling call
0709 12:57:22.565006 (+1706241us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540007dfd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:22.565137 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2526889216}
W20260709 12:57:22.567054 13619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 412) took 2178 ms (client timeout 1528 ms). Trace:
I20260709 12:57:22.495827 16357 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:22.607031 13619 rpcz_store.cc:269] 0709 12:57:20.388755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:20.420953 (+ 32198us) service_pool.cc:224] Handling call
0709 12:57:22.566837 (+2145884us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540013bad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:22.566986 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717070720}
I20260709 12:57:22.608587 16357 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.503391 16181 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:22.609575 13691 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:22.609977 16181 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.610458 16344 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:22.503217 16361 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.610659 16323 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.525645 16191 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:22.558487 16311 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.001s
I20260709 12:57:22.545931 16217 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.547115 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 413) took 2158 ms. Trace:
I20260709 12:57:22.594074 16135 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:22.600643 16351 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:22.526572 16286 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.612246 16361 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.601338 16336 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:22.613788 13689 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:22.613349 13612 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:57:22.614531 14238 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 12:57:22.628098 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 120587us
I20260709 12:57:22.637804 16351 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:22.638767 14212 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:22.639807 15958 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:22.642174 15958 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:22.643589 16362 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:22.644084 16362 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:22.644856 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.644975 16362 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.645807 13686 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:22.646821 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.653558 15958 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:22.544732 16101 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:22.656140 16101 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:22.657128 16101 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.659030 16101 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:22.615291 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.500696 16296 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.661938 16176 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:22.633538 16135 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.635175 14240 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:22.648463 16360 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:22.629886 15946 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:22.665268 15946 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:22.665774 15946 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.635294 16336 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:22.667400 16336 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.664470 16135 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.668624 16336 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:22.636013 13612 pending_rounds.cc:85] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 5
I20260709 12:57:22.676527 13612 pending_rounds.cc:107] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:57:22.677125 13612 raft_consensus.cc:2889] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:22.677802 16148 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:22.678889 16148 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.634083 16217 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:22.680008 16155 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:22.680773 16217 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.681555 16158 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:22.681954 16158 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:22.683418 16217 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.683001 16158 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.628978 16311 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:22.632005 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.632261 13611 rpcz_store.cc:276] 0709 12:57:20.388983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:21.232694 (+843711us) service_pool.cc:224] Handling call
0709 12:57:21.233875 (+  1181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.288459 (+1054584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.546798 (+258339us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b540013abd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:22.547033 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299536768}
I20260709 12:57:22.574208 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:22.633463 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.701727 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:22.671788 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.714903 15968 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:22.703199 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:22.698259 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 348434us
W20260709 12:57:22.716246 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.717342 16323 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:22.717804 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.707134 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.621819 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:22.654632 13618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 421) took 2200 ms (client timeout 1932 ms). Trace:
I20260709 12:57:22.687304 16146 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:22.635771 16361 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.701638 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:22.720067 13618 rpcz_store.cc:269] 0709 12:57:20.454610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:20.719987 (+265377us) service_pool.cc:224] Handling call
0709 12:57:22.654402 (+1934415us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b54000762d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:22.654528 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208366464}
I20260709 12:57:22.720384 16146 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.683622 13685 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:22.721494 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:22.717495 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 143378us
I20260709 12:57:22.655043 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.526911 16171 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.636083 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:22.724454 13686 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:57:22.724747 13689 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:22.725689 13691 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:22.725751 16171 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.718658 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:22.685580 16148 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.699707 16166 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:57:22.739833 16362 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:22.687882 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.712672 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:22.619575 16366 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 12:57:22.742611 14240 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:22.743762 14237 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:57:22.744171 14237 pending_rounds.cc:85] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 6
I20260709 12:57:22.744410 14237 pending_rounds.cc:107] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:57:22.744623 14237 raft_consensus.cc:2889] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:22.744918 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.720301 13540 raft_consensus.cc:2410] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:22.745797 14240 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:22.725835 13687 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:22.746608 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:22.746712 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:22.606602 16221 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.747224 14237 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.747360 14240 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:22.748624 16158 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.750602 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 430) took 1827 ms. Trace:
I20260709 12:57:22.751120 13612 rpcz_store.cc:276] 0709 12:57:20.923349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.148666 (+1225317us) service_pool.cc:224] Handling call
0709 12:57:22.569292 (+420626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.677642 (+108350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.679380 (+  1738us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b540013bad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:22.750414 (+ 71034us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540013bad0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:22.750544 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066731136}
I20260709 12:57:22.752871 16171 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:22.753137 16221 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.735093 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:57:22.735453 16155 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:22.737067 16146 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.743878 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:22.543795 16354 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.753439 16171 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:22.753983 16318 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:22.827023 13536 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:22.755554 13659 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:22.827852 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 72356.7us
I20260709 12:57:22.828248 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.828997 13536 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:22.652949 14212 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:22.762432 13688 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:22.764775 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.734758 16311 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:22.769191 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:22.770018 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.515048 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:22.790109 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.836760 13541 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:22.837890 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 322884us
I20260709 12:57:22.838210 13543 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 5: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:22.838907 13538 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:57:22.838847 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:22.810493 14237 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:22.812462 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.813846 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.843087 13539 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.824666 13615 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:22.690070 16351 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:22.711086 16336 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:22.830864 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 178009us
I20260709 12:57:22.829888 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.877472 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.830196 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.878044 14246 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 8. Ops: []
I20260709 12:57:22.878346 13536 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:22.878705 13535 raft_consensus.cc:2410] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:57:22.841620 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 72549.6us
I20260709 12:57:22.877475 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:22.846390 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:22.846732 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:22.882179 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:22.847745 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:22.851902 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46455.9us
I20260709 12:57:22.883020 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:22.884356 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:22.884770 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:22.885731 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:22.887035 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:22.888278 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:22.855316 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.888979 16225 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:22.889150 16203 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.889005 13610 raft_consensus.cc:2376] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8. Re-sending same reply.
I20260709 12:57:22.890061 16203 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.891139 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.874972 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.754832 16183 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:22.892050 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.870489 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:22.888949 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.892848 13617 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:22.747599 16335 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.888171 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.893496 14241 raft_consensus.cc:2410] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:57:22.894117 13535 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:22.889863 16225 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:22.894781 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.895207 16335 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.895296 16225 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.898473 14241 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:57:22.900601 16309 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:22.875219 16332 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:22.896099 16288 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:22.914675 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.896780 16226 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:22.919842 13543 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:22.897119 16203 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:22.897289 16369 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:22.929023 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1457) took 1669 ms. Trace:
I20260709 12:57:22.929952 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:22.845024 15833 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.930907 13617 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.844393 16323 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:22.842581 16191 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:22.893671 16229 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.890482 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.936383 13611 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:22.889809 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.871904 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.910418 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.937254 13618 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:22.937376 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:22.916185 16376 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.090s	user 0.002s	sys 0.000s
I20260709 12:57:22.913463 16213 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:22.914469 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.915396 13541 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:22.794903 16155 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.917098 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.953327 13536 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:22.954227 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.954881 13536 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:22.844677 16148 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 12:57:22.917932 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.956154 13617 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:22.956446 13542 raft_consensus.cc:2410] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:22.918939 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.921185 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:22.957386 13538 raft_consensus.cc:2376] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4. Re-sending same reply.
I20260709 12:57:22.958065 13610 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:22.921541 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 165779us
I20260709 12:57:22.830960 16166 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:22.960713 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:22.869971 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.927287 13543 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:22.955631 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.963016 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.963464 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.964335 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.965221 13543 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:22.965359 15946 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:22.965927 16166 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:22.966382 16166 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.966320 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.967181 13543 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:22.928035 13539 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:22.968539 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.969565 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.932338 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.929982 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.932911 14243 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:22.929632 13615 rpcz_store.cc:276] 0709 12:57:21.259723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.163038 (+903315us) service_pool.cc:224] Handling call
0709 12:57:22.163297 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:22.845085 (+681788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:22.919124 (+ 74039us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b540013a1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:22.928936 (+  9812us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520222080}
I20260709 12:57:22.971946 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.972275 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.934264 13553 tablet_copy_service.cc:342] P ad2fdb1517c542439654bd3f7c72dad0: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-048528e413ac4c67a1cfd3dfe70673c7 received from {username='slave'} at 127.0.0.1:45566
W20260709 12:57:22.965166 13437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49896 (request call id 303) took 21305 ms (client timeout 15000 ms). Trace:
I20260709 12:57:22.973313 14239 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:22.974298 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.974416 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:22.975082 13542 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:22.938630 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.912791 13612 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:22.873862 16221 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:22.939620 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:22.927261 16372 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:57:22.944945 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:22.984186 13616 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:22.875599 16296 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:22.947448 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:22.909385 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:22.917246 16311 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:22.914942 16288 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:22.919724 15958 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:22.964538 16382 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:57:22.967682 15946 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:22.930516 16181 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:57:22.977162 13537 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:22.926374 16363 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:22.957376 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:23.048985 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 91693.6us
I20260709 12:57:22.995179 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:23.050943 13542 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:23.052479 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:22.998034 13610 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:22.999356 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.999269 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:22.916632 16286 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:57:23.001247 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:23.006613 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:23.014003 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 104704us
I20260709 12:57:22.947108 15833 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:23.017230 14241 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:57:22.925510 16276 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:57:23.076849 13584 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:23.017776 13538 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:22.948798 16213 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:22.917449 16336 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:22.931185 16361 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:57:23.016564 13553 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-048528e413ac4c67a1cfd3dfe70673c7 on tablet 048528e413ac4c67a1cfd3dfe70673c7 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:23.032593 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:23.028671 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:22.969818 16183 consensus_queue.cc:1243] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:57:23.023293 16296 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:23.038558 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:22.942272 15789 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:57:22.992977 16303 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:22.956526 16332 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:23.038543 13535 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:22.971429 16191 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:22.949514 16229 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:23.019485 15958 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:23.060617 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:22.671980 16185 consensus_queue.cc:1175] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" responder_term: 6 status { last_received { term: 5 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:57:23.068190 16363 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:23.070206 13618 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:22.989895 16323 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:23.072175 16144 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:22.904743 16351 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:23.074270 13540 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:22.989499 16171 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:23.181537 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:23.181699 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:23.182274 13538 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:23.075098 14246 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:57:23.182825 14246 pending_rounds.cc:85] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 6
I20260709 12:57:23.183089 14246 pending_rounds.cc:107] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:57:23.183430 14246 raft_consensus.cc:2889] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:23.076084 13615 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:23.078428 13537 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:23.185035 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:23.186090 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:23.067220 16166 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:23.081153 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:23.081715 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:23.190828 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 109206us
I20260709 12:57:23.020848 15865 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:23.012696 16311 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:23.209209 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:23.023372 13616 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:23.120941 16304 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:57:23.120224 13617 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:23.073210 16384 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:57:22.909200 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:23.122187 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 69834.1us
I20260709 12:57:23.243854 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:23.244975 13616 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:23.149515 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:23.245599 13535 raft_consensus.cc:2376] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8. Re-sending same reply.
I20260709 12:57:23.153743 13536 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
W20260709 12:57:22.973201 13437 rpcz_store.cc:269] 0709 12:57:01.660015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:01.660531 (+   516us) service_pool.cc:224] Handling call
0709 12:57:22.965064 (+21304533us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:57:13.711330 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:57:13.729040 (+ 17710us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:57:13.729096 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:57:13.729134 (+    38us) tablet.cc:662] Decoding operations
0709 12:57:13.970727 (+241593us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:57:14.037652 (+ 66925us) write_op.cc:641] Partition lock acquired for write op
0709 12:57:14.040911 (+  3259us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:57:14.097097 (+ 56186us) tablet.cc:701] Row locks acquired
0709 12:57:14.097150 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:57:14.097326 (+   176us) write_op.cc:270] Start()
0709 12:57:14.097477 (+   151us) write_op.cc:276] Timestamp: P: 1783601834097262 usec, L: 0
0709 12:57:14.097495 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:57:14.226512 (+129017us) log.cc:844] Serialized 2820 byte log entry
0709 12:57:14.531801 (+305289us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b540041e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ade28 00007fa0321e8581 00007fa0321e7d58 00007fa0321ec9f1 00007fa0321ec9a9 00007fa0321ec939 00007fa0321ec901 00007fa0321ebb2d 000000000038f354 000000000038f199
0709 12:57:14.945629 (+413828us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b18001664e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6932a6 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db
0709 12:57:14.946680 (+  1051us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:16.290445 (+1343765us) op_driver.cc:464] REPLICATION: finished
0709 12:57:16.950298 (+659853us) write_op.cc:301] APPLY: starting
0709 12:57:16.962173 (+ 11875us) tablet.cc:1367] starting BulkCheckPresence
0709 12:57:16.996345 (+ 34172us) tablet.cc:1370] finished BulkCheckPresence
0709 12:57:16.996409 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:57:17.172347 (+175938us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:57:17.172691 (+   344us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:57:17.172738 (+    47us) write_op.cc:312] APPLY: finished
0709 12:57:17.173060 (+   322us) log.cc:844] Serialized 59 byte log entry
0709 12:57:17.173288 (+   228us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:17.173406 (+   118us) write_op.cc:454] Released schema lock
0709 12:57:17.173464 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:57:17.812789 (+639325us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b540041e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7af32a 00007fa02e734f14 00007fa02e734ec3 00007fa02e734e38 00007fa02e734df3 00007fa02e7340dc 00007fa02e75de16 00007fa02e745cbb 00007fa02e73039d 00007fa02e733d75
0709 12:57:18.826006 (+1013217us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b18001664e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:19.677334 (+851328us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b18001664e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:19.677481 (+   147us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:19.677506 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2540879,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":585531,"apply.run_cpu_time_us":5313,"apply.run_wall_time_us":237319,"num_ops":7,"prepare.queue_time_us":250366,"prepare.run_cpu_time_us":25222,"prepare.run_wall_time_us":850037,"raft.queue_time_us":416616,"raft.run_cpu_time_us":56330,"raft.run_wall_time_us":1675971,"replication_time_us":2116564,"spinlock_wait_cycles":365296000,"thread_start_us":762988,"threads_started":4,"wal-append.queue_time_us":332424,"wal-append.run_cpu_time_us":33559,"wal-append.run_wall_time_us":3612605}]]}
I20260709 12:57:23.248324 13540 pending_rounds.cc:85] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 5
I20260709 12:57:23.248737 13540 pending_rounds.cc:107] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:57:23.249044 13540 raft_consensus.cc:2889] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:23.164290 14242 raft_consensus.cc:2410] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:57:23.118532 16215 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:23.112185 16226 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:57:23.191578 13582 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:23.196273 15865 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:23.191854 13610 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:57:23.196069 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 146108us
W20260709 12:57:23.278504 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 87002.2us
I20260709 12:57:23.140604 16183 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:23.152652 16371 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:23.199185 13542 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:57:23.289213 13542 pending_rounds.cc:85] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 7
I20260709 12:57:23.289480 13542 pending_rounds.cc:107] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:57:23.167343 15958 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:23.163733 16191 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:23.116741 16221 consensus_queue.cc:1059] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:57:23.119897 13541 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:23.211606 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:23.238669 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:23.133340 16146 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 12:57:23.212715 13441 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 4, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:23.246102 16354 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:23.243486 14212 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:23.020642 16288 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:23.208746 16101 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:23.167558 16303 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:23.325546 13441 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 5, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 5 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:23.279579 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:23.329543 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 50035.2us
I20260709 12:57:23.175701 16332 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:23.289714 13542 raft_consensus.cc:2889] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:23.247368 13539 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:23.332772 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:23.310043 13538 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:23.310360 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65565.5us
I20260709 12:57:23.313321 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:23.262887 16376 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:57:23.310477 14241 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:23.315989 13617 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:23.263062 16215 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:23.349093 13615 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:57:23.339267 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:23.340658 13611 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:23.346626 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:23.295060 16373 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:57:23.285506 15865 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:23.354703 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:23.288836 16203 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:23.287284 16144 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:23.289105 16371 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:23.357827 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:23.358774 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:23.368767 13614 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:23.369241 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:23.369787 13615 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:23.328400 16135 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.660s	user 0.006s	sys 0.005s
I20260709 12:57:23.361382 13538 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:23.362123 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:23.362866 13441 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:23.366590 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:23.371665 13617 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:23.366837 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:23.308346 16199 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:57:23.347935 16304 consensus_queue.cc:1243] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:57:23.350023 16221 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:57:23.370924 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:23.424630 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:23.375061 13441 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:23.356987 16215 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:23.378472 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:23.432852 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:23.437772 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:23.438210 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:23.439204 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:23.440388 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:23.440598 13615 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:23.441597 13441 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 4, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:23.360069 16155 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:23.448822 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:23.454351 16410 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:23.400620 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:23.408304 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:23.411859 14237 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:23.468518 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 101753us
I20260709 12:57:23.469774 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:23.474255 13617 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:23.475382 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:23.422362 14241 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:23.373315 15833 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:23.309170 16296 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:23.447424 16393 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:57:23.478056 14237 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:23.448017 16404 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:57:23.487532 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:23.447598 16221 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:23.488132 16304 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:23.480216 13614 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:23.322301 16311 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:23.477919 13543 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.004s
I20260709 12:57:23.346684 16323 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:23.411145 16395 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 12:57:23.345213 16363 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:23.479624 14241 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:57:24.045380 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.047284 14242 raft_consensus.cc:2376] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8. Re-sending same reply.
W20260709 12:57:24.045943 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 613112us
W20260709 12:57:24.054787 13535 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.004s	sys 0.007s
W20260709 12:57:24.046231 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 576495us
W20260709 12:57:24.055341 13535 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.004s	sys 0.007s
I20260709 12:57:23.504936 13441 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 6, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 6 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:57:23.803376 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 842559us
W20260709 12:57:23.805157 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:23.816998 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 573527us
I20260709 12:57:23.821136 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:24.047214 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 478) took 1016 ms. Trace:
I20260709 12:57:24.057798 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:23.718317 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 462) took 1001 ms. Trace:
I20260709 12:57:24.060894 14239 rpcz_store.cc:276] 0709 12:57:22.716809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.871706 (+154897us) service_pool.cc:224] Handling call
0709 12:57:23.718091 (+846385us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b54005915d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:23.718252 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82437376}
I20260709 12:57:24.063318 13543 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:24.063997 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1266) took 1129 ms. Trace:
I20260709 12:57:24.064391 13543 rpcz_store.cc:276] 0709 12:57:22.934921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.966072 (+ 31151us) service_pool.cc:224] Handling call
0709 12:57:24.063903 (+1097831us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:23.534842 13541 raft_consensus.cc:2410] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:57:23.721527 14246 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260709 12:57:24.071287 14246 pending_rounds.cc:85] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 8
I20260709 12:57:24.071650 14246 pending_rounds.cc:107] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 12:57:24.072052 14246 raft_consensus.cc:2889] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:24.073072 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 541) took 1013 ms. Trace:
I20260709 12:57:24.073628 14246 rpcz_store.cc:276] 0709 12:57:23.060037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.720741 (+660704us) service_pool.cc:224] Handling call
0709 12:57:23.721072 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.072730 (+351658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.072996 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:23.945212 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.020s	sys 0.018s
I20260709 12:57:24.058522 14212 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:24.168828 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 111975us
I20260709 12:57:24.170064 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:24.057302 14241 rpcz_store.cc:276] 0709 12:57:23.031097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.478763 (+447666us) service_pool.cc:224] Handling call
0709 12:57:23.479202 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.046961 (+567759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.047160 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:24.171772 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.172611 14241 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:24.173758 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:24.059248 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:24.180084 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.020s	sys 0.022s
I20260709 12:57:24.082267 13617 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:23.743654 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:24.086858 13538 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:24.086362 13441 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 7, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:24.087183 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.087414 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1269) took 1151 ms. Trace:
I20260709 12:57:24.183311 13541 rpcz_store.cc:276] 0709 12:57:22.935613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.975160 (+ 39547us) service_pool.cc:224] Handling call
0709 12:57:24.087342 (+1112182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:23.881732 16335 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.045s	sys 0.063s
W20260709 12:57:23.620462 14240 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.872s	user 0.011s	sys 0.000s
I20260709 12:57:24.047832 16181 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:24.146677 16288 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.796s	user 0.006s	sys 0.004s
I20260709 12:57:23.875795 16332 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.092255 16361 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:23.817741 16364 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:57:24.012485 16413 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:57:23.803767 16136 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.290s
W20260709 12:57:24.211186 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 151994us
I20260709 12:57:23.636346 16229 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:23.844985 16408 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:57:23.665782 16394 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 12:57:23.684021 16166 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:23.692514 16351 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:23.791225 16158 consensus_meta.cc:287] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.042s	user 0.013s	sys 0.000s
I20260709 12:57:24.203519 15946 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:23.506238 16402 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:23.793903 16225 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.895s	user 0.007s	sys 0.004s
I20260709 12:57:24.096853 16354 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:23.666314 16392 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.413s	user 0.000s	sys 0.001s
I20260709 12:57:23.695386 16410 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:23.842880 16303 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:24.046576 16336 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.095366 16176 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:24.100908 16309 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:23.986927 16191 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
I20260709 12:57:23.506685 15789 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:23.549580 16199 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:23.868927 16360 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:23.548714 16203 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:24.085561 16185 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:24.070555 16396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.000s	sys 0.010s
I20260709 12:57:24.186802 16212 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:24.188817 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:24.257565 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.259440 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.260452 14246 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:24.255450 16396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.000s	sys 0.012s
I20260709 12:57:24.254436 16360 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:24.248605 16354 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:23.803558 16226 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:57:24.145370 16215 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.001s
I20260709 12:57:23.716389 16318 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:24.166541 16353 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:24.201678 16335 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.051s	sys 0.072s
I20260709 12:57:24.108543 16304 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.200596 16418 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.201668 16362 raft_consensus.cc:2764] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:57:23.722014 16308 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:24.222978 16416 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:57:23.604203 16371 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.222313 16286 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:24.221587 16364 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:24.220630 16408 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:24.270535 16318 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:24.282531 16311 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:24.288779 16408 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.289681 16354 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.289469 16364 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.290407 16408 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:24.274982 16418 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.299285 16410 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:24.200523 13441 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 6, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 6 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:24.332247 13534 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:24.208159 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 515) took 1194 ms. Trace:
W20260709 12:57:24.334002 13542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.048s	sys 0.075s
W20260709 12:57:24.167554 13537 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.013s	sys 0.008s
W20260709 12:57:24.167446 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 51174.6us
W20260709 12:57:24.334657 13542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.048s	sys 0.075s
I20260709 12:57:24.098973 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1265) took 1164 ms. Trace:
I20260709 12:57:24.334054 13538 rpcz_store.cc:276] 0709 12:57:23.013394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.371450 (+358056us) service_pool.cc:224] Handling call
0709 12:57:24.086688 (+715238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.182044 (+ 95356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.207984 (+ 25940us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54000b86d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:24.208106 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1644134272}
I20260709 12:57:24.336799 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1273) took 1374 ms. Trace:
W20260709 12:57:24.337152 13537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.013s	sys 0.008s
W20260709 12:57:24.202059 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.337445 13542 rpcz_store.cc:276] 0709 12:57:22.962504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.350157 (+387653us) service_pool.cc:224] Handling call
0709 12:57:23.350535 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:23.350812 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:23.350849 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:23.361647 (+ 10798us) log.cc:844] Serialized 26 byte log entry
0709 12:57:24.335762 (+974115us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:24.335910 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.336118 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.336184 (+    66us) raft_consensus.cc:1685] finished
0709 12:57:24.336194 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.336595 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":888303,"threads_started":1,"wal-append.queue_time_us":895179}
I20260709 12:57:24.341536 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1264) took 1407 ms. Trace:
W20260709 12:57:24.206647 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 119536us
I20260709 12:57:24.341920 13537 rpcz_store.cc:276] 0709 12:57:22.934172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.186923 (+252751us) service_pool.cc:224] Handling call
0709 12:57:23.187198 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.340828 (+1153630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:24.340906 (+    78us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:24.340998 (+    92us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:24.341089 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.341224 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.341459 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":169286,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5190,"thread_start_us":1150140,"threads_started":1}
I20260709 12:57:24.343343 13538 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:57:24.343668 13534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1336) took 1579 ms (client timeout 1631 ms). Trace:
I20260709 12:57:24.343964 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:24.344367 13534 rpcz_store.cc:269] 0709 12:57:22.764148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.769845 (+  5697us) service_pool.cc:224] Handling call
0709 12:57:24.343368 (+1573523us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000203d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:24.343533 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3149904128}
I20260709 12:57:24.154336 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1272) took 1215 ms. Trace:
I20260709 12:57:24.212700 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:24.239358 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:24.346442 13535 rpcz_store.cc:276] 0709 12:57:22.939270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.246346 (+307076us) service_pool.cc:224] Handling call
0709 12:57:23.246664 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:23.308977 (+ 62313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:23.309045 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:23.325661 (+ 16616us) log.cc:844] Serialized 26 byte log entry
0709 12:57:24.055930 (+730269us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:24.055989 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.056084 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.154032 (+ 97948us) raft_consensus.cc:1685] finished
0709 12:57:24.154061 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.154246 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72865,"raft.queue_time_us":117082,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":60854,"thread_start_us":734890,"threads_started":2,"wal-append.queue_time_us":718008}
I20260709 12:57:24.348419 13687 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 4: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:24.088828 13539 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:24.349025 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 143714us
I20260709 12:57:24.350816 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:24.223209 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:24.142916 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 441) took 1231 ms. Trace:
I20260709 12:57:24.189363 13610 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:24.353253 13541 raft_consensus.cc:2410] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:57:24.232578 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:24.228420 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:24.385080 13688 raft_consensus.cc:2410] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:57:24.224346 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:24.213970 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:24.388356 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.212808 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.417212 16418 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:24.418915 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 206016us
W20260709 12:57:24.217747 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 474184us
I20260709 12:57:24.209218 13684 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:24.429752 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 476) took 1459 ms. Trace:
I20260709 12:57:24.431264 16304 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:24.211551 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:24.236558 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:24.276330 16286 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:24.201399 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 109970us
W20260709 12:57:24.198582 14237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 474) took 1362 ms (client timeout 1503 ms). Trace:
I20260709 12:57:24.201776 14240 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
W20260709 12:57:24.482782 14240 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 469) took 1763 ms (client timeout 1888 ms). Trace:
W20260709 12:57:24.483418 14240 rpcz_store.cc:269] 0709 12:57:22.718944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.746432 (+ 27488us) service_pool.cc:224] Handling call
0709 12:57:24.482492 (+1736060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:24.484593 13686 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:24.485344 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.486275 16363 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:24.195001 14241 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:24.184317 15663 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.489913 15663 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.244230 15946 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.492144 13539 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:24.494190 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 547) took 1418 ms. Trace:
I20260709 12:57:24.495151 15946 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.221153 15968 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:24.496709 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:24.498535 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:23.620656 16144 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.226598 16146 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:57:24.499542 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.205257 16181 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.500168 16146 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:24.501809 16146 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.503012 16181 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.252579 16191 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.255705 16199 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.506011 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:24.507323 16354 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:24.256450 16212 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:24.508515 16212 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:23.714394 16213 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.247071 16221 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.246503 16225 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.212069 16288 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.253379 16309 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.239038 16417 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:24.544118 13619 raft_consensus.cc:2393] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:24.494663 14243 rpcz_store.cc:276] 0709 12:57:23.061544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:24.047034 (+985490us) service_pool.cc:224] Handling call
0709 12:57:24.047334 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.047543 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:24.359337 (+311794us) log.cc:844] Serialized 14 byte log entry
0709 12:57:24.433118 (+ 73781us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:24.441095 (+  7977us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:24.459270 (+ 18175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.464604 (+  5334us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.479595 (+ 14991us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3828,"prepare.queue_time_us":311570,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":65789,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":239,"thread_start_us":214993,"threads_started":2,"wal-append.queue_time_us":19107}
I20260709 12:57:24.544575 16417 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.546000 16288 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.328981 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:57:24.364151 15958 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.042s	user 0.007s	sys 0.004s
I20260709 12:57:24.255599 16203 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.336392 13540 rpcz_store.cc:276] 0709 12:57:22.934560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.210209 (+275649us) service_pool.cc:224] Handling call
0709 12:57:23.248075 (+ 37866us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:23.295171 (+ 47096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:23.295203 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:23.295759 (+   556us) log.cc:844] Serialized 52 byte log entry
0709 12:57:24.045555 (+749796us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:24.098403 (+ 52848us) log.cc:844] Serialized 14 byte log entry
0709 12:57:24.098629 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.098710 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.098752 (+    42us) raft_consensus.cc:1685] finished
0709 12:57:24.098767 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.098913 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":193028,"prepare.queue_time_us":155081,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":120074,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":55898,"thread_start_us":303845,"threads_started":2,"wal-append.queue_time_us":729730}
I20260709 12:57:24.389264 16318 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.391079 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2719) took 1571 ms. Trace:
I20260709 12:57:24.272282 16353 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.400434 16414 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:24.412792 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1274) took 1396 ms. Trace:
W20260709 12:57:24.413466 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 62977us
I20260709 12:57:24.399168 16335 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:57:24.269598 16226 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:24.427448 15789 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:24.352977 13613 rpcz_store.cc:276] 0709 12:57:22.911161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.911290 (+   129us) service_pool.cc:224] Handling call
0709 12:57:22.911625 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:22.915275 (+  3650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:22.915308 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:22.924166 (+  8858us) log.cc:844] Serialized 52 byte log entry
0709 12:57:23.112529 (+188363us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:23.132451 (+ 19922us) log.cc:844] Serialized 14 byte log entry
0709 12:57:23.132722 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:23.132804 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:23.779722 (+646918us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b18000b45e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:24.101562 (+321840us) raft_consensus.cc:1685] finished
0709 12:57:24.142633 (+ 41071us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.142827 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3209,"prepare.queue_time_us":20292,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":3972,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":35690,"spinlock_wait_cycles":8412032,"thread_start_us":146726,"threads_started":1,"wal-append.queue_time_us":211111}
I20260709 12:57:24.256637 16185 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.430311 14244 rpcz_store.cc:276] 0709 12:57:22.970392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.970697 (+   305us) service_pool.cc:224] Handling call
0709 12:57:23.028208 (+ 57511us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:23.804193 (+775985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:23.804248 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:24.048317 (+244069us) log.cc:844] Serialized 52 byte log entry
0709 12:57:24.427865 (+379548us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:24.428570 (+   705us) log.cc:844] Serialized 14 byte log entry
0709 12:57:24.429000 (+   430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.429168 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.429257 (+    89us) raft_consensus.cc:1685] finished
0709 12:57:24.429276 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.429502 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":788801,"raft.queue_time_us":776009,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":152,"thread_start_us":301717,"threads_started":1,"wal-append.queue_time_us":200033}
I20260709 12:57:24.293893 16371 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:24.442917 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 99354.4us
I20260709 12:57:24.276595 16308 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.448110 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.548386 16417 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.451757 16296 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:24.453709 13441 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 3, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:24.261643 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.283978 16360 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.455598 16286 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:24.461462 13693 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:24.271629 16215 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:24.519021 14237 rpcz_store.cc:269] 0709 12:57:22.836286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.847313 (+ 11027us) service_pool.cc:224] Handling call
0709 12:57:24.198415 (+1351102us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54005f1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:24.198523 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1216301952}
I20260709 12:57:24.520845 16422 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:24.520983 16372 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.507484 16199 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.526268 14238 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:24.526922 16181 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:24.572750 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.527770 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1476) took 1050 ms. Trace:
I20260709 12:57:24.528112 13543 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:24.606989 16372 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:24.607931 16309 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.567252 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:24.608533 15958 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.609464 16353 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.610162 14245 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:24.610394 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:24.561164 13691 rpcz_store.cc:276] 0709 12:57:22.819514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.819666 (+   152us) service_pool.cc:224] Handling call
0709 12:57:22.871987 (+ 52321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:22.922079 (+ 50092us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:22.942786 (+ 20707us) log.cc:844] Serialized 14 byte log entry
0709 12:57:23.133142 (+190356us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:23.150568 (+ 17426us) log.cc:844] Serialized 52 byte log entry
0709 12:57:23.150732 (+   164us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:23.150981 (+   249us) log.cc:844] Serialized 14 byte log entry
0709 12:57:23.498672 (+347691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:23.498773 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.390412 (+891639us) raft_consensus.cc:1685] finished
0709 12:57:24.390470 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.390825 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59620,"prepare.queue_time_us":33777,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":12633,"raft.queue_time_us":430976,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":77472,"thread_start_us":464544,"threads_started":2,"wal-append.queue_time_us":270091}
I20260709 12:57:24.611186 15958 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:24.569517 16203 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.612402 16203 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:24.612762 16181 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.612914 16203 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:24.613409 16181 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.614233 16203 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.569347 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:24.449991 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:24.573100 15789 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:24.575369 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:24.576035 15865 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.211s	user 0.004s	sys 0.017s
I20260709 12:57:24.576524 16225 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:24.636837 16450 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:57:24.577474 15663 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.577833 16308 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.564355 13536 rpcz_store.cc:276] 0709 12:57:23.013674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.847153 (+833479us) service_pool.cc:224] Handling call
0709 12:57:24.045953 (+198800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.046167 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:24.152440 (+106273us) log.cc:844] Serialized 14 byte log entry
0709 12:57:24.409223 (+256783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:24.409325 (+   102us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:24.409432 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.409533 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.409900 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102312,"prepare.queue_time_us":154580,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4214,"raft.queue_time_us":39145,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":166,"thread_start_us":216735,"threads_started":1,"wal-append.queue_time_us":217085}
I20260709 12:57:24.580624 16318 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:24.581403 16311 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.582324 16369 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:24.582497 16191 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.585373 16226 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:24.527472 16357 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:24.590360 13619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 438) took 1706 ms (client timeout 1681 ms). Trace:
I20260709 12:57:24.606105 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.606467 13690 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 12:57:24.565090 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 66704.4us
I20260709 12:57:24.615841 16181 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.610241 16199 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:24.624305 13615 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:57:24.625006 16360 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:24.626101 14212 leader_election.cc:336] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.681s (SENT)
I20260709 12:57:24.626261 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.627259 16144 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:24.626969 13441 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:57:24.626973 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 205200us
I20260709 12:57:24.633960 16215 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.634379 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2541) took 1262 ms. Trace:
I20260709 12:57:24.634660 13611 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:24.635299 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 185616us
W20260709 12:57:24.655257 13619 rpcz_store.cc:269] 0709 12:57:22.883900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.890923 (+  7023us) service_pool.cc:224] Handling call
0709 12:57:24.590002 (+1699079us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540020ec54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:24.590207 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3770623744}
W20260709 12:57:24.635442 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 60235.8us
I20260709 12:57:24.635540 16417 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:24.607728 13614 rpcz_store.cc:276] 0709 12:57:23.476963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:24.062450 (+585487us) service_pool.cc:224] Handling call
0709 12:57:24.143883 (+ 81433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:24.144384 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:24.202305 (+ 57921us) log.cc:844] Serialized 14 byte log entry
0709 12:57:24.283377 (+ 81072us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b18002b8dc0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:24.472535 (+189158us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:24.484971 (+ 12436us) log.cc:844] Serialized 26 byte log entry
0709 12:57:24.485236 (+   265us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:24.485270 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.485398 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.527192 (+ 41794us) raft_consensus.cc:1685] finished
0709 12:57:24.527321 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.527556 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163747,"prepare.queue_time_us":57690,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":195,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":46373,"spinlock_wait_cycles":6287232,"thread_start_us":209666,"threads_started":1,"wal-append.queue_time_us":303061}
I20260709 12:57:24.636044 15865 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.587363 16185 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.636004 15789 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:24.656162 13690 pending_rounds.cc:85] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 5
W20260709 12:57:24.670684 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 59260.9us
I20260709 12:57:24.664232 13618 rpcz_store.cc:276] 0709 12:57:23.371501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.737627 (+366126us) service_pool.cc:224] Handling call
0709 12:57:24.190054 (+452427us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:24.190224 (+   170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.568519 (+378295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:24.568578 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:24.568665 (+    87us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:24.568754 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.633978 (+ 65224us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b540012a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:24.634031 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.634190 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889886848}
I20260709 12:57:24.677567 13690 pending_rounds.cc:107] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:57:24.674825 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.685444 13690 raft_consensus.cc:2889] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:57:24.683605 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 114380us
I20260709 12:57:24.669523 16417 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:24.669078 15958 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:24.659624 16422 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:24.693109 14237 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:24.682658 16212 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.697053 13441 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:24.696264 16417 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:24.715111 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:24.716056 16308 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:24.716349 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:24.716501 16308 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:24.716948 16308 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:24.717447 16410 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:24.720161 13658 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.721241 16410 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:24.721817 16410 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:24.694977 15958 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:24.723788 15958 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:24.724370 16410 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.721777 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:24.704509 13508 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:24.731906 16226 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:24.741444 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.745911 16221 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:57:24.746459 16286 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.700680 16311 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.730710 16191 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:24.695504 16369 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:24.746533 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.738381 16144 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.741489 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:24.753505 16404 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:24.754066 16404 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:24.754297 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:24.754521 16404 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:24.755141 14241 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:24.744756 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:24.761860 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 57463.9us
I20260709 12:57:24.701694 16357 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.771873 13658 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.712545 16181 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 12:57:24.776154 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.782065 15663 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:24.783248 13509 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.754079 14238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 538) took 1695 ms. Trace:
W20260709 12:57:24.785290 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.753539 16286 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:24.790589 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61458.2us
W20260709 12:57:24.789870 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 164013us
I20260709 12:57:24.795775 15663 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:24.796924 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.797654 16408 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:24.798381 16362 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:24.799554 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2537) took 1451 ms. Trace:
I20260709 12:57:24.795663 16166 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:24.800272 16166 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:24.800681 16166 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:24.801046 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:24.802266 15968 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.802840 15968 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.804761 16148 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:24.805176 16148 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:24.805215 15968 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.805670 16148 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:24.798406 16408 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:24.808246 15865 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.808115 13611 rpcz_store.cc:276] 0709 12:57:23.347982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.492611 (+144629us) service_pool.cc:224] Handling call
0709 12:57:24.203540 (+710929us) spinlock_profiling.cc:243] Waited 8.79 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:24.203733 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:24.689007 (+485274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.799233 (+110226us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400333bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:24.799460 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010372864}
I20260709 12:57:24.810674 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:24.786706 14238 rpcz_store.cc:276] 0709 12:57:23.058968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.431632 (+372664us) service_pool.cc:224] Handling call
0709 12:57:24.753833 (+1322201us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54006d15d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:24.754011 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983802240}
I20260709 12:57:24.812556 13540 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:24.814999 14238 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:24.815361 13541 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:24.816480 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:24.816632 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:24.817338 13692 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:24.818100 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:24.818859 13693 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:24.820538 14245 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 859) took 1860 ms (client timeout 2112 ms). Trace:
W20260709 12:57:24.816748 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 60903.5us
I20260709 12:57:24.808200 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:24.783855 13441 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 5, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:24.780606 16221 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:24.832569 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 49460.4us
W20260709 12:57:24.820937 14245 rpcz_store.cc:269] 0709 12:57:22.960221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:22.960395 (+   174us) service_pool.cc:224] Handling call
0709 12:57:24.820301 (+1859906us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540069e954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:24.820483 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3790555392}
I20260709 12:57:24.797143 16304 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:24.773882 16144 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.834511 14237 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:24.839331 16396 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:24.841079 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:24.874207 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:24.875401 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2533) took 1531 ms. Trace:
I20260709 12:57:24.875864 13615 rpcz_store.cc:276] 0709 12:57:23.343508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.441610 (+ 98102us) service_pool.cc:224] Handling call
0709 12:57:23.442033 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:24.674659 (+1232626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.875070 (+200411us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002b4154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:24.875328 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540949248}
I20260709 12:57:24.876510 16357 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.877110 16357 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:24.877892 16372 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.883639 14244 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:24.883965 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.884958 16404 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.887506 16404 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.890048 16144 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:24.843271 16213 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:24.844924 13441 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 6, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 6 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:24.847373 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:24.892306 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 58643.9us
W20260709 12:57:24.852293 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 124469us
I20260709 12:57:24.852641 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:24.852829 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:24.853399 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:24.853585 16304 raft_consensus.cc:697] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:24.852317 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:24.808269 16408 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:24.858647 16286 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:24.858796 13658 connection.cc:256] RPC call timeout handler was delayed by -1.3138789000004181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 12:57:24.898592 13658 connection.cc:256] RPC call timeout handler was delayed by -1.1608789000001707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 12:57:24.899052 13658 connection.cc:256] RPC call timeout handler was delayed by -1.0825789000000441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 12:57:24.859508 16166 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.860711 16311 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:24.864151 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2534) took 1492 ms. Trace:
W20260709 12:57:24.895560 16455 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.210s	user 0.004s	sys 0.000s
I20260709 12:57:24.865253 16155 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:24.866338 16212 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:24.866863 16369 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.893690 14241 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
W20260709 12:57:24.903012 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 65936us
I20260709 12:57:24.864697 16418 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:24.821159 16148 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.915108 13540 raft_consensus.cc:2410] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:57:24.923823 16286 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.924124 16155 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.925050 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:24.926187 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.926286 16286 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:24.928645 13689 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:24.929338 16148 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.930336 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2538) took 1475 ms. Trace:
I20260709 12:57:24.924358 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.923678 13441 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 4, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:24.946882 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:24.955369 13506 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:24.905524 16404 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 12:57:24.963603 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53383.3us
I20260709 12:57:24.922971 16414 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.965611 16414 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:24.966836 15789 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:24.969247 14211 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:24.970067 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:24.970335 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 53633.6us
I20260709 12:57:24.971148 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:24.971650 16172 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:24.972172 16360 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:24.972607 16360 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:24.973002 16360 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:24.974347 16353 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:24.926309 16155 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:24.906020 13616 rpcz_store.cc:276] 0709 12:57:23.343802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.527137 (+183335us) service_pool.cc:224] Handling call
0709 12:57:24.191802 (+664665us) spinlock_profiling.cc:243] Waited 8.59 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:24.191981 (+   179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:24.205832 (+ 13851us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:24.522961 (+317129us) log.cc:844] Serialized 14 byte log entry
0709 12:57:24.712991 (+190030us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:24.753090 (+ 40099us) log.cc:844] Serialized 26 byte log entry
0709 12:57:24.771804 (+ 18714us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:24.772036 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.775876 (+  3840us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.825999 (+ 50123us) raft_consensus.cc:1685] finished
0709 12:57:24.835718 (+  9719us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.835939 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36800,"prepare.queue_time_us":250865,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":96,"raft.queue_time_us":33146,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":43410,"spinlock_wait_cycles":19763584,"thread_start_us":197364,"threads_started":2,"wal-append.queue_time_us":150085}
I20260709 12:57:24.977643 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:24.971225 16371 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.944634 13612 rpcz_store.cc:276] 0709 12:57:23.361780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.717285 (+355505us) service_pool.cc:224] Handling call
0709 12:57:24.190906 (+473621us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:24.191071 (+   165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:24.202097 (+ 11026us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:24.466188 (+264091us) log.cc:844] Serialized 14 byte log entry
0709 12:57:24.579178 (+112990us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b18003becc0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:24.704793 (+125615us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:24.784545 (+ 79752us) log.cc:844] Serialized 26 byte log entry
0709 12:57:24.789335 (+  4790us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:24.789375 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.823162 (+ 33787us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:24.823263 (+   101us) raft_consensus.cc:1685] finished
0709 12:57:24.823276 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:24.835358 (+ 12082us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13404,"prepare.queue_time_us":248409,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":32937,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":14046,"spinlock_wait_cycles":36448256,"thread_start_us":385998,"threads_started":2,"wal-append.queue_time_us":222616}
W20260709 12:57:24.979665 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 69324.9us
I20260709 12:57:24.961031 16148 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:24.996562 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 72285.6us
I20260709 12:57:24.963840 16396 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:24.981653 16172 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:25.006093 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:25.006620 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:24.979446 16369 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.990573 16288 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:24.994884 16148 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:25.001864 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:25.012049 13689 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:57:25.013115 13441 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 4, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:25.017764 16398 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:24.997606 16414 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.020231 16281 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:25.011358 16185 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:25.024276 16369 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.025486 16288 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.046510 16172 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.048388 16288 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.015179 16148 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:25.049463 16335 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:25.051393 15789 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.026633 13684 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:25.052352 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:25.052963 16308 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.043295 16155 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:25.019500 16418 consensus_queue.cc:1059] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:57:25.055680 16372 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:25.056493 16372 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:25.056820 13541 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:57:25.058087 13536 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:25.058557 16418 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:25.059581 16336 raft_consensus.cc:3055] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:25.059980 16336 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:25.060527 16369 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.019033 16304 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.061101 16336 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.062398 16336 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:25.064028 16181 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:25.065963 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:25.044940 16396 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.037063 13441 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 4, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:24.997316 16371 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.074250 13684 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:25.081347 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.082526 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.078574 13441 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:25.085892 13508 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:25.083056 13688 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:25.083850 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:25.089289 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.040336 16281 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:25.055739 16308 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.093609 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:25.105069 13686 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:25.088982 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.105870 13691 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:25.016765 16417 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.109169 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.111339 13540 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:25.111403 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:25.112269 15789 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:25.112844 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 101957us
I20260709 12:57:25.109683 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:25.108184 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 63833.7us
I20260709 12:57:25.123006 13687 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:57:25.074040 16398 consensus_queue.cc:1059] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:57:25.102026 16281 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:25.132634 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:25.138633 14242 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:25.137679 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:25.109364 16335 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:25.159443 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 45864.2us
I20260709 12:57:25.147383 13441 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:25.121987 16166 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:25.162353 13582 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.721s (SENT) [suppressed 27 similar messages]
I20260709 12:57:25.165421 13541 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:25.112851 15789 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.178814 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:25.156427 16335 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:25.182142 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:25.182777 14243 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:25.187147 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.182334 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:25.187094 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.159003 16369 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.185374 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:25.194092 14238 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:25.194270 13684 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:25.204263 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:25.205106 13616 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:25.210423 16166 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.211782 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.191080 16203 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:25.193135 14245 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:25.211093 16166 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.183110 16335 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:25.229774 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 80453.3us
I20260709 12:57:25.230873 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:25.220643 13441 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 4, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:25.217443 16417 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.334007 13687 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
W20260709 12:57:25.260655 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:25.394239 16404 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.394934 16404 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.396122 13686 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:25.396894 16185 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:25.397133 16404 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.400118 16369 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.402300 16289 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.402452 16369 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.402881 16289 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.403800 13692 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:25.404256 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.404755 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:25.405041 16289 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.405300 13692 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:25.405360 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.406502 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:25.406494 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:25.407096 13686 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:25.407477 16289 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:25.412067 14243 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:25.415062 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:25.416579 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.416666 13441 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 5, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 5 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:25.419787 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:25.419661 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:25.420913 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:25.421936 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:25.428437 13685 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:25.413815 16215 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.438052 13612 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:25.438192 13684 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:25.442130 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:25.421509 16191 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.433168 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:25.421700 15946 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:25.451282 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.441533 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.424652 16357 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:25.448982 16213 consensus_meta.cc:287] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.004s
I20260709 12:57:25.453001 13686 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:25.449982 13584 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:25.477838 13508 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:25.454929 13441 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:25.471451 13687 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:57:25.458782 13684 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:25.479483 13687 pending_rounds.cc:85] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 4
I20260709 12:57:25.480027 13687 pending_rounds.cc:107] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:57:25.480268 13687 raft_consensus.cc:2889] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:25.485754 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.487746 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:25.488672 13686 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:25.427445 16203 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.487617 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.490375 13441 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 4, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 4 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:25.458278 16288 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:25.437589 16148 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.468459 16185 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:25.485293 16172 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.452687 16215 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.508514 13613 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:25.528267 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.484211 16369 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.468415 16191 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.529449 13689 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:57:25.514164 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 121662us
I20260709 12:57:25.540177 13684 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:25.555619 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.556766 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.557401 13684 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:25.540290 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:25.541145 16477 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 12:57:25.511629 16480 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:25.541785 16335 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.542994 16398 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:25.544564 16203 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.546993 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.547211 15789 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.561563 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:57:25.541883 16479 log.cc:927] Time spent T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:57:25.567384 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.483752 16357 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.572626 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.534161 16215 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.551868 16289 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:25.584493 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.589483 14242 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
W20260709 12:57:25.582399 16482 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:57:25.597317 13684 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:25.490119 15946 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.572850 16480 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:25.614419 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:25.582957 15958 consensus_meta.cc:287] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.859s	user 0.013s	sys 0.002s
I20260709 12:57:25.580286 16336 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:25.607556 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.610802 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:25.609910 13618 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:25.571789 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.601286 13692 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:25.602669 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:25.565475 16369 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.621505 13689 raft_consensus.cc:2410] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:57:25.617547 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.619566 14242 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:25.620399 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 91721.7us
I20260709 12:57:25.618721 13612 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:25.622056 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:25.622231 13610 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:25.609820 16166 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.628055 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:25.616532 16480 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:25.630700 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:25.621315 16215 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:25.637104 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:25.641203 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 410416us
I20260709 12:57:25.644025 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.611301 16172 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.654757 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:25.619326 15958 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.658887 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.662220 13617 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:57:25.662228 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:25.662734 16357 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.631632 16336 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:25.663077 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:25.664786 16488 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:57:25.665376 16336 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.667287 13538 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:57:25.662642 13617 pending_rounds.cc:85] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 5
I20260709 12:57:25.668512 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:25.670578 13617 pending_rounds.cc:107] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:57:25.670936 13617 raft_consensus.cc:2889] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:25.672792 16191 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.673055 13614 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:25.674221 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:25.674788 13614 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:25.648666 16203 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:25.686833 13534 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:25.643179 16288 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:25.651465 16212 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:25.694810 16148 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.701256 16335 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.702085 16212 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.703732 16286 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.704416 16286 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.704939 16212 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.707386 16286 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.711236 16369 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.713028 15946 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.713932 16481 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:25.714371 16481 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:25.717197 15958 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:25.722275 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:25.722980 13615 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:25.700263 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 68905.7us
I20260709 12:57:25.655776 16215 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:25.715248 16481 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.793879 13618 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 12:57:25.728886 16445 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:57:25.801102 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 68327.4us
I20260709 12:57:25.803350 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:25.800657 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:25.800532 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:25.806407 13534 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:25.729492 15958 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:25.809605 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:25.732203 16398 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:25.807229 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:25.804175 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:25.824461 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:25.828543 16286 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:25.829203 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:25.830148 16286 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:25.832126 13542 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:57:25.804760 16481 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:25.834054 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:25.822788 16281 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.002s
I20260709 12:57:25.804425 15946 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.833971 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:25.846467 16286 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:25.845366 16372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.004s	sys 0.030s
I20260709 12:57:25.854292 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:25.823632 15958 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.857002 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:25.854306 16372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.004s	sys 0.031s
I20260709 12:57:25.860659 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:25.858680 16360 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.885s	user 0.009s	sys 0.005s
I20260709 12:57:25.841924 16281 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.843050 15946 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:25.833671 16369 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:57:25.884035 15958 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:25.884989 16172 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:25.888060 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:25.863117 16185 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:25.896000 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:25.910797 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:25.911548 13618 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:25.897060 16480 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.898092 13534 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:25.891713 16398 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:25.905387 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 198929us
I20260709 12:57:25.890347 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:25.919809 13540 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:25.845167 16288 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:25.927438 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:25.892454 16360 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:25.943907 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 55925.5us
I20260709 12:57:25.944990 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:25.921972 16286 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:25.951956 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 56061.2us
I20260709 12:57:25.957187 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:25.957863 14245 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:25.875983 16215 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:25.970745 14241 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:25.930155 16172 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:25.974303 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:25.930524 16212 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:57:25.958381 16483 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:57:26.018222 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.019311 13613 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:26.020331 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.020942 13613 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:26.015345 16172 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.032912 15946 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.045642 16281 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.013001 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:25.965296 16372 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:26.052276 16372 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:26.052845 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:26.054397 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:26.051343 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 49578.6us
I20260709 12:57:25.972491 16498 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:26.056921 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 194900us
W20260709 12:57:26.059242 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 85055.1us
W20260709 12:57:26.063424 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 50521.2us
I20260709 12:57:26.063727 13615 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:26.064424 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.065531 13509 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:26.065532 14212 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.068614 14245 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:26.073730 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.079998 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.052845 16372 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:26.090607 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:26.062847 16281 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:26.071384 16191 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:26.114378 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.078550 16185 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:26.082542 16484 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:26.116214 13616 raft_consensus.cc:2410] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:26.071254 16335 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:26.068238 16212 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:26.114282 16286 raft_consensus.cc:2764] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:57:26.123673 16360 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:26.124673 16510 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:26.123710 16509 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:26.117353 16191 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:26.118602 16185 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:26.118451 16484 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:26.132833 16483 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:57:26.133559 13618 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:26.085549 16172 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:26.136341 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 63336.9us
I20260709 12:57:26.092442 16497 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:26.068755 16414 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:57:26.128901 16480 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.141479 15946 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:26.090091 16498 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.142871 16191 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:26.087880 16148 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:57:26.142724 16185 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:26.143692 16484 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:26.137938 16281 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.147346 16497 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:26.155784 16509 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.185751 16509 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.156205 16286 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:57:26.175702 16497 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:26.212627 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 112385us
I20260709 12:57:26.173228 15946 raft_consensus.cc:697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:26.161113 16408 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:57:26.220783 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.223047 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:26.180191 16203 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:57:26.225656 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:26.226285 13614 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:26.228133 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.232036 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.235548 13618 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:26.249993 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 450) took 1041 ms. Trace:
W20260709 12:57:26.249255 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 175618us
I20260709 12:57:26.253083 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:26.253113 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:26.244887 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 124643us
I20260709 12:57:26.225028 16318 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:57:26.232560 16418 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:26.261121 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.263732 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:26.251199 13611 rpcz_store.cc:276] 0709 12:57:25.208985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:25.229061 (+ 20076us) service_pool.cc:224] Handling call
0709 12:57:25.954365 (+725304us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:25.954584 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:25.954875 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:25.955220 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 12:57:26.249617 (+294397us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:26.249677 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:26.249741 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:26.249814 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:26.249932 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3087,"prepare.queue_time_us":1134,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"spinlock_wait_cycles":49271168,"thread_start_us":274782,"threads_started":1}
I20260709 12:57:26.269464 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.276357 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 445) took 2076 ms. Trace:
I20260709 12:57:26.276849 13617 rpcz_store.cc:276] 0709 12:57:24.200146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:24.559107 (+358961us) service_pool.cc:224] Handling call
0709 12:57:24.559530 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:25.671490 (+1111960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:26.276037 (+604547us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001293d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:26.276287 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2471085952}
I20260709 12:57:26.280426 16512 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:26.282943 16512 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:26.283349 16512 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:26.286294 13615 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:57:26.286731 13614 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:26.283730 16512 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:26.291498 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:26.295508 14245 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:26.299896 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:26.301407 14243 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
W20260709 12:57:26.289326 16513 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:57:26.272555 16335 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:26.271382 16418 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:26.284348 16414 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:26.316102 16414 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:26.309489 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:26.317684 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 89621.4us
I20260709 12:57:26.270213 15946 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.311651 16335 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:26.314198 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:26.284101 16213 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:57:26.314882 16418 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.308943 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:26.317931 14244 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:26.317242 16414 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.334088 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.337401 13584 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:26.342528 16481 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.001s
I20260709 12:57:26.342128 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:26.347126 16508 log.cc:927] Time spent T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:57:26.348356 16509 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:26.348819 16509 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:26.331990 16335 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:26.349305 16509 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:26.351944 16360 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 12:57:26.354039 16475 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.868s	user 0.004s	sys 0.011s
I20260709 12:57:26.341682 16166 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:26.355077 16172 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:26.354854 16185 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.355700 16172 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:26.356824 16172 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.356851 16371 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.320164 15958 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:57:26.366353 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:26.336018 16215 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:26.405799 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:26.406704 13687 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:26.354892 16166 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.408044 16360 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:26.413064 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:26.418289 13691 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:26.431066 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:26.422550 16510 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:26.381436 16148 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:26.432293 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:26.431795 13690 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:26.433076 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:26.374519 16371 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:26.445604 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 69470.6us
I20260709 12:57:26.446341 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:26.451294 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:26.452620 14242 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:26.440800 16476 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.932s	user 0.002s	sys 0.002s
I20260709 12:57:26.449777 16371 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.438849 16512 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:26.479133 16520 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:57:26.479417 16512 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:26.437872 16510 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:26.428234 16360 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.480921 16510 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.482374 16360 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:26.488843 16514 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:57:26.489482 16521 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:57:26.489799 13687 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
W20260709 12:57:26.492450 16517 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:57:26.494601 16281 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.437832 16511 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:26.495478 16511 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.497862 16511 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:26.498559 16191 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.499783 14246 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:26.477671 16371 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.501544 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:26.503602 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2746) took 1354 ms. Trace:
I20260709 12:57:26.504014 13688 rpcz_store.cc:276] 0709 12:57:25.149676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:25.149835 (+   159us) service_pool.cc:224] Handling call
0709 12:57:25.170210 (+ 20375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:25.180497 (+ 10287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:25.180528 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:25.185842 (+  5314us) log.cc:844] Serialized 26 byte log entry
0709 12:57:25.725435 (+539593us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b18000b4580. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:26.222507 (+497072us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:26.222547 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:26.222623 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:26.503300 (+280677us) raft_consensus.cc:1685] finished
0709 12:57:26.503320 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:26.503542 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9821,"raft.queue_time_us":8077,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6907,"spinlock_wait_cycles":31502592,"thread_start_us":496888,"threads_started":1,"wal-append.queue_time_us":1080539}
I20260709 12:57:26.507513 16166 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:26.508939 16185 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.512490 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 875) took 1531 ms. Trace:
W20260709 12:57:26.514034 16519 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.150s	user 0.001s	sys 0.001s
I20260709 12:57:26.513129 14239 rpcz_store.cc:276] 0709 12:57:24.981110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:25.022842 (+ 41732us) service_pool.cc:224] Handling call
0709 12:57:25.023194 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:25.023631 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:25.023661 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:25.024218 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:57:25.224232 (+200014us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:25.224270 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:25.224331 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:26.266829 (+1042498us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:26.511989 (+245160us) raft_consensus.cc:1685] finished
0709 12:57:26.512017 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:26.512303 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1135,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":38489856,"thread_start_us":199565,"threads_started":1,"wal-append.queue_time_us":400718}
I20260709 12:57:26.514199 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:26.515709 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:26.520337 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:26.521037 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:26.521485 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:26.501700 16191 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.534618 14209 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:26.531512 16497 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.545755 14238 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:26.535526 16286 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:26.555143 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:26.533839 16185 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:26.524166 15958 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:57:26.560992 16360 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:26.575199 16360 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:26.576256 16524 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:26.576206 16360 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.546224 16522 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:26.560673 16536 log.cc:927] Time spent T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:57:26.560230 16498 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.598589 16484 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:26.601136 16484 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.566193 16372 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.566704 16509 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.561740 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:26.580310 16286 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:26.560276 16512 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:57:26.577476 16185 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:26.585438 16422 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:26.590548 16408 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:26.591288 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:26.595039 16335 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.624110 14237 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEARNER]: Advancing to term 6
I20260709 12:57:26.620164 16286 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:26.624727 16185 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:26.630312 16472 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.462s	user 0.003s	sys 0.003s
W20260709 12:57:26.636499 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 299183us
W20260709 12:57:26.645231 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 83578.5us
I20260709 12:57:26.628367 16318 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:26.630681 16509 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:26.686748 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:26.642731 16213 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:26.714820 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:26.627959 16408 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.629572 16422 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.722345 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:26.718211 16215 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:57:26.714900 16510 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:57:26.721015 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 85282.6us
I20260709 12:57:26.725297 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.734040 13617 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:57:26.721356 16539 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:57:26.723527 16509 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:26.719825 16318 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:26.742091 16335 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:26.742524 16527 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 12:57:26.743860 16527 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:26.744266 16527 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:26.745239 16527 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:26.747933 16468 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.572s	user 0.000s	sys 0.003s
I20260709 12:57:26.749327 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1360) took 1931 ms. Trace:
I20260709 12:57:26.749706 13535 rpcz_store.cc:276] 0709 12:57:24.817801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:24.818806 (+  1005us) service_pool.cc:224] Handling call
0709 12:57:24.819143 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:24.861332 (+ 42189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:24.861365 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:24.861974 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 12:57:24.922403 (+ 60429us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:24.922450 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:24.922519 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:26.740401 (+1817882us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:26.749049 (+  8648us) raft_consensus.cc:1685] finished
0709 12:57:26.749069 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:26.749270 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72343,"raft.queue_time_us":43004,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7173,"spinlock_wait_cycles":30871936,"thread_start_us":59833,"threads_started":1,"wal-append.queue_time_us":241180}
W20260709 12:57:26.752424 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:26.753729 16509 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:26.754479 16509 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:26.730053 16510 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:26.754832 16509 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:26.755132 16510 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:26.755766 16509 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.757403 16509 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:26.759305 16289 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:26.759807 16289 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:26.708069 16524 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:57:26.774453 13611 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:57:26.777283 16527 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:26.781589 13615 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:26.781562 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:26.782902 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.783063 13615 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:26.784078 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:26.784314 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.784566 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.784781 13614 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:26.785542 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:26.786326 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:26.787077 14243 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:26.784094 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:26.788162 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:26.789249 16408 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:26.791148 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:26.810536 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 80160.5us
I20260709 12:57:26.794539 14211 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:26.727259 16512 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:57:26.871515 13614 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:26.791716 13538 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 4.
I20260709 12:57:26.890204 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:26.890547 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:26.891132 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 118526us
I20260709 12:57:26.892192 16191 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:26.890302 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:26.893963 16289 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.895318 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:26.896335 16372 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:26.898468 13613 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:26.907546 13540 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:26.906849 13539 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:26.902275 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.903117 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:26.903821 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:26.908622 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:26.910671 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:26.908816 13535 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:26.918293 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:26.916867 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:26.921939 13543 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:57:26.915426 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:26.927644 13538 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:26.900183 16491 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:26.898187 15968 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:26.933073 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:26.911800 16488 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.369s	user 0.003s	sys 0.000s
I20260709 12:57:26.904923 16545 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:26.930683 13541 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:26.935297 13617 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:26.935770 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:26.768836 16318 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.940253 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:26.906888 16148 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:26.951337 13540 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:26.944634 13542 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:26.946702 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:26.931957 16191 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:26.932577 15789 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:26.959169 13541 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:57:26.961957 16318 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:26.942137 15968 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:26.964426 16551 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:26.968827 16546 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:57:26.974153 16185 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:26.975993 16422 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:26.983712 13613 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:26.984587 13538 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:26.940853 16545 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:26.990154 14237 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:26.980571 16148 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.002154 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.003046 13614 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:27.005100 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.006222 13541 raft_consensus.cc:2410] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:57:26.993044 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 270772us
I20260709 12:57:26.995110 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:27.009411 14243 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:27.013240 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.015661 13535 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:27.017702 13539 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:27.013409 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:26.941028 16491 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.040941 13536 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:27.041656 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:27.042467 13612 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:27.044914 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.047155 13535 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:57:27.052037 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 133912us
I20260709 12:57:27.052757 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.056795 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:27.057668 13613 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:27.059214 13616 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:27.067171 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.071839 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.072820 13615 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:27.054979 14240 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:57:27.075716 13534 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:27.080576 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.037642 16166 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:27.081895 13690 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:27.087163 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:27.082916 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.086814 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.090863 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.092077 13693 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:27.094172 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.098356 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.100162 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.096282 13506 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:26.997262 16551 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:27.097396 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.097503 16166 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.107134 13615 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:57:27.109031 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.109087 15968 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.110837 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.120726 13686 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:27.038154 16215 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:57:27.043849 16369 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:57:27.084903 16372 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:27.128579 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 182659us
I20260709 12:57:27.126642 13613 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:27.132870 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 43947.7us
I20260709 12:57:27.140323 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.156425 13618 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:27.135767 13541 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:57:27.158550 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.159775 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.159829 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.160413 13688 raft_consensus.cc:2376] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8. Re-sending same reply.
I20260709 12:57:27.161135 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.161942 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.162321 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:27.162557 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:27.164374 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.165418 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.167963 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:27.172392 13615 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:27.173305 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.172494 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.174363 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.174810 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:27.166522 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:27.136806 16549 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:57:27.128314 16213 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:27.104836 16357 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:57:27.130812 15968 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:27.206452 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.181198 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.173552 16289 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.183590 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.209067 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.210258 13613 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:27.182336 13541 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:27.105407 16191 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.184696 16484 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:27.214578 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.216738 13535 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:27.157743 16372 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.214016 16185 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.186187 16288 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:27.227077 16552 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:57:27.227442 16286 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.003s
I20260709 12:57:27.227845 16286 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.230063 16286 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:27.229904 16561 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:27.235028 16565 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:27.233239 16335 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:57:27.238673 16554 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:57:27.240409 16556 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:57:27.244455 16491 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.246085 16166 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:27.118031 16545 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.254843 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:27.267234 13536 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:27.265013 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:27.273940 16286 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:27.280771 16286 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:27.281656 16286 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.272416 16369 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
W20260709 12:57:27.283066 16510 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.002s
I20260709 12:57:27.252307 16288 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.272558 13541 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:27.284076 16148 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.286166 16288 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.286329 16372 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.235575 16565 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.253549 16422 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:27.290158 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.283524 16510 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.296976 13616 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:27.298667 16551 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:27.300546 16510 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:27.280145 16496 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:27.301149 16496 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:27.301525 16496 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:27.302343 13537 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:27.290810 13612 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:27.304239 16289 raft_consensus.cc:2764] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:57:27.305182 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.305756 13687 raft_consensus.cc:2376] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8. Re-sending same reply.
I20260709 12:57:27.322758 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.327147 13537 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:27.294139 16545 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:27.294431 16422 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:27.330834 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:27.332294 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 147475us
I20260709 12:57:27.332446 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.336473 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.336601 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.352089 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:27.352993 14241 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:27.345651 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.358008 14211 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:27.362298 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:27.368310 16360 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:57:27.364035 14211 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.363484 15958 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:27.386716 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.337885 16422 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:27.371518 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:27.372992 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.385056 16318 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:27.334138 16510 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:27.321388 16288 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:57:27.329416 16512 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:57:27.388588 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:27.363641 16551 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:27.404016 16551 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:27.404537 16550 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:27.405602 16551 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:27.406093 16551 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:27.406948 16551 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.331488 16369 consensus_queue.cc:1059] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:57:27.408001 16522 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:27.408380 16506 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:27.408715 16522 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:27.408984 16512 consensus_queue.cc:1243] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:57:27.330746 16357 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:57:27.397284 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.395962 13686 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:57:27.371213 16212 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:27.398800 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:27.411708 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 202771us
I20260709 12:57:27.415189 16212 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:27.410734 16289 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:27.425671 13686 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:27.431536 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.438091 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.440330 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.441071 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.442669 13538 pending_rounds.cc:85] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 2
I20260709 12:57:27.443009 13538 pending_rounds.cc:107] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:57:27.443284 13538 raft_consensus.cc:2889] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:27.444917 16491 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:27.445528 16565 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:27.446660 15946 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:27.412153 16522 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.448392 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.450655 13543 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:27.418610 16289 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:27.451614 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.452353 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.452301 16289 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.453073 13537 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:27.453308 14245 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:27.453588 16289 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:27.454655 14245 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:27.456278 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.457052 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.456911 13688 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:27.459748 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:27.463536 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.476011 14238 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:27.476275 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.413511 16288 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:27.418552 16212 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:27.482146 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:27.484297 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 58991.4us
I20260709 12:57:27.485908 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:27.484300 13686 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:57:27.488005 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.458379 15958 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:27.491793 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.492113 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.494455 14209 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:27.495285 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:27.499287 14241 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:57:27.499868 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:27.556015 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 155427us
I20260709 12:57:27.479578 15946 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:27.573133 14237 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
W20260709 12:57:27.519533 16572 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:57:27.475291 16491 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:27.506474 13686 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:57:27.594395 13535 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:27.596807 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.600332 13686 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:27.552223 15958 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:27.520880 16480 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:27.518039 16579 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:57:27.540005 13582 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:27.603825 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:27.609524 13535 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:27.492689 16288 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:27.611518 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 119821us
I20260709 12:57:27.504320 14238 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:27.615108 13658 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:27.613767 16497 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.579803 16408 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:57:27.600881 16589 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:27.519970 16568 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 12:57:27.591641 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:27.601878 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.626014 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:27.626832 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 86949.9us
I20260709 12:57:27.636616 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.636858 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.520778 16569 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 12:57:27.638283 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.639003 13686 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:27.645824 16582 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:57:27.646400 16571 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:27.625340 16585 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:27.647544 16585 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:27.647893 16585 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:27.628001 16357 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:27.656404 16571 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:27.656533 16357 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.657312 16571 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:27.658977 16357 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:27.660781 16288 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:27.662853 16491 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.664357 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.664495 16593 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:27.665434 13686 raft_consensus.cc:2410] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:57:27.668767 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.617666 15958 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:57:27.688269 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.689524 13535 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:27.644317 16551 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:27.656458 16369 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:27.698282 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 72326.4us
W20260709 12:57:27.698352 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.696326 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:27.700083 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 61878us
I20260709 12:57:27.701999 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:27.703680 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 341442us
I20260709 12:57:27.711709 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.713471 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.697343 16593 consensus_queue.cc:1243] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:57:27.683183 16422 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.698352 16551 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:27.718686 13535 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:27.722581 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.725776 16408 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:27.726305 16408 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:27.727329 16408 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.728787 16408 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:27.722283 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 227866us
I20260709 12:57:27.731171 13584 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:27.684829 16569 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:57:27.736486 16496 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.736894 16191 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:27.737507 16191 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.738930 16496 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.740110 14241 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:27.740913 16601 consensus_queue.cc:1059] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:27.741848 16589 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:27.742074 16588 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:27.742465 16589 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.742703 16588 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.744740 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:27.746974 16592 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:57:27.749768 16510 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:27.750284 16510 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:27.754498 16422 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.755477 13534 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
W20260709 12:57:27.756450 16527 consensus_meta.cc:287] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.979s	user 0.010s	sys 0.002s
I20260709 12:57:27.757156 16527 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:27.745966 16589 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.731848 16551 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:27.771287 16597 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:27.772538 16545 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:27.773348 16604 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:27.733345 16593 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:27.773959 16155 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:27.776983 16588 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:27.793584 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 94291.6us
I20260709 12:57:27.794839 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:27.793179 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:27.799100 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:27.768879 16369 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:27.799569 13686 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:27.802992 13619 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:27.775092 16585 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:27.797241 13688 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:27.805476 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1481) took 1025 ms. Trace:
W20260709 12:57:27.806686 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.810948 13658 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:27.808483 13619 rpcz_store.cc:276] 0709 12:57:26.780193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.784004 (+  3811us) service_pool.cc:224] Handling call
0709 12:57:27.802682 (+1018678us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b5800126098. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:27.802838 (+   156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:27.803474 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:27.805429 (+  1955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19863680}
I20260709 12:57:27.741951 16584 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 12:57:27.821834 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.804819 16602 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:27.794842 16604 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:27.823603 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.830750 13689 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:27.832613 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:27.833911 13686 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:27.835547 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.813841 16509 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:27.831059 13542 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:27.840950 16569 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:27.841919 16604 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:27.807339 16603 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:27.819893 16288 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:27.851722 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:27.798481 16545 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:27.808529 16496 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:57:27.909437 13541 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:57:27.844750 16602 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:27.844187 16191 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:27.845719 16569 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:27.909818 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:27.917963 13543 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:27.829618 16212 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:27.914896 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 98838.5us
I20260709 12:57:27.915033 13541 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:57:27.857414 16509 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:27.921721 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:27.923379 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 90896.6us
I20260709 12:57:27.925962 13541 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:57:27.929621 14243 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:27.914414 16604 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:27.919322 16569 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:27.935017 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1370) took 1143 ms. Trace:
I20260709 12:57:27.942862 13536 rpcz_store.cc:276] 0709 12:57:26.791136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.791252 (+   116us) service_pool.cc:224] Handling call
0709 12:57:27.934831 (+1143579us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54000889d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:27.934957 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323356416}
I20260709 12:57:27.935247 13540 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:27.944665 14242 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:27.945677 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 561) took 1660 ms. Trace:
I20260709 12:57:27.946250 14246 rpcz_store.cc:276] 0709 12:57:26.285088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.292079 (+  6991us) service_pool.cc:224] Handling call
0709 12:57:27.943747 (+1651668us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54005f1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:27.943902 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409273728}
I20260709 12:57:27.947194 16613 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:27.941624 16584 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:27.949570 13616 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:27.920737 16288 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.929203 16603 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:27.957289 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:27.957279 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.947952 16613 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:27.964058 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:27.967463 13614 raft_consensus.cc:2410] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:57:27.913640 16609 consensus_queue.cc:1059] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:57:27.932574 16369 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:27.953572 15958 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:57:27.993227 16615 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:27.988185 16613 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:27.982776 16524 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:27.997936 13619 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:27.997579 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.944249 16496 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:57:27.999749 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.999866 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:27.966188 16603 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:28.012971 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.012748 16603 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.014420 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.015044 13612 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:27.946382 16289 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:57:28.002195 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.001194 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.004643 13617 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:28.016775 13615 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.007496 13616 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:28.021178 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 316377us
I20260709 12:57:27.985824 16606 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.178s
I20260709 12:57:27.980294 16596 consensus_queue.cc:1059] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.230s
I20260709 12:57:27.994050 16602 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.072403 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.032644 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:28.073153 13618 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:28.057942 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.077814 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:28.066855 16545 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:28.004248 16524 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:28.014431 16369 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.079087 16524 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.038417 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:28.083058 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 44737.1us
I20260709 12:57:28.083709 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.086174 16545 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:28.086897 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.087518 13611 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:28.032191 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.091384 16288 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.091413 16336 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:28.024380 16585 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:28.096267 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.096195 13618 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:28.100191 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.101768 16524 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:28.105770 13615 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
W20260709 12:57:28.107121 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 83562.1us
I20260709 12:57:28.108676 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:28.034297 16605 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:57:28.112248 13611 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:28.088801 16574 raft_consensus.cc:2764] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:57:28.064248 16422 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:57:28.121917 13612 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:57:28.122797 13612 pending_rounds.cc:85] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 7
I20260709 12:57:28.123136 13612 pending_rounds.cc:107] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:57:28.123463 13612 raft_consensus.cc:2889] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:28.110054 16336 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:28.106940 16595 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:28.172290 16568 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:28.133409 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:28.120647 16585 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:57:28.137887 16510 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:28.140360 16480 consensus_meta.cc:287] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.533s	user 0.005s	sys 0.011s
I20260709 12:57:28.140820 16616 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:28.141480 14241 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:28.145165 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 483) took 1620 ms. Trace:
I20260709 12:57:28.146085 13686 raft_consensus.cc:2410] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:57:28.146438 16408 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:28.147106 16569 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.015476 16509 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.003934 16615 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:28.150288 16577 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.695s	user 0.002s	sys 0.000s
W20260709 12:57:28.152525 16570 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.705s	user 0.000s	sys 0.003s
I20260709 12:57:28.165454 16613 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:28.169231 16583 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.717s	user 0.002s	sys 0.000s
I20260709 12:57:28.169386 16551 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.118156 16212 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:28.235466 13688 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:57:28.179232 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 233126us
I20260709 12:57:28.179350 14242 rpcz_store.cc:276] 0709 12:57:26.524212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.524736 (+   524us) service_pool.cc:224] Handling call
0709 12:57:27.944445 (+1419709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:27.945214 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:28.144353 (+199139us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54005f1c50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:28.144523 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3264261888}
I20260709 12:57:28.237107 13543 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
W20260709 12:57:28.210659 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.173482 16336 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.172744 16568 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:28.178443 16480 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:28.236682 14212 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:28.172742 16595 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.252179 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:57:28.251572 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 167921us
I20260709 12:57:28.254827 13509 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.258570 13534 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:28.260557 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:28.268285 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 150642us
I20260709 12:57:28.271368 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.273041 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.216163 16408 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.276377 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:28.216646 16491 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:57:28.280287 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.264164 16589 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:28.263734 16510 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.263096 16360 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:28.269280 16480 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:28.275745 16522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.016s	sys 0.017s
I20260709 12:57:28.298996 16629 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:28.299806 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.300596 13617 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:28.303519 16185 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:28.312144 13619 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:28.313975 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 382) took 1106 ms. Trace:
I20260709 12:57:28.314384 13693 rpcz_store.cc:276] 0709 12:57:27.207300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:27.207621 (+   321us) service_pool.cc:224] Handling call
0709 12:57:27.214791 (+  7170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:27.229729 (+ 14938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:27.229760 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:27.230272 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:57:27.348158 (+117886us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:27.348213 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:27.348307 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:28.149577 (+801270us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:28.313680 (+164103us) raft_consensus.cc:1685] finished
0709 12:57:28.313701 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:28.313916 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107,"raft.queue_time_us":15552,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":23174,"spinlock_wait_cycles":53671296,"thread_start_us":117486,"threads_started":1,"wal-append.queue_time_us":224839}
I20260709 12:57:28.316479 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:28.282511 16621 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:28.316888 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46920.9us
I20260709 12:57:28.211331 16616 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:28.292809 16522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.017s	sys 0.017s
I20260709 12:57:28.304003 16185 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:28.325417 13617 raft_consensus.cc:3055] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:28.293450 16545 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:28.332574 13617 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:28.294544 16551 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:28.326968 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:28.339877 13693 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:28.340772 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:28.323344 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.288039 16602 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:57:28.341828 13693 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:28.335328 13617 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.287240 16619 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:57:28.345430 16616 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.350036 16185 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.382050 16185 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:28.382238 16630 log.cc:927] Time spent T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:57:28.382827 16615 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:28.290963 16568 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.360731 13610 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:57:28.384457 13610 pending_rounds.cc:85] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 2
I20260709 12:57:28.384732 13610 pending_rounds.cc:107] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:57:28.384980 13610 raft_consensus.cc:2889] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:28.360893 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:28.389806 14209 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:28.366520 16480 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:28.368064 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.372707 13684 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
I20260709 12:57:28.359359 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.390053 13686 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:28.365191 16569 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:28.429221 13617 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:28.436448 13686 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:28.438952 13543 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:57:28.456736 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 67006.8us
I20260709 12:57:28.341912 16604 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:57:28.342307 16545 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.473171 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:28.573282 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:28.492046 13686 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:57:28.574144 13543 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:28.556519 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.553786 16595 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:28.575984 16640 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:28.554376 16408 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.576581 16640 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.573679 13542 raft_consensus.cc:2376] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9. Re-sending same reply.
I20260709 12:57:28.391763 16360 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:28.518471 16578 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.068s	user 0.001s	sys 0.004s
I20260709 12:57:28.519789 16191 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:28.496973 16496 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:28.503103 16318 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:28.563851 16615 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:28.509478 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.594866 16615 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.597851 16615 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:28.599929 16166 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:28.604000 16318 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.604187 16360 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.606207 16318 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.595084 16496 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.608786 16496 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.590133 16191 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:28.609670 16191 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.456326 16524 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:28.625653 16318 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:28.576594 16545 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.507421 16589 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:28.642614 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.645143 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.637416 16524 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.650558 16524 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.628610 16336 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.664994 16336 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:28.672689 16512 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:57:28.636907 16318 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.679190 16506 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:28.682799 16615 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:28.683346 16615 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.685599 16615 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:28.686667 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:28.687407 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:28.687721 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.689764 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1361) took 1258 ms. Trace:
I20260709 12:57:28.690176 13688 rpcz_store.cc:276] 0709 12:57:27.431621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:27.814947 (+383326us) service_pool.cc:224] Handling call
0709 12:57:27.815302 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:28.236069 (+420767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:28.689491 (+453422us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400159854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:28.689702 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965556096}
I20260709 12:57:28.690610 16191 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.679730 16506 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.693392 16522 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:28.698258 16609 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:28.698706 16609 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:28.679232 16318 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.699600 16609 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.700611 16609 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:28.706347 16539 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:28.706787 16539 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:28.707983 16506 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.708859 16571 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:28.709371 16571 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.710341 16506 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.715204 16571 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:28.707625 16539 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.717482 16539 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:28.717448 16286 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.723686 16593 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:28.724299 16593 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:28.725422 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.729359 16601 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:28.729714 13541 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:28.729928 16601 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:28.733444 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 899) took 1345 ms. Trace:
I20260709 12:57:28.733814 14243 rpcz_store.cc:276] 0709 12:57:27.388024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:27.388302 (+   278us) service_pool.cc:224] Handling call
0709 12:57:28.733290 (+1344988us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54006009d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:28.733389 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1209742208}
I20260709 12:57:28.744951 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.746775 16506 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:28.746842 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.748052 16506 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:28.748427 16506 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:28.748762 16506 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.752480 16589 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:28.755465 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.756565 16506 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.758364 16550 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:28.758455 13509 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:28.758913 16550 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:28.759161 16506 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.759351 15968 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:28.759805 16506 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:28.760847 16166 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:28.761322 16166 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.761260 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.761664 16506 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:28.763139 13684 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:57:28.764658 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:28.762734 16506 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:28.765513 13684 raft_consensus.cc:2410] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:57:28.769196 16369 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:28.769866 16369 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.759907 16550 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.775981 16550 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:28.778506 16191 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:28.772733 13686 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:28.776274 16369 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.782464 13541 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:57:28.691705 13688 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:28.731215 16601 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.772581 13536 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:28.785322 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.786504 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.792361 13612 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
W20260709 12:57:28.792642 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 213079us
W20260709 12:57:28.788641 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:28.794801 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:28.796059 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.796654 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:28.794616 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.799626 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.803335 13506 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:28.804828 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.809754 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.814582 16155 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:28.802068 15968 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:57:28.805323 16336 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:28.802767 16498 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:28.816368 16555 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:28.817035 16571 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:28.823652 14212 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.808933 16512 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:28.814066 16601 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:28.798002 16610 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:28.814837 16369 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:28.830263 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.834476 13618 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:57:28.840894 13618 pending_rounds.cc:85] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 3
I20260709 12:57:28.841200 13618 pending_rounds.cc:107] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:57:28.841436 13618 raft_consensus.cc:2889] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:28.842729 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.826792 16422 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:28.843816 16422 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:28.844296 16422 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:28.825037 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.848387 13618 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.853099 16498 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:28.853564 16155 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:28.854018 16498 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.854432 13613 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.854355 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:28.855790 16480 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:28.856334 16480 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:28.857215 16480 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.857569 16610 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.860265 16524 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:28.860760 16524 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:28.825351 16289 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:57:28.862296 16524 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.863545 16524 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:28.868640 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:28.870263 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.888867 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:28.871414 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:28.891240 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.891870 16422 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.893436 13610 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:28.894197 16422 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:28.898520 16626 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:28.898893 16166 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.900859 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.901921 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.902601 13534 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:28.903357 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.904246 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.870676 16638 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:28.905825 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.910331 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.859818 16610 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:28.912212 16371 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:28.913404 16527 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:28.913789 16527 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:28.914114 16527 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:28.883496 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1359) took 1462 ms. Trace:
I20260709 12:57:28.917196 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:28.886412 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:28.889874 13688 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:57:28.887091 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.879868 13613 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
W20260709 12:57:28.882090 13539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1375) took 1980 ms (client timeout 2229 ms). Trace:
I20260709 12:57:28.920526 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.877400 13618 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:28.924935 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.918820 13538 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:57:28.927935 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:28.929452 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:28.916937 13689 rpcz_store.cc:276] 0709 12:57:27.421199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:27.680411 (+259212us) service_pool.cc:224] Handling call
0709 12:57:27.680836 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:27.839844 (+159008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:28.883123 (+1043279us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54002963d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:28.883419 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341867264}
W20260709 12:57:28.920135 13539 rpcz_store.cc:269] 0709 12:57:26.901454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.916403 (+ 14949us) service_pool.cc:224] Handling call
0709 12:57:28.881804 (+1965401us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b540007e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:28.882005 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169062656}
I20260709 12:57:28.931593 13534 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:28.918020 13685 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:28.933789 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:28.936656 16626 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:28.937330 16626 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:28.938457 16626 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:28.940258 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:28.945362 13691 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:57:28.946592 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1356) took 1535 ms. Trace:
I20260709 12:57:28.889282 16571 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:28.947000 13691 rpcz_store.cc:276] 0709 12:57:27.411113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:28.919151 (+1508038us) service_pool.cc:224] Handling call
0709 12:57:28.945048 (+ 25897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:28.945926 (+   878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:28.946514 (+   588us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:28.870664 16647 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:28.959272 13658 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:28.942271 16371 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:28.934406 16318 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:28.966614 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:28.970695 13613 raft_consensus.cc:2410] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:57:28.960498 16571 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:28.976182 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 888) took 2340 ms. Trace:
I20260709 12:57:28.976866 16571 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:28.979208 16371 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:28.980257 16371 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:28.982779 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:28.986694 14243 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.976620 14240 rpcz_store.cc:276] 0709 12:57:26.631620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.741817 (+110197us) service_pool.cc:224] Handling call
0709 12:57:26.742261 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:27.081932 (+339671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:28.971888 (+1889956us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400497ad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:28.972339 (+   451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666819840}
I20260709 12:57:28.983148 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:28.989732 14242 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:28.961299 16647 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:29.000173 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:29.002249 14209 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:29.005510 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 485) took 2452 ms. Trace:
W20260709 12:57:28.986574 16588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.008s	sys 0.008s
I20260709 12:57:29.019064 14243 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:29.022341 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:29.026036 16422 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:29.028793 15958 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:29.029367 15958 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.031554 15958 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.032091 16635 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:29.032490 16635 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:29.046239 14242 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:29.027830 16422 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:29.048055 16422 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:29.049054 16422 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.006011 14239 rpcz_store.cc:276] 0709 12:57:26.552958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.553152 (+   194us) service_pool.cc:224] Handling call
0709 12:57:26.558676 (+  5524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:26.603057 (+ 44381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:26.603092 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:26.619039 (+ 15947us) log.cc:844] Serialized 26 byte log entry
0709 12:57:26.909541 (+290502us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:27.090989 (+181448us) log.cc:844] Serialized 14 byte log entry
0709 12:57:27.091445 (+   456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:29.000111 (+1908666us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b54004c2a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:29.000161 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:29.000204 (+    43us) raft_consensus.cc:1685] finished
0709 12:57:29.005269 (+  5065us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:29.005443 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20266,"prepare.queue_time_us":68478,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":10193,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":12136,"spinlock_wait_cycles":100916480,"thread_start_us":409723,"threads_started":2,"wal-append.queue_time_us":417051,"wal-append.run_cpu_time_us":3133,"wal-append.run_wall_time_us":1175326}
I20260709 12:57:28.972932 16561 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:57:28.991952 16318 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:29.004161 16491 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.003044 16647 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:29.060389 16588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.008s	sys 0.008s
I20260709 12:57:29.016377 16360 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:29.105257 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.002571 16602 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:57:28.999281 13618 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:28.987125 16615 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:57:29.010114 16550 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:29.100860 13687 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:28.982779 16593 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.115513 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.116499 13618 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:29.060271 16590 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:57:29.103160 16490 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:29.120666 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.120746 16166 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.122828 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.124053 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.125175 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.125713 13614 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:29.126488 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.127485 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.130182 13542 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:29.120720 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.107167 16360 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.131538 13618 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:29.132946 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.134291 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.033284 16635 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.135857 13686 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:29.136634 13618 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:57:29.132773 16602 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:29.146335 13541 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:29.102595 16491 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.154134 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.154795 13614 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
W20260709 12:57:29.156894 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.134514 16360 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:29.134331 16604 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:29.170856 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 221435us
I20260709 12:57:29.152943 16369 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:29.152704 16336 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:29.152926 16602 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:29.194973 13614 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:57:29.133935 16490 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.181058 16635 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:29.190896 16524 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.217238 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1300) took 2256 ms. Trace:
I20260709 12:57:29.216125 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1363) took 1398 ms. Trace:
I20260709 12:57:29.218178 13540 rpcz_store.cc:276] 0709 12:57:26.952368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:26.952494 (+   126us) service_pool.cc:224] Handling call
0709 12:57:27.934991 (+982497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:27.943844 (+  8853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:29.208288 (+1264444us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54000889d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:29.208443 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258861952}
I20260709 12:57:29.218291 13693 rpcz_store.cc:276] 0709 12:57:27.817841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:28.396233 (+578392us) service_pool.cc:224] Handling call
0709 12:57:28.491843 (+ 95610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:28.494958 (+  3115us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:28.944033 (+449075us) log.cc:844] Serialized 14 byte log entry
0709 12:57:29.215609 (+271576us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:29.215682 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:29.215748 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:29.215857 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:29.216041 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":620274,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":10947,"raft.queue_time_us":1869,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":719656,"threads_started":2}
W20260709 12:57:29.224713 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 202463us
I20260709 12:57:29.196841 16336 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.168401 16318 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.233500 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:29.238377 13614 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:57:29.240927 16289 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:29.241456 16289 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.241959 16289 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:29.267102 16609 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.266968 16590 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.267817 16609 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.268194 16590 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:29.269649 16654 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:57:29.270226 16609 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.270329 16590 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.272890 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.273641 16336 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:29.275061 16649 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:57:29.230252 16166 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.279045 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.279857 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:29.281435 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:29.282482 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.220700 16524 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:29.321398 16632 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.933s	user 0.003s	sys 0.000s
W20260709 12:57:29.322613 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:29.325614 16657 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
W20260709 12:57:29.329308 16655 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 12:57:29.330541 16653 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:57:29.331907 16652 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:57:29.349474 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.277400 16610 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 12:57:29.354977 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.345121 16593 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.292853 16166 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.234279 16602 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.370072 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.371906 13659 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.357852 16561 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:29.380409 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.381325 13612 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:57:29.372757 16610 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.399032 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1310) took 1843 ms. Trace:
I20260709 12:57:29.380023 16550 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.399454 13535 rpcz_store.cc:276] 0709 12:57:27.555186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:27.735565 (+180379us) service_pool.cc:224] Handling call
0709 12:57:27.761240 (+ 25675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:27.788125 (+ 26885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:27.788178 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:27.801223 (+ 13045us) log.cc:844] Serialized 26 byte log entry
0709 12:57:27.908912 (+107689us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:27.908951 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:27.909018 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:28.033865 (+124847us) raft_consensus.cc:1685] finished
0709 12:57:28.033886 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:29.388932 (+1355046us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b540007dfd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:29.398963 (+ 10031us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7843,"raft.queue_time_us":5939,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":92,"spinlock_wait_cycles":56495872,"thread_start_us":62844,"threads_started":1,"wal-append.queue_time_us":111828,"wal-append.run_cpu_time_us":2242,"wal-append.run_wall_time_us":1087715}
W20260709 12:57:29.406076 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.378563 16491 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:29.385743 16593 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:29.417507 13611 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:57:29.419179 16490 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.389710 16674 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:29.445040 16661 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:57:29.407449 16289 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.402765 16610 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:29.464711 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 166311us
W20260709 12:57:29.462657 16656 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 12:57:29.461866 16601 consensus_meta.cc:287] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.597s	user 0.003s	sys 0.007s
I20260709 12:57:29.441953 16336 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:29.416587 16550 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:29.469866 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 98052.4us
I20260709 12:57:29.472154 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.476470 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.483405 16289 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:29.490397 16527 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.005s
I20260709 12:57:29.490881 16527 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.492084 13688 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:57:29.474078 16336 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.497871 16336 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.504653 16166 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.505805 14243 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEARNER]: Advancing to term 7
I20260709 12:57:29.521384 16603 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:29.523061 16553 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:29.524061 16553 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:29.524492 16553 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:29.527169 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.525523 16553 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.547487 16553 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:29.445300 16674 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.478510 16422 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:29.561161 16422 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:29.563510 16422 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.516877 16524 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.656348 14243 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:57:29.668262 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:29.669888 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 192865us
I20260709 12:57:29.670704 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:29.672377 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:29.584551 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.676555 14243 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEARNER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:29.677673 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.572631 16674 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:29.682546 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:29.677312 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.686048 14238 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:29.688778 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.692406 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:29.693068 16318 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:29.694204 16593 raft_consensus.cc:2764] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:57:29.697477 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.694205 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.551710 16647 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.699404 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.674391 16589 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.701485 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.681377 16590 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:29.705777 16369 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:29.706292 16369 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:29.674235 16155 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:57:29.710345 16647 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:29.710906 16662 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 12:57:29.674096 16516 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 12:57:29.714834 16659 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 12:57:29.715168 16610 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:29.716926 16671 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.302s	user 0.001s	sys 0.001s
I20260709 12:57:29.688074 16688 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:29.667721 16360 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.688966 16550 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.696446 16553 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:29.704599 13659 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.723109 13659 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.724972 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:29.725107 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.723363 16688 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:29.726608 16685 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:57:29.729534 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.731590 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:29.732203 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:29.732803 13540 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:29.733536 13610 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:57:29.736300 13542 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:29.718604 16610 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.740875 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:29.741562 14240 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
W20260709 12:57:29.747495 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.748790 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:29.749358 13539 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:29.704481 16674 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:29.721568 16527 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.745458 16601 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:57:29.735283 16593 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:57:29.753033 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.740614 16318 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:29.722214 16553 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:29.763535 14243 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:29.749980 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.778280 16635 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:29.751878 16688 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.785660 16360 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.721091 16590 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.786003 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.787631 14238 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:29.788276 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.796911 13616 raft_consensus.cc:2410] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:57:29.704102 16589 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:29.809656 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.788589 13539 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:29.822916 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:29.823450 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:29.800333 16658 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.523s	user 0.003s	sys 0.004s
I20260709 12:57:29.781466 16318 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:29.791987 16553 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:29.846199 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 93239.5us
I20260709 12:57:29.828301 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.829073 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.829753 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.848403 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.850050 14243 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:57:29.834522 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:29.779760 16686 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.763716 16610 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.789484 15946 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:29.823619 14237 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:29.830439 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.820652 16688 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.797180 16640 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.819077 16524 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.857285 14245 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:29.865710 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:29.871023 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:29.846938 16318 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:29.871457 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:29.852728 16357 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:29.885864 13690 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:57:29.886274 13690 pending_rounds.cc:85] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 3
I20260709 12:57:29.886492 13690 pending_rounds.cc:107] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:57:29.886713 13690 raft_consensus.cc:2889] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:29.884099 16422 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:29.889729 16422 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.892975 16203 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:29.893400 16203 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:29.894923 16606 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.895479 16606 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.884353 16497 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:29.896456 16497 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.897541 16606 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.898569 16497 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:29.887882 16555 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:29.901544 16510 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.902124 16510 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.904309 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:29.905118 13684 raft_consensus.cc:2410] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:57:29.905189 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1377) took 1033 ms. Trace:
W20260709 12:57:29.906435 13687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1351) took 2515 ms (client timeout 1605 ms). Trace:
W20260709 12:57:29.906942 13687 rpcz_store.cc:269] 0709 12:57:27.391234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:27.398542 (+  7308us) service_pool.cc:224] Handling call
0709 12:57:29.906227 (+2507685us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54002968d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:29.906343 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3873686912}
I20260709 12:57:29.908226 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:29.909452 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:29.910179 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:29.910466 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:29.910570 16589 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.912201 13684 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:29.915068 16369 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.916093 16589 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.916601 16589 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.916952 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.918542 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:29.919219 13615 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:29.921221 16590 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.921590 16589 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:29.919862 13619 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:29.825850 16635 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.926149 14211 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:29.905824 16688 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:29.918565 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.932521 13542 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:29.933163 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:29.874505 16603 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.905572 13686 rpcz_store.cc:276] 0709 12:57:28.871354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:28.871491 (+   137us) service_pool.cc:224] Handling call
0709 12:57:28.871863 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:29.136344 (+264481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:29.904962 (+768618us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b540042d1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:29.905127 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606659328}
I20260709 12:57:29.934531 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:29.935547 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:29.936029 13686 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:29.932305 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:29.937506 13541 raft_consensus.cc:2410] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:29.937752 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:29.939568 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:29.940615 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:29.937602 16674 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:57:29.944381 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:29.944581 16674 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:29.945170 16674 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.945806 13692 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:29.948853 13540 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:57:29.953850 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.957669 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:29.932152 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46752.6us
I20260709 12:57:29.924145 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:29.941877 16590 raft_consensus.cc:2764] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:57:29.941685 16603 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.942381 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:29.942696 14245 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:29.925233 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:29.942958 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:29.967886 13536 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:57:29.943728 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:29.943593 16601 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:29.962059 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:29.923825 16422 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:29.925133 16606 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.972302 13619 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:29.928289 16604 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:29.993227 16604 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:29.993809 13686 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:29.938009 16555 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:29.880602 16524 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:29.977012 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:29.984253 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52097us
I20260709 12:57:29.868952 16686 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.002956 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:30.006199 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:29.996130 16516 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:57:30.008271 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:30.014523 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:29.997145 16674 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.022523 14246 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:30.009472 16589 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:57:30.012826 16590 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:30.005570 16606 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:30.000030 16213 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 12:57:30.041522 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 99808.7us
W20260709 12:57:30.042939 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 102420us
I20260709 12:57:30.054961 14211 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:30.019661 16604 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:30.049335 13659 leader_election.cc:336] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.711s (SENT)
I20260709 12:57:30.023232 16601 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:30.033932 16674 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:30.032327 16408 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:30.065351 13686 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:57:30.068409 13686 pending_rounds.cc:85] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:57:30.068704 13686 pending_rounds.cc:107] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:57:30.066829 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:30.068962 13686 raft_consensus.cc:2889] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:30.035061 16590 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:30.073382 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.076598 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1379) took 1204 ms. Trace:
I20260709 12:57:30.076855 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:30.077039 13688 rpcz_store.cc:276] 0709 12:57:28.872239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:28.933839 (+ 61600us) service_pool.cc:224] Handling call
0709 12:57:28.941802 (+  7963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:29.492541 (+550739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:30.076371 (+583830us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b540034f154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:30.076542 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264712576}
I20260709 12:57:30.078167 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.078699 13688 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:30.080915 16524 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.079808 13688 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:30.090071 16706 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:30.090253 15958 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:30.091041 16706 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:30.091579 16706 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:30.056460 16510 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.093751 16688 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:30.067101 16408 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:30.109903 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 76232.2us
I20260709 12:57:30.075573 16590 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:30.112974 13506 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:30.083626 16610 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.129729 16335 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:30.135488 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.136615 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.137368 16568 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:30.137584 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.138339 13691 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:30.139014 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.139663 13691 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 5: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:30.142297 16704 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:57:30.142786 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.143491 13691 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
W20260709 12:57:30.143750 16699 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:57:30.145183 13610 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:30.145241 13691 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:30.147197 16710 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:30.094668 16640 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:57:30.123764 16601 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:30.125705 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:30.179205 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 171011us
I20260709 12:57:30.180217 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:30.181094 16674 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:30.181531 16590 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.182416 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:30.132787 16369 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.184707 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:30.186558 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:30.179216 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:30.185339 16674 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:30.151758 16635 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.201095 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.102746 16688 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:30.202972 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:30.177709 16498 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:30.208181 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:30.196830 16286 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:30.204649 13693 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:30.193835 16603 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.215999 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.216434 16635 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:30.216975 13691 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:30.218904 16717 consensus_queue.cc:1059] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:30.177943 16601 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.203656 16422 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:30.228446 16701 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:57:30.232141 16686 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:30.242851 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:30.081704 16555 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.214339 16185 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:30.255499 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:30.259735 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:30.264703 13617 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:30.198357 16553 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.179788 16408 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.268903 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.269963 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.270627 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.270627 14242 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:30.269963 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.273765 16318 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.274868 13689 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:30.258119 16606 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:30.273778 13541 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
W20260709 12:57:30.276053 16716 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:57:30.278640 16717 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:30.092538 16706 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.284087 14242 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:57:30.285272 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.285828 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:30.220161 16635 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:30.290691 16408 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:30.295670 13692 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:30.282845 16688 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:30.310667 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:30.287956 16185 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.314872 16688 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:30.289436 16635 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:30.289953 16318 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:30.295893 16698 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:30.296535 16717 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:30.313704 16371 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:30.403416 16722 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:57:30.406075 14242 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:30.354743 16698 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:30.413879 16698 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.417562 16601 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.397218 16185 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:30.419894 16725 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:30.420661 16725 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.422933 16725 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.425313 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.426105 14242 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
W20260709 12:57:30.427806 16719 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:57:30.432103 16684 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:30.432714 16684 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.362746 16371 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.437908 16371 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:30.547096 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:30.602890 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:30.649160 16318 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:30.660656 16318 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.665009 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.667557 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:30.358286 16717 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.681285 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:30.686407 13615 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:30.641726 16706 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:30.691150 14212 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:30.673085 16723 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
W20260709 12:57:30.708189 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:30.711238 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:30.717146 16686 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:30.717780 16686 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:30.718202 16686 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:30.725296 16601 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:30.732604 16601 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:30.741434 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:30.741724 16601 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:30.742601 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:30.743710 13693 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:30.744366 16203 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.745441 16203 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:30.746923 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.752353 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:30.756201 16615 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:30.760087 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.760921 13692 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:30.761943 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:30.747781 13534 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:30.763170 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:30.763988 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.764813 13693 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:30.765519 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.765822 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.766568 13693 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:30.766727 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.767699 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.691327 16710 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:30.768347 14238 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:30.768690 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.769974 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.776894 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.777845 14245 raft_consensus.cc:2410] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:57:30.778103 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.779670 13690 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:57:30.691152 16357 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:30.804942 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.773492 16555 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:30.810623 13692 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:57:30.773767 16603 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:30.767876 13693 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:30.781796 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:30.780154 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:30.773361 16710 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:30.785156 16726 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:57:30.790649 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:30.792400 13690 pending_rounds.cc:85] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:57:30.805696 14237 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:30.822135 13690 pending_rounds.cc:107] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:57:30.822568 13690 raft_consensus.cc:2889] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:30.675191 16318 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.822945 13690 pending_rounds.cc:107] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:57:30.823302 13690 raft_consensus.cc:2889] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:30.816802 16589 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:30.820722 16555 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.827000 16589 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.829366 16555 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.831699 16318 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:30.832051 16555 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:30.832204 16318 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:30.832594 16318 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:30.832547 16555 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.834546 16682 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:30.835119 16682 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.835520 16555 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.837162 16682 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.839030 16318 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.840898 16318 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.841346 16318 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:30.841727 16318 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.843125 16318 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.849536 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:30.850806 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.851522 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 918) took 1856 ms. Trace:
I20260709 12:57:30.851938 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.816712 16357 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:30.856473 13616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2592) took 2769 ms (client timeout 2044 ms). Trace:
W20260709 12:57:30.857043 13616 rpcz_store.cc:269] 0709 12:57:28.090460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:28.090595 (+   135us) service_pool.cc:224] Handling call
0709 12:57:30.856196 (+2765601us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b540012b1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:30.856360 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3876139392}
I20260709 12:57:30.719857 16706 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.862601 16706 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.863370 16706 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:30.867139 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:30.831579 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51509.7us
I20260709 12:57:30.870110 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:30.870390 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1312) took 3276 ms. Trace:
I20260709 12:57:30.870920 13538 rpcz_store.cc:276] 0709 12:57:27.594121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:27.745738 (+151617us) service_pool.cc:224] Handling call
0709 12:57:28.884421 (+1138683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:28.927221 (+ 42800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:29.746963 (+819742us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b540007e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:30.870135 (+1123172us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540007e9d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:30.870315 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696308992}
I20260709 12:57:30.873044 16696 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:30.873602 16696 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:30.873952 16696 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:30.878825 16590 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.852035 14240 rpcz_store.cc:276] 0709 12:57:28.995080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:28.995380 (+   300us) service_pool.cc:224] Handling call
0709 12:57:29.016602 (+ 21222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:29.749171 (+732569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:30.851204 (+1102033us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400686d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:30.851445 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666807168}
I20260709 12:57:30.883162 13536 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:57:30.821215 16710 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:30.830055 16603 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.831382 16550 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:30.792258 16494 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:30.829427 16589 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.889560 13584 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:30.858841 16357 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.902200 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:30.909222 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:30.911610 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.913604 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.812960 16553 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.862515 16684 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:30.921253 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:30.921715 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:30.923259 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:30.896405 16155 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:30.925331 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:30.935461 14238 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:30.918151 14244 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:30.925376 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2795) took 1001 ms. Trace:
I20260709 12:57:30.936513 13686 rpcz_store.cc:276] 0709 12:57:29.924455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:29.994808 (+ 70353us) service_pool.cc:224] Handling call
0709 12:57:29.995212 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:30.069440 (+ 74228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:30.925155 (+855715us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x7b54002986d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:30.925320 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160186624}
I20260709 12:57:30.938776 13691 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:30.929261 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:30.916745 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:30.957041 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:30.958411 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:30.900141 16723 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:57:30.932717 13538 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:30.960489 13536 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:30.934158 16568 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:30.964260 16357 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:30.965849 16728 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:57:30.947103 16550 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:30.991462 16550 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.992425 16698 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:30.993846 16550 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:30.970305 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:30.941180 16155 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:30.930085 16494 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:30.978029 16357 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:31.030289 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 60150.7us
W20260709 12:57:31.090955 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 59926.8us
I20260709 12:57:31.092499 16155 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:31.093346 16553 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:31.094817 16684 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:31.095304 16684 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:31.096447 16717 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.096191 16684 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:31.097573 16688 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.782s	user 0.011s	sys 0.000s
I20260709 12:57:31.098205 16684 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:31.098393 13688 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:57:31.098845 13688 pending_rounds.cc:85] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 7
I20260709 12:57:31.099076 13688 pending_rounds.cc:107] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:57:31.099299 13688 raft_consensus.cc:2889] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:31.032823 16568 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:31.033568 16357 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:31.124575 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:31.128539 16688 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:31.149459 16568 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:31.149966 16568 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:31.150307 16568 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:31.157243 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:31.157735 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:31.158344 13691 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:31.159164 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:31.160250 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:31.162114 13693 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:57:31.164390 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:31.166760 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:31.167177 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:31.168257 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:31.179248 16494 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.180570 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 395) took 1256 ms. Trace:
W20260709 12:57:31.197249 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:31.205914 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:31.209185 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2782) took 1347 ms. Trace:
I20260709 12:57:31.209733 13689 rpcz_store.cc:276] 0709 12:57:29.861366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:29.864151 (+  2785us) service_pool.cc:224] Handling call
0709 12:57:29.864539 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:30.275241 (+410702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:31.202333 (+927092us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540015ead4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:31.209085 (+  6752us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943214336}
W20260709 12:57:31.155705 16727 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.737s	user 0.000s	sys 0.010s
I20260709 12:57:31.204228 13684 rpcz_store.cc:276] 0709 12:57:29.925291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:29.975425 (+ 50134us) service_pool.cc:224] Handling call
0709 12:57:29.975736 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:29.997495 (+ 21759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:29.997524 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:29.999934 (+  2410us) log.cc:844] Serialized 26 byte log entry
0709 12:57:30.098915 (+ 98981us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:30.098961 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:30.099023 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:31.162942 (+1063919us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b2400244d18. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b703fb0 00007fa02e37a488 00007fa02e37abe8 00007fa02e352398 00007fa02e34d76a 00007fa02e34cbb6
0709 12:57:31.180096 (+ 17154us) raft_consensus.cc:1685] finished
0709 12:57:31.180125 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:31.180493 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6423,"raft.queue_time_us":14309,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":10039,"spinlock_wait_cycles":11684096,"thread_start_us":71250,"threads_started":1,"wal-append.queue_time_us":177332}
I20260709 12:57:31.221961 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:31.222807 13691 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:31.135807 16688 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.234699 16585 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:31.252133 16585 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:31.252689 16585 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:31.215690 16603 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.220302 16674 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:31.260246 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 398) took 1334 ms. Trace:
I20260709 12:57:31.260838 13688 rpcz_store.cc:276] 0709 12:57:29.926031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:30.083737 (+157706us) service_pool.cc:224] Handling call
0709 12:57:30.102528 (+ 18791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:31.099728 (+997200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:31.259864 (+160136us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54001eebd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:31.260149 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289432448}
I20260709 12:57:31.214803 16717 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 12:57:31.274451 16686 consensus_meta.cc:287] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.004s
I20260709 12:57:31.315299 14244 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:31.276036 16603 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:31.317175 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:31.333076 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.020s	sys 0.027s
I20260709 12:57:31.278586 16203 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:31.343271 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.021s	sys 0.027s
W20260709 12:57:31.346818 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.021s	sys 0.027s
W20260709 12:57:31.332295 16606 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.002s	sys 0.011s
W20260709 12:57:31.334509 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.007s	sys 0.006s
W20260709 12:57:31.357978 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.007s	sys 0.006s
I20260709 12:57:31.329540 16603 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:31.348080 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 515588us
W20260709 12:57:31.365711 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:31.366674 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:31.343000 16408 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.018s	sys 0.029s
W20260709 12:57:31.341849 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.020s	sys 0.029s
I20260709 12:57:31.334597 16674 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:31.335842 16550 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:31.427511 16626 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:31.435460 16724 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:31.345716 16480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.025s	sys 0.014s
I20260709 12:57:31.320434 16686 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:31.362991 16606 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.002s	sys 0.011s
I20260709 12:57:31.444406 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:31.450103 13691 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 12:57:31.429735 16408 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.018s	sys 0.030s
W20260709 12:57:31.451718 13659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.008s	sys 0.008s
W20260709 12:57:31.476583 13659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.008s	sys 0.008s
W20260709 12:57:31.476924 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 721931us
W20260709 12:57:31.452121 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.010s	sys 0.005s
W20260709 12:57:31.440537 16697 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.000s	sys 0.016s
W20260709 12:57:31.470825 16710 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
W20260709 12:57:31.410616 13583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.008s	sys 0.005s
W20260709 12:57:31.487291 13583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.009s	sys 0.006s
W20260709 12:57:31.481424 16697 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.000s	sys 0.016s
W20260709 12:57:31.475664 16480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.025s	sys 0.014s
I20260709 12:57:31.494040 14241 raft_consensus.cc:2393] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:31.473567 16674 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:31.495559 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:31.452968 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:31.407642 16203 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:31.408104 16603 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:31.464039 13617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.000s	sys 0.017s
I20260709 12:57:31.426972 16494 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:31.492184 16602 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:31.372354 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:31.325093 13536 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:31.475126 13691 pending_rounds.cc:85] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 4
W20260709 12:57:31.480265 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.011s	sys 0.005s
I20260709 12:57:31.510494 13691 pending_rounds.cc:107] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:57:31.510802 13691 raft_consensus.cc:2889] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:57:31.456693 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 637366us
W20260709 12:57:31.498332 16590 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.008s	sys 0.009s
I20260709 12:57:31.479522 16724 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.471635 16550 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.503728 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:31.435122 16506 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:31.505422 13617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.000s	sys 0.017s
I20260709 12:57:31.490804 16593 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:31.504416 16672 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:31.473779 16626 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.503196 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:31.521826 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:31.522406 16506 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.522910 16686 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.523665 16555 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:31.504709 16688 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.524540 16703 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:31.525434 16550 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:31.527040 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:31.523761 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:31.528098 16593 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.529376 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:31.529587 16698 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.537s	user 0.002s	sys 0.009s
I20260709 12:57:31.530496 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:31.531085 16706 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:31.532608 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:31.497990 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45122.7us
W20260709 12:57:31.534713 13441 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41440 (request call id 207) took 13039 ms (client timeout 15000 ms). Trace:
I20260709 12:57:31.534089 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:31.541026 16734 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:57:31.541739 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.007s	sys 0.016s
W20260709 12:57:31.592346 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.007s	sys 0.017s
W20260709 12:57:31.593065 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 634689us
W20260709 12:57:31.594465 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:31.504078 16717 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:31.595072 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:31.544600 16553 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:31.595736 14244 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
W20260709 12:57:31.544399 13509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.066s	sys 0.088s
I20260709 12:57:31.546252 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:31.597072 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 50942.9us
I20260709 12:57:31.597927 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:31.543869 13534 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 5.
W20260709 12:57:31.512617 16616 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.015s	sys 0.004s
W20260709 12:57:31.600445 13585 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.012s	sys 0.020s
I20260709 12:57:31.492326 16710 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.606573 16710 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:31.516803 16590 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.009s	sys 0.009s
I20260709 12:57:31.608121 13438 catalog_manager.cc:5697] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 3 to 5, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 5 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:31.583801 13536 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:31.579775 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.038s	sys 0.055s
I20260709 12:57:31.543116 16672 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:31.505604 16203 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:31.578034 16484 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.011s	sys 0.020s
W20260709 12:57:31.599551 13509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.066s	sys 0.088s
W20260709 12:57:31.642055 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 716639us
W20260709 12:57:31.574486 14246 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.034s	sys 0.050s
I20260709 12:57:31.513886 16494 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:31.603386 13585 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.012s	sys 0.021s
I20260709 12:57:31.607985 14242 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:31.643810 14246 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.035s	sys 0.050s
I20260709 12:57:31.609921 16497 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:31.608982 16682 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:57:31.581395 16498 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.015s	sys 0.017s
W20260709 12:57:31.624625 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.038s	sys 0.055s
W20260709 12:57:31.648475 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 857844us
I20260709 12:57:31.649953 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:31.650228 13508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 1.750s (SENT) [suppressed 32 similar messages]
I20260709 12:57:31.630297 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:31.600776 16616 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.015s	sys 0.004s
I20260709 12:57:31.590718 16688 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:31.635578 13536 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:31.640125 13438 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 4 to 8, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 8 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:31.610896 16687 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:57:31.625695 16555 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:31.681391 16203 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:31.681541 16555 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.683219 16203 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:31.613297 16553 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.684568 16724 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:31.686076 16553 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.686609 16553 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:31.687096 16553 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.600139 16674 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:31.641937 16484 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.011s	sys 0.021s
I20260709 12:57:31.636847 16672 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:31.600162 16371 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:57:31.661688 14209 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:31.590309 16703 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:31.588670 13441 rpcz_store.cc:269] 0709 12:57:18.495851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:18.506425 (+ 10574us) service_pool.cc:224] Handling call
0709 12:57:31.534618 (+13028193us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:57:25.885481 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:57:25.886221 (+   740us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:57:25.886272 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:57:25.886308 (+    36us) tablet.cc:662] Decoding operations
0709 12:57:25.954592 (+ 68284us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:57:25.954687 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:57:25.954713 (+    26us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:57:25.956595 (+  1882us) tablet.cc:701] Row locks acquired
0709 12:57:25.956647 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:57:25.956816 (+   169us) write_op.cc:270] Start()
0709 12:57:25.956974 (+   158us) write_op.cc:276] Timestamp: P: 1783601845956747 usec, L: 0
0709 12:57:25.956996 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:57:25.982658 (+ 25662us) log.cc:844] Serialized 8514 byte log entry
0709 12:57:26.952035 (+969377us) op_driver.cc:464] REPLICATION: finished
0709 12:57:27.273461 (+321426us) write_op.cc:301] APPLY: starting
0709 12:57:27.273581 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:57:27.274035 (+   454us) tablet.cc:1370] finished BulkCheckPresence
0709 12:57:27.274181 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:57:27.297482 (+ 23301us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:57:27.297952 (+   470us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 12:57:27.298004 (+    52us) write_op.cc:312] APPLY: finished
0709 12:57:27.298539 (+   535us) log.cc:844] Serialized 152 byte log entry
0709 12:57:27.298734 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:27.298937 (+   203us) write_op.cc:454] Released schema lock
0709 12:57:27.299010 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:57:27.300732 (+  1722us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:27.300771 (+    39us) write_op.cc:454] Released schema lock
0709 12:57:28.697538 (+1396767us) spinlock_profiling.cc:243] Waited 8.06 ms on lock 0x7b540041e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7af32a 00007fa02e734f14 00007fa02e734ec3 00007fa02e734e38 00007fa02e734df3 00007fa02e7340dc 00007fa02e75de16 00007fa02e745cbb 00007fa02e73039d 00007fa02e733d75
Metrics: {"mutex_wait_us":3968931,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":275496,"apply.run_cpu_time_us":5622,"apply.run_wall_time_us":27238,"num_ops":22,"prepare.queue_time_us":192091,"prepare.run_cpu_time_us":26565,"prepare.run_wall_time_us":346155,"raft.queue_time_us":356017,"raft.run_cpu_time_us":27491,"raft.run_wall_time_us":1807921,"replication_time_us":959414,"spinlock_wait_cycles":18536320,"thread_start_us":584485,"threads_started":4,"wal-append.queue_time_us":274232,"wal-append.run_cpu_time_us":52676,"wal-append.run_wall_time_us":1989151}]]}
I20260709 12:57:31.666483 16708 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:31.663645 16710 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 12:57:31.671224 16357 consensus_meta.cc:287] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.000s
I20260709 12:57:31.675887 16626 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:31.687902 16696 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.813s	user 0.009s	sys 0.000s
I20260709 12:57:31.637676 16335 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:57:31.600600 16706 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:31.653559 16498 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.015s	sys 0.017s
I20260709 12:57:31.665627 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:31.660936 16744 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:31.695097 16688 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:31.710207 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 44698.3us
W20260709 12:57:31.692327 13619 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.035s
I20260709 12:57:31.710978 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:31.660033 14245 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:31.683837 16203 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.611670 16717 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.683740 16555 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:31.695659 13438 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 4, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 4 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:57:31.693821 16684 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.594s	user 0.003s	sys 0.008s
I20260709 12:57:31.679627 16494 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.663973 16185 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 12:57:31.696874 16568 consensus_meta.cc:287] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.546s	user 0.004s	sys 0.008s
I20260709 12:57:31.694979 16585 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.707954 16748 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:57:31.710973 13619 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.003s	sys 0.035s
I20260709 12:57:31.754657 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 494) took 1074 ms. Trace:
I20260709 12:57:31.719677 16506 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:31.728904 16555 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:31.729367 16684 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:31.757385 16555 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.757921 16684 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:31.758522 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 492) took 1079 ms. Trace:
I20260709 12:57:31.758893 13617 rpcz_store.cc:276] 0709 12:57:30.679618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:30.680909 (+  1291us) service_pool.cc:224] Handling call
0709 12:57:30.710289 (+ 29380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:30.710682 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:30.710710 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:30.720833 (+ 10123us) log.cc:844] Serialized 26 byte log entry
0709 12:57:31.521621 (+800788us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:31.521659 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:31.521847 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:31.758319 (+236472us) raft_consensus.cc:1685] finished
0709 12:57:31.758340 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:31.758465 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":13703,"thread_start_us":639522,"threads_started":1,"wal-append.queue_time_us":935419}
I20260709 12:57:31.739148 13686 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:31.756639 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:31.706857 16696 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:31.712030 14238 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.021s
W20260709 12:57:31.780156 14238 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.009s	sys 0.021s
I20260709 12:57:31.765630 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:31.731283 13438 catalog_manager.cc:5697] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:31.781869 14244 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:31.704082 16357 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.783897 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:31.759706 16555 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:31.735076 16635 consensus_meta.cc:287] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.380s	user 0.013s	sys 0.000s
I20260709 12:57:31.717830 16550 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:57:31.716104 16686 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:31.708802 16739 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:31.734761 16585 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.730719 16593 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.740599 16746 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:31.756726 16688 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:31.728514 16741 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:31.808460 14244 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:57:31.741314 16553 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.730266 16717 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.759819 16684 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.803642 14241 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:31.755601 13619 rpcz_store.cc:276] 0709 12:57:30.680275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:30.680387 (+   112us) service_pool.cc:224] Handling call
0709 12:57:30.680604 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:30.680770 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:30.709516 (+ 28746us) log.cc:844] Serialized 14 byte log entry
0709 12:57:31.540463 (+830947us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:31.754227 (+213764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:31.754299 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:31.754357 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:31.754438 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:31.754586 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35922,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":37,"raft.queue_time_us":523695,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":5839232,"thread_start_us":1044241,"threads_started":2,"wal-append.queue_time_us":735479}
I20260709 12:57:31.733464 16703 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.811245 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:31.745590 16744 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:57:31.747768 16603 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.733242 16568 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:31.802084 15968 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.006s	sys 0.034s
I20260709 12:57:31.812888 13438 catalog_manager.cc:5697] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 3 to 5, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 5 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:31.829533 16696 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.832422 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 585) took 1076 ms. Trace:
I20260709 12:57:31.834023 16203 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:31.834692 16706 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:31.836478 13690 raft_consensus.cc:2368] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8.
I20260709 12:57:31.837308 16568 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.837579 16357 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:31.813141 16752 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:31.802139 16635 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:31.814177 16688 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:31.838914 14241 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:31.840620 13685 raft_consensus.cc:2410] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:57:31.841245 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:32.083577 14244 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:57:31.843465 16589 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:31.844285 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:31.811949 16746 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:31.855512 14246 rpcz_store.cc:276] 0709 12:57:30.756220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:30.760065 (+  3845us) service_pool.cc:224] Handling call
0709 12:57:30.760486 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:30.760962 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:30.761004 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:30.761574 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:57:31.644257 (+882683us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:31.644306 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:31.644439 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:31.832180 (+187741us) raft_consensus.cc:1685] finished
0709 12:57:31.832201 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:31.832356 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12926,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":882207,"threads_started":1,"wal-append.queue_time_us":1068724}
I20260709 12:57:31.815299 16686 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:31.845793 15968 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.006s	sys 0.034s
I20260709 12:57:32.030432 16568 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:31.985329 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.090813 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:31.936235 13535 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.013s	sys 0.017s
W20260709 12:57:32.091572 13535 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.021s	sys 0.018s
I20260709 12:57:32.091653 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:32.091545 13611 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:31.991220 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:32.093067 13540 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:31.990453 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.092942 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:31.992481 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.007608 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:32.008646 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:31.989588 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:32.034255 16408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.017s	sys 0.021s
I20260709 12:57:32.035326 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:32.035326 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:32.036672 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:31.986310 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:31.997069 16710 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:31.963485 14237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.009s	sys 0.013s
W20260709 12:57:32.037654 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.002s	sys 0.023s
W20260709 12:57:32.052129 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:32.051963 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:31.968441 16203 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:32.106858 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.055073 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:32.078055 16603 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:32.107702 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 121501us
I20260709 12:57:32.108194 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:32.109468 13693 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:32.060087 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:32.110164 13691 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:31.962320 16672 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:32.041358 16696 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:32.104840 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.002s	sys 0.023s
W20260709 12:57:32.062741 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 412526us
I20260709 12:57:32.134034 13615 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:57:32.063146 16606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.015s	sys 0.024s
I20260709 12:57:32.075456 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:32.076696 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:32.139600 13617 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:32.139742 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:32.138880 13619 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:32.144665 14209 leader_election.cc:400] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8.
I20260709 12:57:32.145030 14209 leader_election.cc:403] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:32.146215 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:32.075610 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 443189us
W20260709 12:57:32.102156 16408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.017s	sys 0.022s
I20260709 12:57:32.031129 13438 catalog_manager.cc:5697] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 4, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 4 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:32.149463 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.089118 14240 raft_consensus.cc:2368] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7.
I20260709 12:57:32.090911 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:32.151512 13536 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:31.992122 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:31.816242 16741 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:32.106374 16756 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:31.955363 13541 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:32.094561 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:32.075203 16703 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:32.094015 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:32.092729 13614 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:32.101313 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1423) took 1223 ms. Trace:
W20260709 12:57:32.138479 16606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.016s	sys 0.024s
I20260709 12:57:32.134778 16745 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:32.101415 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:32.184587 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 83255.3us
I20260709 12:57:32.185297 13584 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:32.053963 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 456107us
I20260709 12:57:32.188771 16603 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:32.192350 16768 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:32.193454 16768 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:32.102310 14243 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:57:32.107447 13540 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:32.199760 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.200959 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.201494 13540 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:32.202286 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.202786 13540 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:32.102612 13690 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:32.105273 13534 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:32.204847 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:32.203593 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.205542 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.206679 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.206950 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:31.960918 16752 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:32.210613 16752 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:32.135428 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 624168us
W20260709 12:57:32.104439 14237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.012s	sys 0.013s
I20260709 12:57:32.079651 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:32.155580 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:32.212796 13613 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:32.157792 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:32.213755 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 56063us
I20260709 12:57:32.093926 13537 raft_consensus.cc:2410] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:32.216527 16706 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:32.217718 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:32.170938 13542 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 6: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:32.172813 13616 raft_consensus.cc:2410] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:57:32.174216 13617 raft_consensus.cc:2410] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:32.175981 13535 rpcz_store.cc:276] 0709 12:57:30.877846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:30.878044 (+   198us) service_pool.cc:224] Handling call
0709 12:57:30.878414 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:30.878668 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:31.346548 (+467880us) log.cc:844] Serialized 14 byte log entry
0709 12:57:32.092477 (+745929us) log.cc:844] Serialized 14 byte log entry
0709 12:57:32.100966 (+  8489us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:32.101032 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:32.101086 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:32.101154 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:32.101263 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":773613,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":11843,"raft.queue_time_us":24852,"raft.run_wall_time_us":26191,"spinlock_wait_cycles":56016000,"thread_start_us":1210127,"threads_started":2,"wal-append.queue_time_us":729134}
I20260709 12:57:32.075443 16602 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:32.220402 16602 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.223016 16602 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:32.182076 13438 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:32.224220 13438 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 4 to 5, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66), now has a pending config: VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65), VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66), VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67), NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 5 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:57:32.226643 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:32.228008 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:32.088876 16589 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:32.105511 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:32.232206 13684 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:32.232398 13658 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:32.207381 13534 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:32.233755 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.234544 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:32.235713 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:32.144418 16635 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:32.100883 16568 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:32.101879 16357 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:57:32.243690 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:32.276396 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:32.267903 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.159349 16684 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:32.273300 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:32.171809 16756 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:32.273883 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:57:32.134563 16710 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.135082 16203 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.172770 16647 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:32.195329 16768 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.240222 16736 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:32.318332 13613 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:32.270066 16585 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:32.255154 16752 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:57:32.347432 13617 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:32.349119 13535 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:32.277494 13619 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:32.145190 16696 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.301023 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.353048 13539 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:32.170567 16498 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 12:57:32.354492 16696 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:32.319496 13616 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:32.283416 13509 leader_election.cc:336] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.044s (SENT)
W20260709 12:57:32.355719 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 72345us
I20260709 12:57:32.262024 16745 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:32.240530 16703 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:32.328343 16616 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:32.240659 16688 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:32.278322 16698 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:32.292274 16756 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:32.329779 16602 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:32.373101 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 105649us
I20260709 12:57:32.340963 16585 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:32.343887 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 111535us
I20260709 12:57:32.281010 13536 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:32.421770 13537 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:57:32.238420 14243 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:32.356240 16408 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:32.303843 16767 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:57:32.371181 16765 raft_consensus.cc:3055] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:32.371528 16422 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:32.249666 16746 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.269711 16686 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.384857 16616 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:32.465065 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.390842 16698 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:32.427670 16585 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:32.439988 16760 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 12:57:32.426314 16590 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8.
W20260709 12:57:32.482324 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 264718us
I20260709 12:57:32.434829 16765 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:32.488411 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 67756.8us
I20260709 12:57:32.489563 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:32.495743 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.470767 16616 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:32.498658 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.470590 16698 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:32.462284 16706 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:57:32.458478 16422 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:32.540055 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 138093us
I20260709 12:57:32.427465 16602 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.439459 16408 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:32.545032 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:32.557649 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 68153us
I20260709 12:57:32.558318 13536 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:32.548595 14240 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:32.562597 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:32.579031 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 92500.2us
I20260709 12:57:32.583084 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:32.589926 13582 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:32.601758 16568 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:32.602197 16703 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:32.502254 16357 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:32.573060 16784 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.109s	user 0.000s	sys 0.002s
I20260709 12:57:32.625957 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:32.626667 13539 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:32.603148 16739 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:57:32.630925 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 72870.6us
W20260709 12:57:32.634662 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 44830.5us
I20260709 12:57:32.637435 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:32.603111 16602 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:32.607750 16589 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:32.618806 16760 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:57:32.608569 16741 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:32.489530 16745 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:32.637981 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:33.019804 13612 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:33.017213 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.032178 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:32.645138 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:32.641994 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:33.047695 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:33.049279 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 412003us
I20260709 12:57:33.051141 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:33.051385 13543 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:57:33.054317 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 416371us
I20260709 12:57:33.055706 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:33.060405 13506 connection.cc:256] RPC call timeout handler was delayed by -1.3267821299996285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 12:57:32.813977 16555 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:33.017148 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:32.616716 16710 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:32.727017 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:33.079674 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.074553 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:32.809728 16794 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 12:57:32.811815 16776 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
W20260709 12:57:32.811121 16763 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:57:33.142999 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 92170us
I20260709 12:57:32.646472 16422 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:32.814262 16805 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:32.636032 16568 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:33.134321 13534 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:33.129401 16553 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:33.134848 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 55246.5us
I20260709 12:57:32.812393 16811 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:33.149219 16593 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:33.014395 16408 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.130456 16602 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:57:32.634133 16767 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:33.185415 16767 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.164546 16568 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:32.632344 16765 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.192198 16811 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.194314 16811 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.155818 16805 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:33.195340 16585 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.707s	user 0.006s	sys 0.000s
I20260709 12:57:33.196120 16805 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.197371 16805 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:33.143859 13582 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:33.199415 16603 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.010s	user 0.007s	sys 0.004s
W20260709 12:57:33.199904 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 56152.5us
I20260709 12:57:33.077613 16741 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:33.202347 16741 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:33.202734 16741 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:33.165149 16553 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.052642 16708 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:32.721777 16799 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:33.025943 16797 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.077975 16606 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:33.152283 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.055s	sys 0.061s
I20260709 12:57:32.714121 16739 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:33.122320 16769 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:32.812707 16806 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:33.133783 16808 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:33.187763 16767 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.191744 16568 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.217531 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.172259 16408 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:33.173292 16813 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:33.173810 16555 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:57:33.209038 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.056s	sys 0.061s
I20260709 12:57:33.213943 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:33.171339 13619 consensus_meta.cc:287] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.819s	user 0.004s	sys 0.003s
I20260709 12:57:33.222489 13619 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:33.212671 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.213231 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.212350 13691 raft_consensus.cc:2410] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:57:32.713153 16203 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.223454 13618 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:33.224125 13610 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:33.166980 16593 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:33.233887 14212 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:33.235195 14212 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.199829 16603 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:33.240613 13584 connection.cc:256] RPC call timeout handler was delayed by -1.0021743190009147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 12:57:33.212335 16763 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:57:33.210577 16808 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:33.243467 13618 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:33.244228 13610 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:33.207875 16553 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.229635 16555 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:33.237200 16593 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:33.239835 16815 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:33.257208 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.333573 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.208686 16797 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.239522 16812 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:33.213198 16585 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:33.210407 16806 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.210153 16769 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.212858 16767 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:57:33.341284 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:33.248490 16808 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:33.345221 13617 raft_consensus.cc:2410] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:57:33.230331 16408 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.347740 16408 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.331861 16647 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.209885 16739 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.343473 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:33.254274 16555 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:33.237839 16813 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.208217 16708 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.208458 16799 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.331403 16815 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:33.394428 13613 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:33.395726 16815 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:33.404145 13610 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:57:33.404114 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.407120 13618 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:57:33.361490 13615 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:33.362185 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:33.255968 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.375411 13617 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:57:33.426730 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 66826.1us
I20260709 12:57:33.427698 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:33.431049 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:33.436009 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 193207us
W20260709 12:57:33.436875 13584 leader_election.cc:336] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.550s (SENT)
W20260709 12:57:33.437870 13584 leader_election.cc:336] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.594s (SENT)
I20260709 12:57:33.438313 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:33.465267 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 209446us
I20260709 12:57:33.467847 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.469869 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.470813 13584 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.465047 16553 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:33.465140 16818 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:33.476048 16603 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:33.475852 16741 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.478083 16741 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.481788 16806 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.397576 16816 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.482924 16816 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.393217 16796 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.486639 16796 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.489480 16796 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.489902 16708 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.493330 16585 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.500458 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.358560 16647 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.502924 13613 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:33.469411 16568 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.498247 16606 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.503082 16814 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:33.514057 16616 consensus_meta.cc:287] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.990s	user 0.011s	sys 0.000s
I20260709 12:57:33.514631 16616 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.516172 13690 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:33.516713 16781 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.069s	user 0.005s	sys 0.005s
I20260709 12:57:33.496448 16408 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:33.469538 16185 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.518893 16185 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.520059 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 928) took 1777 ms. Trace:
I20260709 12:57:33.520888 16185 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.520557 14245 rpcz_store.cc:276] 0709 12:57:31.742391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:31.742575 (+   184us) service_pool.cc:224] Handling call
0709 12:57:31.742937 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:31.751129 (+  8192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:31.751187 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:31.779964 (+ 28777us) log.cc:844] Serialized 26 byte log entry
0709 12:57:32.094385 (+314421us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:32.094424 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:32.094485 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:33.516162 (+1421677us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:33.519644 (+  3482us) raft_consensus.cc:1685] finished
0709 12:57:33.519686 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:33.519984 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3180,"raft.queue_time_us":7761,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":3257,"spinlock_wait_cycles":86606592,"thread_start_us":290549,"threads_started":1,"wal-append.queue_time_us":655180}
I20260709 12:57:33.498518 16203 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:33.524891 16769 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.526301 16797 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.526115 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:33.527513 13690 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
W20260709 12:57:33.528498 16766 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.348s	user 0.000s	sys 0.036s
I20260709 12:57:33.529876 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:33.530310 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 58670.5us
I20260709 12:57:33.530640 13690 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:33.531031 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.508967 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.532793 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1428) took 2045 ms. Trace:
I20260709 12:57:33.532970 13611 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:33.533922 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.533190 13538 rpcz_store.cc:276] 0709 12:57:31.487003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:31.487158 (+   155us) service_pool.cc:224] Handling call
0709 12:57:31.487490 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:31.491637 (+  4147us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:31.491670 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:31.494468 (+  2798us) log.cc:844] Serialized 26 byte log entry
0709 12:57:31.970760 (+476292us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:31.970805 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:31.970900 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:33.527487 (+1556587us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:33.529536 (+  2049us) raft_consensus.cc:1685] finished
0709 12:57:33.529558 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:33.532733 (+  3175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":2809,"spinlock_wait_cycles":551205248,"thread_start_us":465004,"threads_started":1,"wal-append.queue_time_us":594683}
I20260709 12:57:33.534950 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:33.509549 16698 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.973s	user 0.011s	sys 0.000s
I20260709 12:57:33.536060 16698 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.506089 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.535420 16767 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:33.538589 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:33.506654 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 158849us
I20260709 12:57:33.506207 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 412) took 1450 ms. Trace:
I20260709 12:57:33.560976 13688 rpcz_store.cc:276] 0709 12:57:32.056204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.058379 (+  2175us) service_pool.cc:224] Handling call
0709 12:57:32.064154 (+  5775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:32.064589 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:32.064630 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:32.073899 (+  9269us) log.cc:844] Serialized 26 byte log entry
0709 12:57:32.180319 (+106420us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:32.260025 (+ 79706us) log.cc:844] Serialized 14 byte log entry
0709 12:57:33.469078 (+1209053us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b18000d2220. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287
0709 12:57:33.469282 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:33.469357 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:33.469395 (+    38us) raft_consensus.cc:1685] finished
0709 12:57:33.505951 (+ 36556us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:33.506132 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47625,"prepare.queue_time_us":79261,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":43804,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":34363,"spinlock_wait_cycles":33996672,"thread_start_us":142738,"threads_started":2,"wal-append.queue_time_us":736500}
I20260709 12:57:33.537809 13613 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:33.568059 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:33.569789 16790 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.123s	user 0.008s	sys 0.015s
I20260709 12:57:33.570187 16769 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.570705 16769 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.571496 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1341) took 1415 ms. Trace:
I20260709 12:57:33.572295 16744 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:33.572700 16744 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.573050 16744 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:33.573474 13685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2786) took 3710 ms (client timeout 1726 ms). Trace:
W20260709 12:57:33.573853 13685 rpcz_store.cc:269] 0709 12:57:29.862623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:29.865440 (+  2817us) service_pool.cc:224] Handling call
0709 12:57:33.573268 (+3707828us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400297cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:33.573408 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4525222144}
I20260709 12:57:33.575106 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 571) took 1986 ms. Trace:
I20260709 12:57:33.575466 13541 rpcz_store.cc:276] 0709 12:57:31.588885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:31.589041 (+   156us) service_pool.cc:224] Handling call
0709 12:57:31.589360 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:32.172011 (+582651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:33.574888 (+1402877us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54003b8654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:33.575050 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":798395136}
I20260709 12:57:33.576205 16698 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.577188 13687 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
W20260709 12:57:33.577328 16756 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.159s	user 0.011s	sys 0.000s
I20260709 12:57:33.577550 13687 pending_rounds.cc:85] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:57:33.577813 13687 pending_rounds.cc:107] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:57:33.578258 16799 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.578500 16769 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.582559 16706 consensus_queue.cc:1059] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:33.577749 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 413) took 1518 ms. Trace:
I20260709 12:57:33.583424 16706 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:33.583866 16706 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:33.584692 16706 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.585770 16706 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:33.586037 16811 raft_consensus.cc:2764] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:57:33.587435 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.588585 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:33.583295 13689 rpcz_store.cc:276] 0709 12:57:32.059027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.060083 (+  1056us) service_pool.cc:224] Handling call
0709 12:57:32.060361 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:32.060720 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:32.075400 (+ 14680us) log.cc:844] Serialized 14 byte log entry
0709 12:57:33.535664 (+1460264us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b180018a960. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a
0709 12:57:33.535844 (+   180us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:33.536328 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:57:33.536470 (+   142us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:33.536504 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:33.536568 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:33.577449 (+ 40881us) raft_consensus.cc:1685] finished
0709 12:57:33.577541 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:33.577678 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3273,"prepare.queue_time_us":16983,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":1615,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":31646,"spinlock_wait_cycles":27002112,"thread_start_us":306784,"threads_started":1,"wal-append.queue_time_us":299969}
I20260709 12:57:33.589046 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:33.498723 16745 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:33.589509 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.589797 16745 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:33.590234 16745 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:33.590526 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.592180 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.593065 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.578281 13687 raft_consensus.cc:2889] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:33.571837 13540 rpcz_store.cc:276] 0709 12:57:32.156265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.210391 (+ 54126us) service_pool.cc:224] Handling call
0709 12:57:32.219982 (+  9591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:32.220187 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:32.220510 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 12:57:32.327958 (+107448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:32.328023 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:32.328078 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:32.328148 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:33.571332 (+1243184us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b540028b750. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:33.571440 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59894,"prepare.queue_time_us":179,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":83,"raft.queue_time_us":70848,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3403,"spinlock_wait_cycles":63542912,"thread_start_us":48151,"threads_started":1,"wal-append.queue_time_us":146845}
I20260709 12:57:33.472138 16768 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:57:33.611353 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:33.612783 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:33.613915 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:33.641592 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:33.648900 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.650493 13536 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:57:33.513299 16647 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.505155 16816 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.507746 16818 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:33.657903 14211 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:33.513545 16813 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.637562 16806 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:33.660013 16812 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:33.510174 16568 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.681067 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:33.680907 16812 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.672729 16806 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:33.577750 16756 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.510756 16606 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.672436 16816 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:33.692075 13611 raft_consensus.cc:2393] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:33.683987 16806 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:33.683983 16742 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:33.701699 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.703090 13618 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:33.712108 16742 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:33.713274 16756 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.713132 16742 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.715955 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:33.714062 16818 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.718940 16769 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:33.722393 16606 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.724959 16555 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.725795 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.725725 16815 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.727088 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.727268 16555 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.728112 13612 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:33.699644 16698 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:57:33.730818 16808 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:33.731475 16672 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:33.735872 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:33.736428 13688 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:57:33.737172 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.700232 16593 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.740545 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.741469 13618 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 4.
I20260709 12:57:33.742385 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:33.743076 13618 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:33.738533 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.746650 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:33.748214 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:33.750180 13610 raft_consensus.cc:2368] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7.
I20260709 12:57:33.752805 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:33.753369 16555 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:33.757257 16736 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:33.756894 16555 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.757838 16736 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:33.759188 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.759560 16555 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:33.762261 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.738121 16616 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.737651 16815 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.773260 16744 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.775147 16808 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:33.775604 16761 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.603s	user 0.002s	sys 0.022s
I20260709 12:57:33.777361 16568 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:33.784343 16793 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.333s	user 0.004s	sys 0.015s
I20260709 12:57:33.789243 13538 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:33.790516 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.791088 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.791353 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.801453 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.801977 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.814810 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.815480 13618 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:33.803377 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:33.765997 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:33.820466 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.821626 13539 raft_consensus.cc:2410] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:57:33.823446 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.815762 13541 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:33.827210 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:33.827852 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.827888 13535 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:33.828873 13610 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:33.818665 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:33.829972 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.830154 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:33.816913 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.832062 13618 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:33.833695 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.833976 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:33.835121 13610 raft_consensus.cc:2410] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:57:33.835024 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.836879 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.797103 16706 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:33.805078 16602 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:33.840214 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.849153 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:33.850049 13543 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:33.852582 13535 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:33.863199 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:33.864097 13610 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:33.825280 16717 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:33.870039 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.869397 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.872691 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.838363 16568 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:33.884014 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.837508 16812 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:33.885272 13618 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:33.898319 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.895896 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:33.899794 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:33.840322 16555 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:33.909338 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.910032 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.910487 13618 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:33.912899 13543 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:33.914125 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.916328 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:33.916723 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:33.883200 16816 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:33.917409 14242 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:33.862766 16706 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.922333 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:33.906780 16568 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:33.868768 16602 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:33.925195 14239 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
W20260709 12:57:33.918176 16752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.546s	user 0.009s	sys 0.017s
I20260709 12:57:33.942538 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2821) took 1204 ms. Trace:
I20260709 12:57:33.943499 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:33.941680 16746 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:33.930595 14245 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:33.940824 16744 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:33.939380 16745 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.960160 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.908079 16555 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.962980 13658 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:33.942767 16752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.577s	user 0.010s	sys 0.017s
I20260709 12:57:33.969274 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.969964 13612 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:33.971243 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.971807 13612 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:33.972922 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.975090 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:33.978860 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.988512 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:33.990855 13612 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:33.981405 16799 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:33.994771 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:34.005519 16765 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:33.942986 13689 rpcz_store.cc:276] 0709 12:57:32.738639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.589369 (+850730us) service_pool.cc:224] Handling call
0709 12:57:33.589630 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:33.589950 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:33.589983 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:33.590471 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:57:33.693837 (+103366us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:33.941989 (+248152us) log.cc:844] Serialized 14 byte log entry
0709 12:57:33.942230 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:33.942309 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:33.942355 (+    46us) raft_consensus.cc:1685] finished
0709 12:57:33.942366 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:33.942485 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17328,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":40540,"thread_start_us":344390,"threads_started":2,"wal-append.queue_time_us":138395}
I20260709 12:57:34.011087 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.012809 13689 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:34.013521 16745 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:34.013715 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.014560 13689 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:34.015374 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:33.977207 16606 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:57:33.992282 16568 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.987465 16769 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:34.019738 16836 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:57:33.989287 16635 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.024086 16635 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.024626 16602 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:34.025362 16832 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:57:34.026630 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2822) took 1287 ms. Trace:
I20260709 12:57:34.028637 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 414) took 1967 ms. Trace:
I20260709 12:57:33.987840 16837 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:57:33.987481 15968 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:34.029157 13686 rpcz_store.cc:276] 0709 12:57:32.061186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.103328 (+ 42142us) service_pool.cc:224] Handling call
0709 12:57:32.103695 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:32.372084 (+268389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:32.372117 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:32.372638 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:57:32.501258 (+128620us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:32.501309 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.003896 (+1502587us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540002dd54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:34.028145 (+ 24249us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:34.028216 (+    71us) raft_consensus.cc:1685] finished
0709 12:57:34.028236 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:34.028543 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60421,"raft.queue_time_us":20847,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":11955,"spinlock_wait_cycles":574183680,"thread_start_us":39446,"threads_started":1,"wal-append.queue_time_us":279437}
I20260709 12:57:34.032410 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:34.032616 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.033450 13612 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:34.034250 13686 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:34.027020 13690 rpcz_store.cc:276] 0709 12:57:32.738949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.615259 (+876310us) service_pool.cc:224] Handling call
0709 12:57:33.660265 (+ 45006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:33.660616 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:33.660644 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:33.684917 (+ 24273us) log.cc:844] Serialized 26 byte log entry
0709 12:57:33.833605 (+148688us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:33.833660 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:33.833746 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:34.026416 (+192670us) raft_consensus.cc:1685] finished
0709 12:57:34.026439 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:34.026567 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":633,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":10546,"thread_start_us":148389,"threads_started":1,"wal-append.queue_time_us":236754}
I20260709 12:57:34.040405 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:34.042949 13610 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:34.015904 13689 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:34.050045 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.035939 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.054147 13686 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:34.056308 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:33.996189 16809 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:34.057734 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.059336 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.072395 14240 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:34.059769 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.060235 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.074000 14244 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:34.060886 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.061885 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.063325 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:34.075260 14242 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:34.075305 14243 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:34.076574 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.077271 14246 raft_consensus.cc:2393] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:34.077498 14243 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:34.010861 16736 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:33.995625 16799 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:34.068270 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.101577 13690 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
W20260709 12:57:34.068159 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 121969us
W20260709 12:57:34.069911 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 68448.7us
I20260709 12:57:34.105809 13658 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:34.051020 13689 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:34.107758 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.069555 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.070734 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.031747 15968 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:34.026803 16635 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.080956 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.082093 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.082679 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.083499 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.083855 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.084707 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.112322 13538 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 4.
I20260709 12:57:34.113404 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.114877 14238 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:34.030611 16806 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.117460 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.119145 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:34.125031 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.120174 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:34.126180 13689 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.121392 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:34.121714 13684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 409) took 2051 ms (client timeout 1507 ms). Trace:
W20260709 12:57:34.127853 13684 rpcz_store.cc:269] 0709 12:57:32.051453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.051620 (+   167us) service_pool.cc:224] Handling call
0709 12:57:34.102624 (+2051004us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540001c054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:34.102731 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291754880}
I20260709 12:57:34.129288 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.066907 16815 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.132094 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:34.133348 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.133455 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:34.134162 13540 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:34.132169 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.039494 16422 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:57:34.140934 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:34.141494 13690 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:34.138608 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.076370 16809 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:34.148893 13538 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:34.116809 16799 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:34.122854 16769 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:34.157420 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:34.158536 13618 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:34.164299 13540 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:34.170433 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.172111 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.172106 13690 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:34.174299 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:34.176664 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:34.181329 14239 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:34.182344 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:34.183029 13543 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:34.183984 14242 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:34.186034 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:34.121165 16506 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:57:34.136595 16635 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:57:34.198935 14242 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:57:34.183985 16757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.964s	user 0.027s	sys 0.027s
I20260709 12:57:34.194990 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.175522 16837 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:34.191861 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:34.201005 13542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1333) took 2052 ms (client timeout 2242 ms). Trace:
I20260709 12:57:34.178927 16706 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.189134 16811 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:34.204231 14244 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:57:34.175770 16742 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:34.182091 16770 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.733s	user 0.005s	sys 0.016s
W20260709 12:57:34.207707 13542 rpcz_store.cc:269] 0709 12:57:32.148462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.149151 (+   689us) service_pool.cc:224] Handling call
0709 12:57:34.200756 (+2051605us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b54000abc54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:34.200931 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22609152}
I20260709 12:57:34.140534 16815 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:34.178402 16800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.729s	user 0.018s	sys 0.044s
I20260709 12:57:34.179772 16769 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:34.193354 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.196785 16768 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:34.209461 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.180703 16371 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:34.192514 16756 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:57:34.208756 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 119682us
I20260709 12:57:34.217942 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.227205 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:34.207146 16757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.985s	user 0.027s	sys 0.028s
I20260709 12:57:34.199613 16842 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:34.232859 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:34.212458 16811 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:34.218627 16800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.770s	user 0.018s	sys 0.044s
I20260709 12:57:34.230463 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.247742 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.248654 16616 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.250128 16408 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:34.253568 16803 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:34.254001 16803 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:34.257727 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2824) took 1518 ms. Trace:
I20260709 12:57:34.258258 13688 rpcz_store.cc:276] 0709 12:57:32.739535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.642800 (+903265us) service_pool.cc:224] Handling call
0709 12:57:33.660945 (+ 18145us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:33.738499 (+ 77554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.257376 (+518877us) spinlock_profiling.cc:243] Waited 71.5 ms on lock 0x7b54002986d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:34.257648 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164553984}
I20260709 12:57:34.260429 16848 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:34.261044 16848 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:34.261458 16848 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:34.225010 16602 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:34.242826 16773 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:57:34.270402 16736 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.236151 16842 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:34.243355 16555 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.210033 16837 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:34.238583 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.221858 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:34.241429 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.242357 16808 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:34.219575 16792 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:34.268357 16602 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:34.211189 16809 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:34.280414 16769 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.272894 16357 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:34.272118 16842 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.220774 16768 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:34.259752 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.291976 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:34.254782 16803 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.296922 16837 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:34.233536 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 155186us
I20260709 12:57:34.235683 16606 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:34.288858 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:34.295802 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 54490.3us
I20260709 12:57:34.297374 16808 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:34.239472 16756 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:34.296824 13534 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:57:34.303937 16357 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:34.288604 16770 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:57:34.224064 16805 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:57:34.231123 16371 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.301916 16568 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.310105 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1344) took 2069 ms. Trace:
I20260709 12:57:34.338511 13508 leader_election.cc:400] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7.
I20260709 12:57:34.337986 13537 rpcz_store.cc:276] 0709 12:57:32.240603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.240839 (+   236us) service_pool.cc:224] Handling call
0709 12:57:32.421454 (+180615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:32.426909 (+  5455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.309863 (+1882954us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54003b8150. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:34.310032 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336921984}
I20260709 12:57:34.339038 13508 leader_election.cc:403] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:34.213353 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 499) took 1971 ms. Trace:
I20260709 12:57:34.298864 16845 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:34.339974 13614 rpcz_store.cc:276] 0709 12:57:32.242341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.264261 (+ 21920us) service_pool.cc:224] Handling call
0709 12:57:32.264558 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:32.264766 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:34.052084 (+1787318us) spinlock_profiling.cc:243] Waited 960 us on lock 0x7b1800168220. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf
0709 12:57:34.052223 (+   139us) log.cc:844] Serialized 14 byte log entry
0709 12:57:34.212211 (+159988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:34.212885 (+   674us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:34.213013 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.213125 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:34.213279 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":199006,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":8769,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":3115,"spinlock_wait_cycles":2208512,"thread_start_us":366592,"threads_started":2,"wal-append.queue_time_us":216504}
I20260709 12:57:34.316884 16606 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:34.175514 15968 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.316179 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:34.302451 16602 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:34.245333 16811 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.306897 16768 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.294814 16703 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:57:34.312111 16717 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:34.317950 16806 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.315061 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.320444 16808 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.330580 16357 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:34.333925 16842 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.352494 13686 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:34.390616 13689 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:34.348557 13614 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:34.389405 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 74430.5us
W20260709 12:57:34.405149 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 89074.9us
I20260709 12:57:34.410844 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.412949 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.414599 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.416136 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:34.370119 16841 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
W20260709 12:57:34.419872 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 303813us
I20260709 12:57:34.389717 16606 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.398064 16805 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:34.399987 16717 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:34.346377 16408 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.368449 16815 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.345839 16506 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 7
W20260709 12:57:34.437433 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 95909.1us
I20260709 12:57:34.439612 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 507) took 1205 ms. Trace:
I20260709 12:57:34.418723 16845 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:34.439682 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:34.444619 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:34.418853 16800 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:34.446425 13619 rpcz_store.cc:276] 0709 12:57:33.234625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.234837 (+   212us) service_pool.cc:224] Handling call
0709 12:57:33.235267 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:33.389121 (+153854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:33.389164 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:33.389222 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:33.389267 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.439311 (+1050044us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540026eb54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:34.439374 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:34.439537 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353248640}
I20260709 12:57:34.414849 15968 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:34.342298 16568 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.447149 16799 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.449573 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 498) took 2207 ms. Trace:
W20260709 12:57:34.437938 16841 raft_consensus.cc:1068] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Unable to promote non-voter 0f34cda059124f839711dbef61bb2c28: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:57:34.445999 16856 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:34.346670 16850 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 12:57:34.453289 13508 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:34.454842 13689 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:34.459875 13616 rpcz_store.cc:276] 0709 12:57:32.242035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.243716 (+  1681us) service_pool.cc:224] Handling call
0709 12:57:32.319326 (+ 75610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:32.357268 (+ 37942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.449338 (+2092070us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b540013bad0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:34.449488 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104145792}
I20260709 12:57:34.434582 16616 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:34.444576 16717 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:34.360318 16756 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:34.468340 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 276567us
I20260709 12:57:34.452029 16408 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.476418 16815 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.472528 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.484474 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:34.485756 16858 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:34.489347 16799 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:34.493266 16568 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.495174 16742 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:34.495905 16742 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:34.455605 16806 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:57:34.497378 16757 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:34.497813 16757 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:34.498588 16757 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:34.459270 16555 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:34.504721 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.499763 16757 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:34.487542 16858 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:34.471849 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:34.471134 16841 raft_consensus.cc:2764] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:57:34.481206 16593 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:34.483037 16773 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.628522 16357 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.629127 16773 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.630467 16506 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7.
I20260709 12:57:34.630641 16357 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.619167 16868 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:34.631212 16773 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.631603 16868 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:34.632100 16868 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:34.633615 16408 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.508384 16815 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.635133 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:34.636616 16815 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.500634 16848 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.626796 16746 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.739954 16746 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.742573 16746 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:34.544519 16555 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:34.744978 16555 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.626860 16606 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:34.702135 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:34.720228 16506 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:34.721544 16808 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.723347 16848 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.724061 16710 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:34.727818 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2818) took 1991 ms. Trace:
I20260709 12:57:34.731984 16768 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.734483 16602 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.737419 16811 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.738646 15968 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:34.623637 16717 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.771255 13693 rpcz_store.cc:276] 0709 12:57:32.736804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.736950 (+   146us) service_pool.cc:224] Handling call
0709 12:57:34.103614 (+1366664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:34.103968 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:34.104009 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:34.104066 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:34.104111 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.104188 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:34.727614 (+623426us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540001c050. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:34.727744 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3142278528}
I20260709 12:57:34.784382 13688 raft_consensus.cc:2393] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 6: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:34.768690 16506 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:34.789906 13690 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:34.811368 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.812417 13693 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:34.814293 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:34.814412 13685 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:34.784485 16773 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:34.790241 15968 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:34.784981 16859 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:34.775671 13689 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:34.835566 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:34.833374 13693 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:34.863034 16859 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:34.864282 16506 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.835105 15968 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:34.782706 16837 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:34.821902 16805 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:34.782510 16710 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:34.830430 16739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.322s	user 0.014s	sys 0.019s
I20260709 12:57:34.875134 16837 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.883867 14244 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:34.890734 16808 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.891630 16811 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:34.893224 16768 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.894750 16602 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.900535 16717 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:34.834291 16773 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:34.882050 16739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.374s	user 0.015s	sys 0.019s
I20260709 12:57:34.922982 16606 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.913697 16859 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:34.942334 16868 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:34.672680 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 201721us
I20260709 12:57:34.981953 13508 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:57:34.988124 13509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.003s	sys 0.021s
W20260709 12:57:34.989226 13509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.003s	sys 0.021s
W20260709 12:57:34.989849 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 552501us
I20260709 12:57:34.993202 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:34.988771 13507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.014s	sys 0.009s
I20260709 12:57:35.001235 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2631) took 1430 ms. Trace:
W20260709 12:57:35.001715 13507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.014s	sys 0.009s
I20260709 12:57:34.998282 13610 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:34.999419 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1560) took 1344 ms. Trace:
W20260709 12:57:34.998095 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.025s
I20260709 12:57:35.008100 16805 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:35.000003 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1556) took 1383 ms. Trace:
W20260709 12:57:35.017578 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.001s	sys 0.025s
I20260709 12:57:35.022395 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:35.022537 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:35.028345 14238 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:35.028826 16856 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:35.029551 16756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.011s	sys 0.020s
W20260709 12:57:35.030040 16756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.011s	sys 0.020s
I20260709 12:57:35.031648 16870 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:35.035295 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:35.035881 16710 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:35.036299 14239 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.036904 16850 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:35.039134 16872 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:35.040266 16872 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.044325 14238 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:35.046025 14241 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:35.009805 13619 rpcz_store.cc:276] 0709 12:57:33.655151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.467706 (+812555us) service_pool.cc:224] Handling call
0709 12:57:34.503660 (+ 35954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:34.504154 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:34.504195 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:34.504736 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:57:34.998315 (+493579us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:34.998415 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.998569 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:34.998720 (+   151us) raft_consensus.cc:1685] finished
0709 12:57:34.998741 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:34.999086 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2814,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":31081,"thread_start_us":380524,"threads_started":1,"wal-append.queue_time_us":417318}
I20260709 12:57:35.025269 16866 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:57:35.046021 16850 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:35.049297 16773 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.050166 16773 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:35.010075 13614 rpcz_store.cc:276] 0709 12:57:33.616184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.421201 (+805017us) service_pool.cc:224] Handling call
0709 12:57:34.421501 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:34.421733 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:34.443962 (+ 22229us) log.cc:844] Serialized 14 byte log entry
0709 12:57:34.998997 (+555035us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:34.999150 (+   153us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:34.999306 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:34.999439 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:34.999753 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14684,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":171338,"raft.run_cpu_time_us":635,"raft.run_wall_time_us":635,"thread_start_us":389236,"threads_started":1,"wal-append.queue_time_us":457346}
I20260709 12:57:35.050064 16850 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.001735 13612 rpcz_store.cc:276] 0709 12:57:33.570280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.043152 (+472872us) service_pool.cc:224] Handling call
0709 12:57:34.044307 (+  1155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:34.044553 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:34.051685 (+  7132us) log.cc:844] Serialized 14 byte log entry
0709 12:57:34.726782 (+675097us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b18002bf6c0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a 00007fa02e7bca2a
0709 12:57:35.000448 (+273666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:35.000592 (+   144us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:35.000731 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.000847 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.001051 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26832,"prepare.queue_time_us":6862,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"spinlock_wait_cycles":12738304,"thread_start_us":270861,"threads_started":1,"wal-append.queue_time_us":881416}
I20260709 12:57:35.053690 13619 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 8. Ops: []
I20260709 12:57:35.054205 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 515) took 1289 ms. Trace:
I20260709 12:57:35.054386 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1558) took 1399 ms. Trace:
W20260709 12:57:35.054937 13611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 512) took 1522 ms (client timeout 1858 ms). Trace:
I20260709 12:57:35.054638 16805 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.055891 16606 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:35.056228 13615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 502) took 2812 ms (client timeout 2016 ms). Trace:
I20260709 12:57:35.057233 13612 rpcz_store.cc:276] 0709 12:57:33.764483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.052501 (+1288018us) service_pool.cc:224] Handling call
0709 12:57:35.053220 (+   719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.053744 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:35.053781 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:35.053861 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:35.053896 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.053957 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.054099 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114048}
I20260709 12:57:35.061924 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2642) took 1352 ms. Trace:
I20260709 12:57:35.062304 13619 rpcz_store.cc:276] 0709 12:57:33.708593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.052376 (+1343783us) service_pool.cc:224] Handling call
0709 12:57:35.053220 (+   844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.060506 (+  7286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.060710 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43392}
W20260709 12:57:35.057400 13611 rpcz_store.cc:269] 0709 12:57:33.532357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.534729 (+  2372us) service_pool.cc:224] Handling call
0709 12:57:35.054493 (+1519764us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540013abd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.054760 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359481728}
W20260709 12:57:35.060065 13615 rpcz_store.cc:269] 0709 12:57:32.243279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.243421 (+   142us) service_pool.cc:224] Handling call
0709 12:57:35.055816 (+2812395us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b540020a154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.056014 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223038848}
I20260709 12:57:35.064220 13613 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:35.064464 13617 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:35.067751 13659 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.070317 16868 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.070446 16739 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:35.073715 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2807) took 4964 ms. Trace:
I20260709 12:57:35.057293 13618 rpcz_store.cc:276] 0709 12:57:33.654567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.913782 (+259215us) service_pool.cc:224] Handling call
0709 12:57:35.053786 (+1140004us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540030a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.054130 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556784896}
I20260709 12:57:35.076114 13659 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:35.076807 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 592354us
I20260709 12:57:35.080631 16870 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:35.081266 13687 rpcz_store.cc:276] 0709 12:57:30.109245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:30.175205 (+ 65960us) service_pool.cc:224] Handling call
0709 12:57:33.576794 (+3401589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:33.594984 (+ 18190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.073225 (+1478241us) spinlock_profiling.cc:243] Waited 3.4 s on lock 0x7b5400297cd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.073480 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7822031744}
I20260709 12:57:35.085853 13687 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:35.086489 16739 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:35.086623 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2830) took 2028 ms. Trace:
I20260709 12:57:35.087512 13687 rpcz_store.cc:276] 0709 12:57:33.058017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.084895 (+2026878us) service_pool.cc:224] Handling call
0709 12:57:35.085452 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.086282 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.086527 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.090548 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2828) took 2258 ms. Trace:
W20260709 12:57:35.092113 16762 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 2.764s	user 0.011s	sys 0.030s
I20260709 12:57:35.092557 16872 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.095697 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 592) took 4250 ms. Trace:
I20260709 12:57:35.096443 14237 rpcz_store.cc:276] 0709 12:57:30.845374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:30.848038 (+  2664us) service_pool.cc:224] Handling call
0709 12:57:30.848453 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:31.317224 (+468771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:31.317258 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:31.329814 (+ 12556us) log.cc:844] Serialized 26 byte log entry
0709 12:57:32.212295 (+882481us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:32.212348 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:32.212438 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:35.090815 (+2878377us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:35.095187 (+  4372us) raft_consensus.cc:1685] finished
0709 12:57:35.095207 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.095472 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7245,"raft.queue_time_us":17174,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":28326,"spinlock_wait_cycles":301861760,"thread_start_us":1226320,"threads_started":2,"wal-append.queue_time_us":741254}
I20260709 12:57:35.106637 13538 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:35.110353 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:35.111534 13538 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
W20260709 12:57:35.111521 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 118551us
I20260709 12:57:35.112751 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.113669 14238 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:35.115134 13509 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:35.051347 16773 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:35.116245 16858 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:57:35.091173 13693 rpcz_store.cc:276] 0709 12:57:32.855459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.884452 (+2028993us) service_pool.cc:224] Handling call
0709 12:57:35.023548 (+139096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.024182 (+   634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:35.024227 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:35.024310 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:35.024342 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.024424 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.025604 (+  1180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.124670 16773 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:35.121335 14239 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:35.063170 16606 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.086234 16805 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.132345 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.133070 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.131110 13584 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:35.148391 14246 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:35.152379 14241 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:35.154981 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:35.158802 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:35.150099 16606 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.161195 16858 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:35.161088 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.161796 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:35.162967 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2826) took 2422 ms. Trace:
I20260709 12:57:35.168689 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.172343 16805 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:35.172575 13693 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:57:35.172843 16792 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:35.173205 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:35.176064 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.002s	sys 0.009s
W20260709 12:57:35.184878 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.003s	sys 0.010s
I20260709 12:57:35.187667 16845 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:35.185828 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2832) took 2109 ms. Trace:
I20260709 12:57:35.188905 13693 rpcz_store.cc:276] 0709 12:57:33.074140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.154220 (+2080080us) service_pool.cc:224] Handling call
0709 12:57:35.172074 (+ 17854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:35.182295 (+ 10221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.183409 (+  1114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.181165 16858 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:35.182013 16739 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:35.190044 13538 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:35.185479 16792 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:35.193703 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:35.197371 16861 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:35.197882 16861 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:35.198181 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:35.198654 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:35.200459 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:35.204432 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 411) took 3148 ms. Trace:
I20260709 12:57:35.204265 16817 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:35.181273 16805 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:35.204849 13692 rpcz_store.cc:276] 0709 12:57:32.055897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:32.056034 (+   137us) service_pool.cc:224] Handling call
0709 12:57:32.056381 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:32.056880 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:32.063334 (+  6454us) log.cc:844] Serialized 14 byte log entry
0709 12:57:32.181915 (+118581us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:32.182462 (+   547us) log.cc:844] Serialized 52 byte log entry
0709 12:57:32.182588 (+   126us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:32.260103 (+ 77515us) log.cc:844] Serialized 14 byte log entry
0709 12:57:32.393101 (+132998us) log.cc:844] Serialized 14 byte log entry
0709 12:57:32.462172 (+ 69071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:32.462250 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:33.173663 (+711413us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4800212958. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:57:35.065445 (+1891782us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b18003c1000. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:35.190968 (+125523us) raft_consensus.cc:1685] finished
0709 12:57:35.190991 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.204264 (+ 13273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":266779,"prepare.queue_time_us":99152,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":4250,"raft.queue_time_us":189612,"raft.run_cpu_time_us":9966,"raft.run_wall_time_us":882772,"spinlock_wait_cycles":38573056,"thread_start_us":164997,"threads_started":2,"wal-append.queue_time_us":264184}
I20260709 12:57:35.206838 16805 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:35.207418 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 88171.4us
I20260709 12:57:35.192775 16858 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:35.211077 16703 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:35.214390 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:35.189934 16845 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:35.216111 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:35.216838 13540 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:35.218168 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2827) took 2450 ms. Trace:
I20260709 12:57:35.211639 16703 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:35.222191 16703 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.218592 13690 rpcz_store.cc:276] 0709 12:57:32.767396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.295650 (+1528254us) service_pool.cc:224] Handling call
0709 12:57:34.295959 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:34.791619 (+495660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.217605 (+425986us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540002ce54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:35.217992 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135527808}
I20260709 12:57:35.216133 16845 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:35.177165 13684 rpcz_store.cc:276] 0709 12:57:32.766893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.140897 (+1374004us) service_pool.cc:224] Handling call
0709 12:57:34.141309 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:34.141749 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:34.141790 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:34.142339 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:57:34.313200 (+170861us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:34.334762 (+ 21562us) log.cc:844] Serialized 14 byte log entry
0709 12:57:35.072514 (+737752us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b180023bf60. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 12:57:35.162097 (+ 89583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.162228 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:35.162338 (+   110us) raft_consensus.cc:1685] finished
0709 12:57:35.162487 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.162734 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19829,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":19395,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":2352,"spinlock_wait_cycles":3075456,"thread_start_us":205738,"threads_started":2,"wal-append.queue_time_us":234685}
I20260709 12:57:35.220687 13536 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:57:35.229775 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 154053us
I20260709 12:57:35.230576 13582 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.230747 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:35.230955 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.231823 16555 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:35.232076 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:35.233995 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.234383 16590 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:35.234891 16590 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:35.235260 16590 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:35.238401 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.239593 16765 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:35.240916 16765 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:35.241660 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.242699 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.244125 13684 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:57:35.244333 13693 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:57:35.250490 16506 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:57:35.192507 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 776862us
I20260709 12:57:35.199239 16861 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.266918 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1410) took 1320 ms. Trace:
I20260709 12:57:35.267306 13684 rpcz_store.cc:276] 0709 12:57:33.946648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.243200 (+1296552us) service_pool.cc:224] Handling call
0709 12:57:35.243723 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:35.266675 (+ 22952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.266862 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.270180 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2831) took 2211 ms. Trace:
I20260709 12:57:35.269618 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2833) took 2190 ms. Trace:
I20260709 12:57:35.272439 13693 rpcz_store.cc:276] 0709 12:57:33.074468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.228906 (+2154438us) service_pool.cc:224] Handling call
0709 12:57:35.243876 (+ 14970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:35.264807 (+ 20931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.265036 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77056}
I20260709 12:57:35.270714 13687 rpcz_store.cc:276] 0709 12:57:33.058327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.100115 (+2041788us) service_pool.cc:224] Handling call
0709 12:57:35.100717 (+   602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:35.101436 (+   719us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:35.101477 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:35.130689 (+ 29212us) log.cc:844] Serialized 26 byte log entry
0709 12:57:35.157363 (+ 26674us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:35.157411 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.157453 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:35.269978 (+112525us) raft_consensus.cc:1685] finished
0709 12:57:35.270004 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.270125 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":26267,"threads_started":1,"wal-append.queue_time_us":92093}
W20260709 12:57:35.276963 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 162107us
W20260709 12:57:35.275389 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.269208 16555 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:35.292222 13693 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:57:35.293859 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1413) took 1346 ms. Trace:
I20260709 12:57:35.294258 13693 rpcz_store.cc:276] 0709 12:57:33.947427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.278431 (+1331004us) service_pool.cc:224] Handling call
0709 12:57:35.289578 (+ 11147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.292631 (+  3053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.293768 (+  1137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.294978 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1399) took 1430 ms. Trace:
I20260709 12:57:35.295437 13686 rpcz_store.cc:276] 0709 12:57:33.864333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.083190 (+218857us) service_pool.cc:224] Handling call
0709 12:57:35.294574 (+1211384us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400244854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.294746 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536114944}
I20260709 12:57:35.296387 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 418) took 1175 ms. Trace:
I20260709 12:57:35.296026 15968 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.297472 13686 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:35.298199 16817 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:35.298220 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 424) took 1128 ms. Trace:
I20260709 12:57:35.291163 16792 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.299722 16371 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.300230 16817 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:35.300724 16817 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:35.296818 13693 rpcz_store.cc:276] 0709 12:57:34.120948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.295134 (+1174186us) service_pool.cc:224] Handling call
0709 12:57:35.295641 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.295982 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:35.296007 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:35.296076 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:35.296145 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.296202 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.296310 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.298930 13686 rpcz_store.cc:276] 0709 12:57:34.169534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.296649 (+1127115us) service_pool.cc:224] Handling call
0709 12:57:35.297046 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.297981 (+   935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.298170 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.274113 16739 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:35.319386 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 85583.1us
W20260709 12:57:35.320703 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 77071.9us
I20260709 12:57:35.298629 15968 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:35.322543 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 80932.8us
W20260709 12:57:35.322046 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.332173 16739 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.334681 16739 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.335752 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2636) took 1764 ms. Trace:
I20260709 12:57:35.337211 13684 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:35.352881 16881 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:35.353847 16881 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:35.356755 14246 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:57:35.361925 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1412) took 1390 ms. Trace:
I20260709 12:57:35.336189 13610 rpcz_store.cc:276] 0709 12:57:33.571762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.419377 (+847615us) service_pool.cc:224] Handling call
0709 12:57:34.419620 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.020521 (+600901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.335425 (+314904us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54002fc6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:35.335691 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328881536}
I20260709 12:57:35.362365 13684 rpcz_store.cc:276] 0709 12:57:33.947174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.281885 (+1334711us) service_pool.cc:224] Handling call
0709 12:57:35.322266 (+ 40381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.337678 (+ 15412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.337863 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:35.368641 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:35.369206 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.370677 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.372848 13684 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:35.373104 13659 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.373577 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2838) took 1071 ms. Trace:
I20260709 12:57:35.373968 16773 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:35.374593 16773 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:35.374792 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:35.375602 16773 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.373281 13509 leader_election.cc:400] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7.
I20260709 12:57:35.373988 13684 rpcz_store.cc:276] 0709 12:57:34.302091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.372059 (+1069968us) service_pool.cc:224] Handling call
0709 12:57:35.372568 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.373316 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.373526 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.385281 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2837) took 1084 ms. Trace:
I20260709 12:57:35.393617 13686 rpcz_store.cc:276] 0709 12:57:34.300470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.322334 (+1021864us) service_pool.cc:224] Handling call
0709 12:57:35.363452 (+ 41118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.384942 (+ 21490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:35.384971 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:35.385013 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:35.385049 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.385104 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.385214 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.384944 16703 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:35.382555 16739 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:35.408752 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.355422 16881 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:35.412380 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 96151us
I20260709 12:57:35.413616 16861 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.372946 16845 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.433974 14246 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:57:35.428238 16739 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:35.384580 16856 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:57:35.448850 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.453510 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.450196 16739 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:35.463218 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 425) took 1157 ms. Trace:
I20260709 12:57:35.463943 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2630) took 1894 ms. Trace:
I20260709 12:57:35.466650 16792 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.470467 14246 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:35.471226 13509 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.481666 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:35.486302 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 77732.5us
I20260709 12:57:35.487396 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.487612 16635 consensus_queue.cc:1059] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:57:35.488765 16635 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:35.489159 16635 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:35.490157 14238 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:35.490303 16635 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:35.461781 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:35.484025 13686 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:35.475849 13613 rpcz_store.cc:276] 0709 12:57:33.569933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.914610 (+344677us) service_pool.cc:224] Handling call
0709 12:57:35.059152 (+1144542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.066108 (+  6956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.463636 (+397528us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540013abd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.463833 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2626417408}
I20260709 12:57:35.475369 13684 rpcz_store.cc:276] 0709 12:57:34.305593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.385943 (+1080350us) service_pool.cc:224] Handling call
0709 12:57:35.395424 (+  9481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.395742 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:35.396099 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 12:57:35.454482 (+ 58383us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:35.454803 (+   321us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:35.462965 (+  8162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.463036 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.463160 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":735,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":3498,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":54552,"threads_started":1}
I20260709 12:57:35.492013 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 426) took 1180 ms. Trace:
I20260709 12:57:35.496134 13686 rpcz_store.cc:276] 0709 12:57:34.311731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.394650 (+1082919us) service_pool.cc:224] Handling call
0709 12:57:35.451849 (+ 57199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.491714 (+ 39865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.491884 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.497675 16635 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.498090 16813 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:57:35.502988 13659 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.503762 16818 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:35.505054 16606 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:35.505566 16717 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:35.506027 16717 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:35.506465 16717 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:35.506784 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.507701 16606 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:35.508107 16606 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:35.508446 16606 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.510676 16858 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:35.516572 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 69742.5us
I20260709 12:57:35.518376 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:35.519812 16898 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:57:35.520700 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 421) took 1372 ms. Trace:
I20260709 12:57:35.521327 16422 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:35.522320 16422 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:35.527243 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:35.521111 13688 rpcz_store.cc:276] 0709 12:57:34.148273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.259389 (+111116us) service_pool.cc:224] Handling call
0709 12:57:35.483299 (+1223910us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b540034e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.520532 (+ 37233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099826432}
I20260709 12:57:35.528548 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:35.533355 16856 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:35.533977 16856 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.536012 13693 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:35.537787 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2836) took 1237 ms. Trace:
I20260709 12:57:35.538154 13693 rpcz_store.cc:276] 0709 12:57:34.300231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.317904 (+1017673us) service_pool.cc:224] Handling call
0709 12:57:35.495768 (+177864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.536499 (+ 40731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.537567 (+  1068us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540034e9d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.537722 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390074240}
I20260709 12:57:35.543536 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:35.544267 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.544695 14241 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:35.546331 16732 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:35.546797 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.547318 16817 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.548161 16422 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:57:35.550141 16889 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.197s	user 0.004s	sys 0.001s
W20260709 12:57:35.550657 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 104497us
I20260709 12:57:35.551203 16371 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:35.508368 16845 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:35.552043 16850 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:35.553156 16371 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:35.553385 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:35.553658 16371 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:35.555735 16371 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.555989 16590 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.557094 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:35.557690 13538 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:35.550205 16817 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:35.553293 16850 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:35.561848 16850 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:35.562724 16850 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.564481 16850 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:35.566421 16805 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.557118 16371 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:35.570025 14245 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:35.570405 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 419) took 1433 ms. Trace:
I20260709 12:57:35.570763 13685 rpcz_store.cc:276] 0709 12:57:34.137279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:34.265061 (+127782us) service_pool.cc:224] Handling call
0709 12:57:35.570040 (+1304979us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400297cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.570247 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135795584}
I20260709 12:57:35.574337 13692 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:35.574993 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1397) took 1737 ms. Trace:
I20260709 12:57:35.575289 13692 rpcz_store.cc:276] 0709 12:57:33.837826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.496108 (+1658282us) service_pool.cc:224] Handling call
0709 12:57:35.574066 (+ 77958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.574764 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.574945 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.575760 16803 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.454661 16873 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.123s
W20260709 12:57:35.577847 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.579849 16859 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:35.580276 16859 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:35.581274 16859 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.582758 16859 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:35.583199 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:35.583590 16873 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:57:35.584930 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 138332us
I20260709 12:57:35.589591 13689 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:35.590250 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1405) took 1646 ms. Trace:
I20260709 12:57:35.590569 13689 rpcz_store.cc:276] 0709 12:57:33.945164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.588957 (+1643793us) service_pool.cc:224] Handling call
0709 12:57:35.589277 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.590006 (+   729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.590194 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:35.590675 16803 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:35.497548 16602 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:35.591871 16602 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.594180 16602 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:35.595850 13538 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
W20260709 12:57:35.597760 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56541.4us
W20260709 12:57:35.621363 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 69495.9us
I20260709 12:57:35.623818 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.622155 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:35.626420 16878 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:35.626925 16878 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:35.627312 16878 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.628087 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:35.629241 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:35.630780 16773 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:35.646418 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 93138.3us
I20260709 12:57:35.650292 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2626) took 2223 ms. Trace:
I20260709 12:57:35.644845 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.637661 16773 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:35.634316 16817 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:35.652112 13617 rpcz_store.cc:276] 0709 12:57:33.425944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:33.426102 (+   158us) service_pool.cc:224] Handling call
0709 12:57:35.061501 (+1635399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.066249 (+  4748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.648915 (+582666us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540020a150. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:35.649116 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3749915520}
I20260709 12:57:35.639671 16765 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:35.639376 16602 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:57:35.653843 16773 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:35.674135 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 91022.2us
W20260709 12:57:35.677356 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53653.5us
I20260709 12:57:35.660682 16765 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:35.662071 16703 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:35.676836 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:35.663846 16817 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:35.637480 16856 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:35.695096 13618 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:35.682632 16765 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.700634 13615 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 5. Current term is 6. Ops: []
W20260709 12:57:35.702072 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:35.702664 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.704325 14209 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:35.688802 16805 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.721148 13616 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:35.728861 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.722853 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.714844 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:35.732237 13536 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:35.723382 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.681650 16606 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.715039 13613 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:35.730690 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:35.759261 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.710543 16858 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:35.781769 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:35.781687 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.789913 13615 raft_consensus.cc:2393] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:35.722550 16752 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:35.782574 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:35.791975 13693 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:35.804018 13611 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:35.808588 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 111133us
I20260709 12:57:35.809548 16744 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:35.810626 16805 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:35.742046 16710 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:35.811209 16805 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.811930 16710 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.814023 16710 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:35.816018 16506 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.816866 16606 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:35.817555 16606 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:35.822011 16817 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.825471 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:35.828632 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.829305 13613 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:35.829753 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:35.832397 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:35.832608 16899 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:57:35.833143 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:35.833707 14242 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.834558 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:35.835115 16894 log.cc:927] Time spent T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.382s	user 0.000s	sys 0.003s
I20260709 12:57:35.835125 13613 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:35.837846 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:35.838435 13611 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:35.841413 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.781792 16616 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:57:35.843166 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54223.9us
I20260709 12:57:35.843644 16616 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:35.844291 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.844700 16616 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.845218 13610 raft_consensus.cc:2410] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:57:35.846383 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.847105 13614 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:35.848476 16870 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:35.849320 16870 consensus_queue.cc:1243] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:57:35.849797 16870 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:35.850132 16870 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:35.850924 16870 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.790956 16590 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:35.852185 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.845278 16739 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.856720 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:35.857559 13542 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:35.802505 16752 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:35.858793 16752 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.840555 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:35.801177 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:35.809577 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:35.870977 13613 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:35.815243 16773 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.859807 16752 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:35.889353 13536 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:35.889940 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:35.784066 16881 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:35.937233 13618 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:35.946312 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1407) took 1996 ms. Trace:
I20260709 12:57:35.948400 13542 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:35.947772 16744 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:35.950178 16744 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:35.952224 16900 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.325s	user 0.004s	sys 0.000s
I20260709 12:57:35.957098 16744 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:35.961520 16861 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.002s
I20260709 12:57:35.944446 16881 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:35.970299 16881 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:35.972548 13611 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:35.974102 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:35.974704 13689 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:35.976384 16805 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:35.981828 16811 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:35.982607 16811 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:35.984062 16774 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:35.985599 16506 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:35.988116 16870 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:35.989622 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:35.990666 16800 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:35.990899 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:35.991241 16800 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:35.992151 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:35.993932 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:35.994416 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:35.995662 14242 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:36.004405 13689 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:57:35.940836 16873 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:36.014145 13684 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:36.013878 13619 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:36.015511 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:35.946908 13690 rpcz_store.cc:276] 0709 12:57:33.945815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:35.586392 (+1640577us) service_pool.cc:224] Handling call
0709 12:57:35.586741 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:35.587026 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:35.587453 (+   427us) log.cc:844] Serialized 14 byte log entry
0709 12:57:35.937360 (+349907us) log.cc:844] Serialized 14 byte log entry
0709 12:57:35.941543 (+  4183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:35.941612 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:35.941672 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:35.941834 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:35.941978 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3541,"prepare.queue_time_us":3761,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":82,"raft.queue_time_us":128123,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":43921,"thread_start_us":290052,"threads_started":2,"wal-append.queue_time_us":232142}
I20260709 12:57:36.008562 14212 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.021653 13616 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:36.025550 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.027151 14238 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:57:36.029593 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 485677us
I20260709 12:57:36.016803 16739 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:36.039762 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.044193 13612 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
W20260709 12:57:36.047746 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 238267us
W20260709 12:57:36.060518 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 128866us
W20260709 12:57:36.061091 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:36.062788 14209 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.067133 16850 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:36.067816 16850 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.070075 16850 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.070325 16811 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.070938 16869 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:36.055642 16873 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.071640 16869 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.073254 16873 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:36.073873 16869 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.074798 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.078781 16800 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.079326 13691 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:57:36.051905 16911 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:36.076939 16869 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:36.081507 16869 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.084765 16797 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:36.085371 16797 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.086452 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:36.087517 16797 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:36.087996 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:36.088573 13538 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:36.090613 16859 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.004s
I20260709 12:57:36.096714 16773 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.062422 16739 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:36.098201 16739 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.100338 16739 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:36.102005 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:36.103394 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.104234 16850 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:57:36.104621 16850 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:36.104996 16850 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:36.022186 16878 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.107611 16371 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:36.108754 16878 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:36.109467 13539 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:36.104871 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.112004 16856 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:36.112761 16872 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:57:36.114172 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 84239.4us
I20260709 12:57:36.115409 16872 consensus_queue.cc:1059] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:36.117235 16506 raft_consensus.cc:3055] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:36.117655 16506 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.119009 16422 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:36.119660 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:36.120422 16635 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:36.120630 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.120949 16635 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.121337 13612 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:36.121974 16635 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.123333 16635 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:36.123426 16850 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:36.124447 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 52702.3us
I20260709 12:57:36.125660 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:36.120441 13536 raft_consensus.cc:606] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.127686 16912 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:36.128548 16912 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:36.120318 16422 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:36.129557 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.129685 16912 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.131523 13619 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:57:36.132324 13618 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: [5.8-7.9]
I20260709 12:57:36.127275 16744 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:36.127271 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.132614 16706 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:36.133852 16706 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:36.134186 16706 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:36.134727 14239 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:36.134838 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.135035 16706 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.135499 13616 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:36.137528 13509 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:36.147351 16732 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:36.147904 16732 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:36.148366 16732 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.120609 16800 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:36.149430 16800 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.137478 16773 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:36.151821 16800 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.152437 16773 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:36.152853 16773 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:36.153928 16773 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.155298 16773 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.132529 16911 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:36.169368 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:36.170379 16911 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.139081 13612 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:57:36.172760 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.173002 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.174621 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.175288 13612 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:36.139183 16792 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:36.135866 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:36.176548 16792 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:36.177687 14238 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:36.140110 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.140435 16422 consensus_queue.cc:1243] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:57:36.141573 13611 pending_rounds.cc:85] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 4
I20260709 12:57:36.180469 13611 pending_rounds.cc:107] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:57:36.180955 13543 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:36.141778 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.141961 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.181558 13543 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.181618 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.142967 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.182143 13614 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:36.182452 13610 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:36.182518 13543 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.125816 16850 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.127558 13536 raft_consensus.cc:614] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:57:36.158121 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.155658 13538 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:36.159965 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:36.161221 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:36.158859 16732 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.161173 16878 consensus_queue.cc:1059] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:36.185451 13534 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:36.177726 16792 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.186131 16878 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:36.186690 16878 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.186970 16732 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.121068 16506 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.188248 16506 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.144486 16706 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.162304 16774 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:36.167212 16765 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.190805 16765 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:36.191633 13685 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:36.182461 13612 raft_consensus.cc:2368] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6.
I20260709 12:57:36.186637 16792 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:36.195389 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.168345 16703 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:36.197453 13542 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:36.085225 16869 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:36.181044 13611 raft_consensus.cc:2889] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:36.181821 13613 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:36.183821 13543 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.187784 16878 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:36.168808 16717 consensus_meta.cc:287] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.001s
I20260709 12:57:36.162235 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.199330 13584 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.196259 13610 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:36.203094 16878 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.166354 13617 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:36.204926 13534 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:36.209753 13616 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:57:36.217402 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:36.220208 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47585.4us
I20260709 12:57:36.223449 13613 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 5.
I20260709 12:57:36.197496 16590 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:36.228367 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.232999 13685 raft_consensus.cc:2393] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:36.236238 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.237656 13618 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
W20260709 12:57:36.238600 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:36.213330 16873 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:36.245272 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.246382 13534 raft_consensus.cc:2410] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:57:36.215170 16710 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:36.255017 13691 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:36.259876 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 211251us
I20260709 12:57:36.260691 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.260092 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.262558 13685 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.268231 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.239995 16635 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:36.215648 16811 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:57:36.272702 13542 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:36.208684 16717 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:36.263660 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 101544us
I20260709 12:57:36.277088 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:36.285339 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.259181 16803 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:36.290249 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:36.290243 13689 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:36.259671 16873 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:36.293365 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.294495 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.263770 16590 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:36.294104 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.297379 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.302098 13618 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.298852 13686 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:36.304984 13509 leader_election.cc:400] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6.
I20260709 12:57:36.305639 13509 leader_election.cc:403] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:57:36.301869 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.306810 13689 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:36.269161 16555 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:57:36.296984 16870 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:36.293059 16920 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:36.313723 13437 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 3 to 5, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 5 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:36.312377 16774 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6.
I20260709 12:57:36.316197 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:36.313445 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 83104.9us
I20260709 12:57:36.317835 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.324940 14209 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:36.325984 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:36.326951 16698 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:36.327349 16767 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:36.327751 16767 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:36.299823 16803 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.329878 16803 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:36.331074 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.332042 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:36.332729 14242 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:36.305990 16506 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:36.327353 16698 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:36.335632 16506 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:36.335932 16698 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:36.336073 16506 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:36.337878 16873 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.335408 16817 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:36.340037 16555 consensus_queue.cc:1243] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:57:36.341022 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.355759 13542 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:57:36.342885 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.358094 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:36.359619 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53402.4us
I20260709 12:57:36.318490 16911 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:36.360533 13509 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.373569 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.376644 13614 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:57:36.318312 16920 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:36.380189 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:36.381929 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:36.332748 16922 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:36.385044 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.398685 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.359786 16803 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:36.403244 13618 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:36.372843 16703 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:36.384235 13689 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:36.410883 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.409165 13509 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.413924 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.414683 13689 raft_consensus.cc:2393] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:36.417706 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.379120 16869 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:57:36.355063 16870 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:36.421875 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:36.381950 16635 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:36.379345 16920 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:36.430656 16914 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:36.431522 16922 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:36.439783 13534 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:36.440287 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.441776 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.443080 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.443666 13539 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:36.425125 15968 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:36.445137 13535 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:36.444142 13437 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 5, term changed from 5 to 6, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to <none>, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added, no longer has a pending config: VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65), VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66), VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67), NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 6 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:57:36.446394 13437 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 4 to 5, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:36.449545 16698 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.451449 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.451886 16698 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.405122 16765 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:36.428586 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.439867 13538 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:57:36.455472 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:36.408280 16803 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:36.457053 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 101171us
I20260709 12:57:36.410019 16797 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:36.465302 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.452684 13437 catalog_manager.cc:5697] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 5 to 7, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:36.466233 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:36.468824 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 63842.7us
W20260709 12:57:36.472750 13438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49892 (request call id 318) took 13258 ms (client timeout 15000 ms). Trace:
I20260709 12:57:36.473892 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.475371 14209 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.460837 15968 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.476774 15968 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.477553 16710 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:36.479049 16710 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:36.480336 13437 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 6, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 6 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:36.482372 13437 catalog_manager.cc:5697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:36.466104 16803 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:36.464776 16717 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:36.485057 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.487003 14212 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.452872 16869 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.487648 16744 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:36.488135 16590 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:36.488687 16744 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:36.488994 16930 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:36.489460 16930 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:36.489786 16930 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:36.490375 16744 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:36.490782 16744 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:36.491540 16744 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.488739 16590 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:36.492336 16706 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:36.492964 16590 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.490386 16767 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.496753 16744 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:36.497224 16870 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.496752 16767 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:36.501328 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.501809 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.502332 13685 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:36.470923 16635 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.503976 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:36.504817 13618 raft_consensus.cc:2410] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:57:36.504323 16635 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.505535 13687 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:57:36.502179 16710 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:36.519022 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.519654 13543 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:36.529006 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.529886 13689 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:36.528697 13437 catalog_manager.cc:5697] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 5, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 5 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:36.488704 16869 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.517015 16698 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:36.510463 16797 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:36.593564 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.594587 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.571321 16506 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.600356 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:36.600968 13617 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:36.604465 16506 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:36.605405 13615 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:36.606359 13612 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:36.473176 13438 rpcz_store.cc:269] 0709 12:57:23.245453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:23.310913 (+ 65460us) service_pool.cc:224] Handling call
0709 12:57:36.472497 (+13161584us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:57:33.253738 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:57:33.476295 (+222557us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:57:33.476351 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:57:33.476377 (+    26us) tablet.cc:662] Decoding operations
0709 12:57:33.477581 (+  1204us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:57:33.477689 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 12:57:33.477729 (+    40us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:57:33.478670 (+   941us) tablet.cc:701] Row locks acquired
0709 12:57:33.478691 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:57:33.478891 (+   200us) write_op.cc:270] Start()
0709 12:57:33.479039 (+   148us) write_op.cc:276] Timestamp: P: 1783601853478828 usec, L: 0
0709 12:57:33.479064 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:57:33.505270 (+ 26206us) log.cc:844] Serialized 3472 byte log entry
0709 12:57:33.935016 (+429746us) op_driver.cc:464] REPLICATION: finished
0709 12:57:34.076138 (+141122us) write_op.cc:301] APPLY: starting
0709 12:57:34.076248 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:57:34.136444 (+ 60196us) tablet.cc:1370] finished BulkCheckPresence
0709 12:57:34.154101 (+ 17657us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:57:34.246615 (+ 92514us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:57:34.246938 (+   323us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:57:34.247006 (+    68us) write_op.cc:312] APPLY: finished
0709 12:57:34.247365 (+   359us) log.cc:844] Serialized 65 byte log entry
0709 12:57:34.247723 (+   358us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:34.247880 (+   157us) write_op.cc:454] Released schema lock
0709 12:57:34.247959 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:57:34.319023 (+ 71064us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:34.319548 (+   525us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15842,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":116667,"apply.run_cpu_time_us":8195,"apply.run_wall_time_us":230943,"num_ops":8,"prepare.queue_time_us":952214,"prepare.run_cpu_time_us":28756,"prepare.run_wall_time_us":451648,"raft.queue_time_us":166169,"raft.run_cpu_time_us":33693,"raft.run_wall_time_us":50969,"replication_time_us":455726,"thread_start_us":580768,"threads_started":4,"wal-append.queue_time_us":204587,"wal-append.run_cpu_time_us":66230,"wal-append.run_wall_time_us":1487680}]]}
I20260709 12:57:36.610458 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.613564 13437 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 5 to 7, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:57:36.617460 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 142162us
I20260709 12:57:36.525599 16817 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:36.620234 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.621019 13685 raft_consensus.cc:2368] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6.
I20260709 12:57:36.620702 16817 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.621888 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.622421 16872 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:57:36.623976 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:36.624153 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:36.624516 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 122802us
I20260709 12:57:36.630210 13617 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:36.631850 13692 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:36.653160 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:36.660192 16859 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:36.660398 16914 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:36.660907 14245 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:36.660836 16859 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.662130 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.644538 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.645620 14240 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:36.578701 16915 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:36.592511 16616 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:36.666826 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 90111.3us
I20260709 12:57:36.623610 16872 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:57:36.675693 13536 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:36.642436 14246 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:36.676365 13685 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:36.643306 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:36.644977 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:36.624784 16765 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:36.666275 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.637382 13611 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:36.639704 14239 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:36.644321 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.666962 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.660264 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:36.679477 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.686079 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.699676 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:36.728360 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.729643 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.733613 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.902593 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:36.746538 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:36.728034 16616 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:36.913460 13534 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
W20260709 12:57:36.874229 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 216695us
W20260709 12:57:36.914122 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 247917us
I20260709 12:57:36.914870 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:36.916417 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:36.656904 16792 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:36.732846 16732 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:57:36.644060 16861 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:36.674322 16506 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:57:36.902114 13543 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:57:36.780985 16797 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:36.904194 13536 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:36.907794 13693 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:36.909732 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.908876 14243 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:36.749415 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:36.926841 13687 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:36.917438 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:36.928202 14239 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:36.921856 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 243653us
W20260709 12:57:36.931115 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 264207us
I20260709 12:57:36.938460 13688 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:36.938683 14240 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:57:36.940815 14237 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:57:36.941156 13508 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:36.934286 13437 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 4 to 5, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:36.915624 16703 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:36.922981 16861 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:36.910518 16915 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:36.941560 16924 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.241s	user 0.000s	sys 0.009s
I20260709 12:57:36.967128 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:36.976401 13542 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:36.927521 16706 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:36.968745 13508 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:36.934751 16922 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:36.941994 16869 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:36.957015 15968 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:36.923807 16803 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.000244 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:37.013652 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:36.940068 16765 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:57:36.976825 16932 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:36.997396 16922 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:36.980157 16767 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:37.001204 16555 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:37.035131 16859 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:37.035144 16717 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:37.015654 16914 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:57:36.975893 16850 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 12:57:37.037668 16878 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.822s	user 0.012s	sys 0.005s
I20260709 12:57:37.022974 16915 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:37.018822 16870 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:57:37.029354 16932 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.042645 13542 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
W20260709 12:57:37.042796 16939 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:57:37.043025 14241 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:37.044730 16803 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:37.045651 16937 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.280s	user 0.002s	sys 0.008s
I20260709 12:57:36.999765 16792 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:37.026809 13437 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 6 to 7, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:37.064455 13540 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:37.077541 14245 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:37.026319 16922 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:37.095597 16920 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.005s
W20260709 12:57:37.096971 16930 consensus_meta.cc:287] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.607s	user 0.014s	sys 0.006s
I20260709 12:57:37.093245 16555 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:37.099048 16869 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:37.099632 16635 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:37.100239 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:37.112006 16924 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:57:37.130013 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 198200us
I20260709 12:57:37.109186 16616 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:37.100667 16850 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:37.102491 16870 consensus_queue.cc:1243] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:57:37.150511 13437 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 5 to 7, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:37.099700 15968 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:57:37.153676 13686 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:37.165186 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:37.166039 13687 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:37.166446 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 487336us
I20260709 12:57:37.093391 16717 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.167727 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:37.173921 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:37.174154 13541 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 12:57:37.167107 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:37.182341 13684 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:37.122481 16767 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.181748 13541 pending_rounds.cc:85] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 8
I20260709 12:57:37.203608 13541 pending_rounds.cc:107] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 12:57:37.203896 13541 raft_consensus.cc:2889] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:37.122716 16920 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:37.204417 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 260270us
I20260709 12:57:37.206544 13693 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:57:37.124032 16930 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.124871 16792 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:37.204690 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:37.217723 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 43485.6us
I20260709 12:57:37.213711 13437 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 5, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:37.223299 13534 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:37.228147 14242 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:57:37.225200 16811 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:57:37.362047 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 150690us
W20260709 12:57:37.398478 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70950.2us
I20260709 12:57:37.315656 16932 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:37.400724 13506 leader_election.cc:400] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 5 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6.
I20260709 12:57:37.401518 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:37.404693 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 185485us
I20260709 12:57:37.407707 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:37.413578 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:37.214572 16869 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:37.453281 13617 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:57:37.454967 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:37.457780 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:37.453359 13437 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 4 to 5, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:57:37.453145 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 276640us
W20260709 12:57:37.455869 14244 consensus_meta.cc:287] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.542s	user 0.013s	sys 0.003s
I20260709 12:57:37.456419 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:37.402180 16947 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:37.461387 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:37.460172 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:37.462498 14244 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:37.472927 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:37.221143 16555 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:37.456710 16616 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:57:37.485371 13689 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 9. Ops: []
I20260709 12:57:37.491205 13611 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:57:37.493431 13437 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 6, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 6 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:37.605260 13612 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:57:37.611759 13617 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:37.633008 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 172944us
I20260709 12:57:37.633234 13437 catalog_manager.cc:5697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 3 to 4, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 4 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:37.633868 16946 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:37.634469 16946 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:37.634981 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:37.634770 13437 catalog_manager.cc:5697] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 5, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:37.636708 13437 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 5, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:37.638360 16861 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:37.639645 16861 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:37.649080 13437 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 4 to 5, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 5 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:37.465466 16774 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:37.653077 16774 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.655309 16774 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:37.641779 16817 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:37.644502 16949 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
W20260709 12:57:37.645313 16945 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.319s	user 0.000s	sys 0.000s
W20260709 12:57:37.646204 16944 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
I20260709 12:57:37.655889 16774 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:37.660435 16774 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:37.661732 16774 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.663978 16774 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:37.669931 16920 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:37.486295 16947 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:37.673307 16947 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:37.674508 16792 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
W20260709 12:57:37.677517 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:37.646992 16948 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.323s	user 0.000s	sys 0.000s
I20260709 12:57:37.656256 16817 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.635505 16946 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:37.652508 16956 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
I20260709 12:57:37.641144 16930 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:37.740832 13687 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.026s	sys 0.045s
W20260709 12:57:37.824302 13687 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.027s	sys 0.045s
I20260709 12:57:37.456983 16811 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:37.806353 13684 consensus_meta.cc:287] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.003s
W20260709 12:57:37.675626 16703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.078s	sys 0.083s
I20260709 12:57:37.629966 15968 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:37.828215 13684 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
W20260709 12:57:37.828545 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 422162us
I20260709 12:57:37.832270 16958 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:37.852298 13692 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:37.823094 13535 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:57:37.845678 14246 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.004s	sys 0.070s
W20260709 12:57:37.853761 14246 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.005s	sys 0.071s
I20260709 12:57:37.800264 16859 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:37.832815 16703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.078s	sys 0.083s
I20260709 12:57:37.829649 16797 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:37.834506 16957 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:57:37.830034 16765 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:57:37.908490 16964 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:37.909440 16965 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:37.909181 16964 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:37.910327 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 62904.2us
I20260709 12:57:37.874404 16859 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:37.925478 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:37.875442 16767 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:37.874982 16703 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:37.876091 16797 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:37.932708 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 549) took 1244 ms. Trace:
I20260709 12:57:37.916687 15968 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:37.935889 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:37.936228 13612 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:57:37.478087 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 509366us
I20260709 12:57:37.960129 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:37.942627 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:37.910104 16965 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:37.900144 16717 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:37.961781 13617 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:37.941979 14246 rpcz_store.cc:276] 0709 12:57:36.679523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.689596 (+ 10073us) service_pool.cc:224] Handling call
0709 12:57:36.903192 (+213596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:36.903407 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:37.154201 (+250794us) log.cc:844] Serialized 14 byte log entry
0709 12:57:37.922138 (+767937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:37.922868 (+   730us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:37.923019 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:37.923273 (+   254us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:37.923706 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60761,"prepare.queue_time_us":503547,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":60937,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":95,"thread_start_us":950455,"threads_started":2}
I20260709 12:57:37.926402 16773 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:37.863422 16958 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:37.968097 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1471) took 1325 ms. Trace:
I20260709 12:57:37.945367 16797 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:37.945176 16792 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:37.962944 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.012s
W20260709 12:57:37.976562 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.009s	sys 0.012s
I20260709 12:57:37.969920 13538 rpcz_store.cc:276] 0709 12:57:36.638560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.639222 (+   662us) service_pool.cc:224] Handling call
0709 12:57:36.663159 (+ 23937us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:36.909007 (+245848us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:36.909037 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:36.911198 (+  2161us) log.cc:844] Serialized 52 byte log entry
0709 12:57:37.188401 (+277203us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:37.188460 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:37.188563 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:37.963832 (+775269us) raft_consensus.cc:1685] finished
0709 12:57:37.963850 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:37.964079 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172998,"raft.queue_time_us":57094,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":4878,"thread_start_us":465769,"threads_started":2,"wal-append.queue_time_us":459635}
W20260709 12:57:37.971446 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 486601us
W20260709 12:57:38.019761 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 83999.9us
W20260709 12:57:38.025413 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 611820us
I20260709 12:57:38.029008 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.031025 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.031383 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:38.032768 13659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.022s	sys 0.017s
W20260709 12:57:38.034117 13659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.022s	sys 0.017s
I20260709 12:57:38.036775 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1481) took 1078 ms. Trace:
I20260709 12:57:38.038508 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:38.039048 14237 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:38.041978 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.044348 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.037197 13540 rpcz_store.cc:276] 0709 12:57:36.958154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.958343 (+   189us) service_pool.cc:224] Handling call
0709 12:57:37.054169 (+ 95826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:37.120345 (+ 66176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.036591 (+916246us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b5400002650. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:38.036720 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219448064}
W20260709 12:57:38.070755 13507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.007s	sys 0.033s
W20260709 12:57:38.071435 13507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.007s	sys 0.033s
I20260709 12:57:37.966080 16947 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:57:37.959508 15968 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.032668 16803 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.076907 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.077996 13690 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.076156 16930 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:38.094327 14237 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:38.094964 16930 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.022416 16767 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:38.102385 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.034364 16773 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:38.096773 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.104799 13685 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:38.098569 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:38.072579 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 437638us
I20260709 12:57:38.106117 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.095748 16930 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:37.971249 16811 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:38.166105 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 144521us
I20260709 12:57:38.167270 16970 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.167734 16974 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:38.168155 16954 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.201s
W20260709 12:57:38.169530 16972 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:57:38.169925 16960 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:38.176501 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 440) took 1017 ms. Trace:
W20260709 12:57:38.201750 13686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1427) took 1748 ms (client timeout 1671 ms). Trace:
I20260709 12:57:38.079177 16911 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
W20260709 12:57:38.202221 13686 rpcz_store.cc:269] 0709 12:57:36.453616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.453904 (+   288us) service_pool.cc:224] Handling call
0709 12:57:38.201552 (+1747648us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54002391d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:38.201675 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586258304}
I20260709 12:57:38.202790 16911 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:38.203270 16911 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:38.107154 16773 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.215503 16773 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:38.181524 16966 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.245s
I20260709 12:57:38.219067 16966 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:57:38.220273 16966 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.186925 16964 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.221431 15968 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.184655 16973 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.221776 16964 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:57:38.187469 16976 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.222545 16973 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.223080 16976 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.224985 16973 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:38.225493 16976 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:38.080600 16703 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:38.188422 16961 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:57:38.230763 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1428) took 1761 ms. Trace:
I20260709 12:57:38.231166 13691 rpcz_store.cc:276] 0709 12:57:36.469355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.469484 (+   129us) service_pool.cc:224] Handling call
0709 12:57:36.472279 (+  2795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:36.510606 (+ 38327us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:36.951414 (+440808us) log.cc:844] Serialized 14 byte log entry
0709 12:57:37.424546 (+473132us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:37.462718 (+ 38172us) log.cc:844] Serialized 26 byte log entry
0709 12:57:37.462922 (+   204us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:57:37.463151 (+   229us) log.cc:844] Serialized 14 byte log entry
0709 12:57:37.472701 (+  9550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:37.472773 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:38.230520 (+757747us) raft_consensus.cc:1685] finished
0709 12:57:38.230542 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:38.230676 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91453,"prepare.queue_time_us":424338,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":8478,"raft.queue_time_us":463102,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":27371,"thread_start_us":839052,"threads_started":2,"wal-append.queue_time_us":902411}
W20260709 12:57:38.189319 16969 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:57:38.189527 13690 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:38.199505 16977 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:38.234935 16977 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.203500 13688 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:38.204610 16911 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.236980 16911 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.237870 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 436) took 1090 ms. Trace:
I20260709 12:57:38.148249 16817 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.238312 13688 rpcz_store.cc:276] 0709 12:57:37.147026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.147181 (+   155us) service_pool.cc:224] Handling call
0709 12:57:38.203290 (+1056109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:38.236339 (+ 33049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.237676 (+  1337us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002391d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:38.237807 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2383744384}
I20260709 12:57:38.208333 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 618) took 1076 ms. Trace:
I20260709 12:57:38.209553 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 441) took 1050 ms. Trace:
I20260709 12:57:38.240442 13692 rpcz_store.cc:276] 0709 12:57:37.159543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.180350 (+ 20807us) service_pool.cc:224] Handling call
0709 12:57:37.847853 (+667503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:37.852771 (+  4918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.209343 (+356572us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54002eccd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:38.209486 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1467690880}
I20260709 12:57:38.179136 16968 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:38.200237 13687 rpcz_store.cc:276] 0709 12:57:37.159317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.172513 (+ 13196us) service_pool.cc:224] Handling call
0709 12:57:37.203572 (+ 31059us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:37.824800 (+621228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:37.824848 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:37.833152 (+  8304us) log.cc:844] Serialized 26 byte log entry
0709 12:57:38.047398 (+214246us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:38.047452 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.047548 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:38.176262 (+128714us) raft_consensus.cc:1685] finished
0709 12:57:38.176292 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:38.176444 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67931,"raft.queue_time_us":280631,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":114350,"thread_start_us":759420,"threads_started":2,"wal-append.queue_time_us":294316}
W20260709 12:57:38.242744 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 136728us
I20260709 12:57:38.239640 13534 rpcz_store.cc:276] 0709 12:57:37.131593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.131872 (+   279us) service_pool.cc:224] Handling call
0709 12:57:37.132225 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:37.223766 (+ 91541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.208040 (+984274us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b54000ab754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:38.208263 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208883584}
W20260709 12:57:38.186018 16967 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:57:38.262215 13542 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
W20260709 12:57:38.257896 16635 consensus_meta.cc:287] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.128s	user 0.005s	sys 0.007s
I20260709 12:57:38.271363 13582 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.221446 16966 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.174083 16767 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.279337 16977 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:38.280251 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1430) took 1746 ms. Trace:
I20260709 12:57:38.310106 13689 rpcz_store.cc:276] 0709 12:57:36.533433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.591702 (+ 58269us) service_pool.cc:224] Handling call
0709 12:57:36.612308 (+ 20606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:37.511779 (+899471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.279938 (+768159us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b540042d1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:38.280175 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2007398272}
I20260709 12:57:38.276618 16858 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:57:38.321834 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 50593.7us
I20260709 12:57:38.322705 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:38.323776 14241 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 960) took 1682 ms. Trace:
I20260709 12:57:38.324479 14241 rpcz_store.cc:276] 0709 12:57:36.641393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.641534 (+   141us) service_pool.cc:224] Handling call
0709 12:57:37.042834 (+401300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:37.091670 (+ 48836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.317613 (+1225943us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b5400650250. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:38.323697 (+  6084us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":916085120}
I20260709 12:57:38.306856 16635 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.294972 16859 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.340950 16977 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.354244 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:38.332156 13508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 1.548s (SENT) [suppressed 14 similar messages]
I20260709 12:57:38.323999 16981 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:38.322204 16858 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:38.174429 16965 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.183416 16974 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.328795 16970 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:38.334190 16873 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.257601 16968 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.364230 13536 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:38.334769 16861 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:38.350035 16922 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.220s	user 0.012s	sys 0.000s
I20260709 12:57:38.377328 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.363148 16859 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:38.349086 16984 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:38.359514 16981 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:38.386526 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:38.390390 16811 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.312440 16966 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.392686 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 545) took 1760 ms. Trace:
I20260709 12:57:38.392755 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.402364 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:38.382776 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 210845us
I20260709 12:57:38.367384 16985 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:38.403548 13534 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.413769 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.408839 14242 rpcz_store.cc:276] 0709 12:57:36.632413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.632652 (+   239us) service_pool.cc:224] Handling call
0709 12:57:36.633078 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:37.314061 (+680983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.392347 (+1078286us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54005d0a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:38.392609 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1360516864}
I20260709 12:57:38.391059 16963 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:38.365943 16977 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.428668 13538 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:57:38.429690 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:38.432196 13538 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:38.433810 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.435184 13658 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:38.400319 16947 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.408682 16975 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.438462 14238 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:57:38.438572 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 106441us
I20260709 12:57:38.440330 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.441435 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.441673 14242 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:57:38.446667 16422 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:38.392002 16861 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.448487 16989 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:57:38.449259 16988 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:57:38.450234 16861 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.439714 16973 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:57:38.451031 16765 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:38.451547 16765 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:38.394039 16922 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.452548 16765 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.454201 16922 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:38.453938 16765 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:38.455283 16983 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:57:38.437434 16985 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.422847 16859 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:38.457207 16966 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.479139 13534 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:57:38.429786 16703 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.430436 16984 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.481077 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:38.483372 14240 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.375375 16970 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.481695 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.421193 16873 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 12:57:38.489686 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 76025.2us
I20260709 12:57:38.484126 13538 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.487951 14242 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:38.490633 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.477110 16850 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.477802 16914 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:38.423520 16811 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.460445 16976 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:38.437386 16706 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:38.478927 16717 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:57:38.498020 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:38.525007 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.439687 16858 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.485762 16988 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:57:38.436792 16963 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.496310 16859 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:38.561740 13613 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:38.503348 16974 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.485575 16981 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.504510 16767 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.505296 16970 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.505852 16922 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:38.610237 16922 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:38.610365 16974 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.611125 16974 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:38.614095 16963 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.614539 16974 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.495318 16966 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:38.507295 16811 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.547658 16997 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.508607 16873 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:57:38.528824 16803 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:38.489620 16984 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.615281 16966 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.615681 16803 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.515453 15968 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:38.616906 15968 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:38.617265 16984 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.617751 16966 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.497567 16985 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.618641 16974 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:38.527875 16960 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:38.533689 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.520304 16978 consensus_queue.cc:1059] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:57:38.520646 13541 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:57:38.520710 13535 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:57:38.620441 16978 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:38.620801 16978 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:38.520975 16991 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.521661 16959 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.461570 16947 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.621662 16978 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.622498 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.501906 16977 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:38.623142 13689 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:38.543697 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.623592 16590 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.623744 16947 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.517168 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 619) took 1385 ms. Trace:
I20260709 12:57:38.624223 16590 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.624392 13536 rpcz_store.cc:276] 0709 12:57:37.132049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.132166 (+   117us) service_pool.cc:224] Handling call
0709 12:57:37.132413 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:38.374591 (+1242178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.516912 (+142321us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003cad54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:38.517102 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832416128}
I20260709 12:57:38.625555 13689 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:38.625747 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.519815 16952 raft_consensus.cc:3055] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:57:38.626791 16590 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.626957 16952 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:38.481467 16616 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:57:38.627563 16767 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:38.595340 13615 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:38.596230 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:38.645617 13617 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.491528 16773 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.593823 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.604146 13611 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:38.543620 16964 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:38.510957 16797 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.617764 16803 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.619184 16974 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.622222 16959 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.622872 16978 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:38.523546 16930 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.635905 16934 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.594039 13618 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:38.629091 13693 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:57:38.662518 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.632932 16990 raft_consensus.cc:2764] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260709 12:57:38.635195 14245 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 548) took 1967 ms (client timeout 1859 ms). Trace:
I20260709 12:57:38.668216 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:38.628732 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:38.668845 13535 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.669242 13610 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:38.669341 16930 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.670058 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:38.638110 16981 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.637250 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.664434 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.496249 16968 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:38.478250 16975 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.599687 16878 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.600211 16872 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.656886 16970 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:38.558285 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.597009 16555 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:57:38.648391 14212 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:38.650045 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.586848 16912 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:38.715593 16912 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.717820 16912 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.651157 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.591617 16744 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:38.653864 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 120273us
I20260709 12:57:38.719035 16744 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.721172 16744 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.597573 16920 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.588479 13613 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:38.723203 16920 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.724630 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.725701 13613 raft_consensus.cc:2410] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:57:38.659385 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.726361 16920 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.659385 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.661736 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:38.729595 16774 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:57:38.730861 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 69170.3us
I20260709 12:57:38.662681 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.636446 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.732263 13543 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:38.637266 16946 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.733392 16996 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:38.673312 16934 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.665258 13616 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:38.663744 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.736635 13539 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:38.667183 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:38.668018 16792 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.001s
I20260709 12:57:38.738341 16792 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:38.527709 16706 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.648952 16997 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:57:38.668543 14245 rpcz_store.cc:269] 0709 12:57:36.668090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:36.674259 (+  6169us) service_pool.cc:224] Handling call
0709 12:57:38.634973 (+1960714us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b54001c3c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:38.635085 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45217024}
I20260709 12:57:38.649935 16811 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.745071 13537 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 6.
I20260709 12:57:38.672614 13687 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:38.745431 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:38.650331 16767 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.714007 13535 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:38.611505 16922 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.680697 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.681329 16797 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.682338 16959 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.683117 13534 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:38.683255 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.592283 16859 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.495710 16914 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.659523 16991 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:38.688153 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.693192 16872 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.765198 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.696372 16878 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.697114 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1484) took 1618 ms. Trace:
I20260709 12:57:38.771024 13539 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:57:38.698091 16773 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.699620 14244 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:57:38.756606 16985 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:38.714370 13617 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:38.775271 13612 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []
I20260709 12:57:38.775553 13615 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:57:38.737532 16934 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.776670 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 530) took 1380 ms. Trace:
I20260709 12:57:38.688474 16803 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:57:38.742547 16706 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.749035 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.726619 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.753084 16710 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:57:38.756552 14240 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:38.756436 16744 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 12:57:38.758678 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 75485.3us
I20260709 12:57:38.617921 15968 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.684329 16964 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:57:38.798141 16872 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.798812 16953 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:38.798828 17002 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:38.672329 16590 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:57:38.800179 16859 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.767331 16920 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:57:38.695142 16950 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:38.801116 16950 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:38.801214 16920 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:38.801645 16920 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:38.802003 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.802088 16950 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.802881 13540 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:38.770090 13542 rpcz_store.cc:276] 0709 12:57:37.079211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.079334 (+   123us) service_pool.cc:224] Handling call
0709 12:57:38.696916 (+1617582us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003ca354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:38.697030 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375852416}
I20260709 12:57:38.803238 16950 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:38.702239 16974 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.803768 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.804416 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.804879 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.805302 14242 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:38.805335 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.496419 16850 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.804335 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.772365 16996 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:57:38.777274 13610 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.758064 16912 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:57:38.808379 16767 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.767105 16990 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:38.808964 16990 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.809376 16990 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.809881 16997 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:38.810233 16878 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.808414 16850 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.759081 16811 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.822403 16811 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:38.712376 13536 raft_consensus.cc:2410] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:57:38.780557 13613 raft_consensus.cc:2410] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:57:38.754997 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 106655us
I20260709 12:57:38.823833 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.824506 13536 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:38.777285 13614 rpcz_store.cc:276] 0709 12:57:37.396819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.400571 (+  3752us) service_pool.cc:224] Handling call
0709 12:57:37.400913 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:37.458204 (+ 57291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:37.458254 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:37.473586 (+ 15332us) log.cc:844] Serialized 26 byte log entry
0709 12:57:38.513139 (+1039553us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b18003cce80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:38.649239 (+136100us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:38.761948 (+112709us) log.cc:844] Serialized 14 byte log entry
0709 12:57:38.762444 (+   496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:38.762520 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:38.762573 (+    53us) raft_consensus.cc:1685] finished
0709 12:57:38.776421 (+ 13848us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:38.776596 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276803,"prepare.queue_time_us":123117,"raft.queue_time_us":89640,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":137262,"spinlock_wait_cycles":76219648,"thread_start_us":221459,"threads_started":2,"wal-append.queue_time_us":1052824}
I20260709 12:57:38.773099 16922 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:38.825138 14212 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.825430 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.826026 13536 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:38.764358 16914 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:38.826731 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.827476 13536 raft_consensus.cc:2410] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 9 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:57:38.785261 13612 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:57:38.747635 16970 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.829721 16970 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.776820 16985 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.830299 16985 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.731659 16774 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:38.832118 16774 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:38.677167 16952 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.799784 16953 consensus_queue.cc:1243] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:57:38.802424 16920 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.804527 13542 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:38.833191 14212 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.763084 13618 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:38.836863 13610 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:38.832653 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.796404 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:38.840137 13541 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:38.835189 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.685482 16930 consensus_queue.cc:1059] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:57:38.842033 13614 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:57:38.832535 16774 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:38.843892 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 94272.8us
I20260709 12:57:38.845376 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:38.845889 13542 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:38.850200 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:38.840237 16952 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:57:38.840713 16710 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:38.856266 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.857913 13539 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:38.860110 13584 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.836578 16960 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:57:38.844115 16811 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:38.838240 16590 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:57:38.838081 16912 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:57:38.851751 16920 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.853400 16976 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:38.866750 14245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:57:38.867837 14245 raft_consensus.cc:606] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:38.868326 14245 raft_consensus.cc:614] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:57:38.868440 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:38.862082 16710 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.861325 16935 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:38.873333 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.875831 13542 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:38.876456 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.864727 16811 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:38.881496 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.880254 16803 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:38.865480 16958 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:38.865267 16873 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:38.884692 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.886394 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.876540 16710 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.890067 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:38.892657 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.883888 16555 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:38.893709 13618 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:57:38.896373 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:38.883859 16803 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.885174 16959 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:38.900060 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 438709us
I20260709 12:57:38.884312 16935 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:38.904968 13617 pending_rounds.cc:85] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 3
I20260709 12:57:38.906418 13617 pending_rounds.cc:107] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:57:38.906690 13617 raft_consensus.cc:2889] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:38.889045 16965 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:38.888933 16792 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:38.876858 16930 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:57:38.896608 16767 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:38.908803 16965 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:38.906268 16803 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:38.919819 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.931447 14244 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:38.920794 16922 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:38.921222 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.921451 16978 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:38.925151 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.888382 16873 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.926338 13686 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:57:38.939239 13686 pending_rounds.cc:85] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:57:38.939568 13686 pending_rounds.cc:107] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:57:38.939491 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:38.940135 13686 raft_consensus.cc:2889] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:38.919320 17005 consensus_queue.cc:1059] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:57:38.943392 17005 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:38.944168 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.944911 13691 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:38.910507 16792 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:38.947554 16792 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:38.931596 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:38.909021 16555 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:38.934460 14239 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:57:38.909873 16966 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:38.965852 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:38.932571 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.966754 13690 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:38.904498 17000 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.938738 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.942426 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:38.946933 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.931756 13541 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:38.946959 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:38.932437 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.964489 16767 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:38.932967 17002 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:38.971423 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.971403 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.971824 16767 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.972525 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.973160 13685 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:38.971778 17002 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.974298 13685 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:38.943773 17005 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:38.975634 17005 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.976437 16555 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.977056 17005 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:38.978390 16914 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:38.979045 14244 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEARNER]: Advancing to term 8
I20260709 12:57:38.895807 16990 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:38.838950 16968 consensus_queue.cc:1175] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" responder_term: 5 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:57:38.972779 13692 raft_consensus.cc:2393] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:38.979959 13693 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:38.963538 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.934001 16965 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:38.975394 16985 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:38.982522 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.990324 13687 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:38.987669 13686 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:38.981715 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 74044.2us
I20260709 12:57:38.995092 13690 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:57:38.970558 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:38.994532 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:39.000706 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:39.005640 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:38.988766 16773 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:39.007391 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:39.010612 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 96258.5us
I20260709 12:57:39.013150 13613 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:39.017673 14244 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:57:39.019379 13690 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:39.027956 16984 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:39.028589 17002 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:39.028736 16984 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:39.031478 16984 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:39.032618 16914 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:39.032781 13618 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:39.033756 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.034300 13618 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:39.035122 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:39.036388 13612 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:39.018224 14244 pending_rounds.cc:85] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 6
I20260709 12:57:39.037590 14244 pending_rounds.cc:107] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:57:39.037935 14244 raft_consensus.cc:2889] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:57:39.038077 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 67630.5us
I20260709 12:57:39.041730 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.043457 13689 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:57:38.983755 16732 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:57:39.048327 13618 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:39.050367 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.051404 13692 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:39.052727 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.053530 13613 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:39.009675 16981 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:39.061057 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:39.063120 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 197139us
I20260709 12:57:39.072455 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:39.067426 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.075524 13618 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:39.048246 16980 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:39.088162 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:39.039067 16703 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:57:39.086580 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.059751 16953 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:39.094515 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.096414 13690 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:39.096498 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:39.101866 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.108515 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.109773 14241 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:39.111692 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.113516 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:39.120661 14238 raft_consensus.cc:2376] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8. Re-sending same reply.
I20260709 12:57:39.097203 16952 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:57:39.093106 16963 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:39.126288 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.099458 16706 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.136246 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:39.125557 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.104190 17002 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:39.021024 16935 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.108734 16953 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:39.118415 16978 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:57:39.081319 16765 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:57:39.125936 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:39.159700 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.127755 13685 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:57:39.160526 13615 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:39.146965 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.161386 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.161896 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:39.162845 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:39.163010 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.163595 13541 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:39.143206 14237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d73171142594388972ed22eae44f95f"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:57:39.164366 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.164526 14237 raft_consensus.cc:606] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:39.145042 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:39.165764 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:39.142966 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.145680 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:39.168789 16811 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.170106 16811 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:39.126089 16952 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:39.172021 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:39.172359 16990 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:39.172736 16952 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.133519 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.102095 16590 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:57:39.174472 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.174911 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.175525 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.176937 13540 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:57:39.177075 16985 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:39.154420 13612 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:39.144621 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.148016 13536 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:57:39.147075 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.148185 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.178982 13542 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:39.147099 16980 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.101923 16922 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:57:39.177162 13619 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:39.147080 13538 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:39.177376 13540 pending_rounds.cc:85] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 4
I20260709 12:57:39.155803 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.185570 13540 pending_rounds.cc:107] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:57:39.183305 13610 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:57:39.179018 13536 pending_rounds.cc:85] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 6
I20260709 12:57:39.186331 13536 pending_rounds.cc:107] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:57:39.157226 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.186697 13540 raft_consensus.cc:2889] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:39.186712 13536 raft_consensus.cc:2889] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:39.126467 16873 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:39.202613 14244 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:57:39.257714 14244 raft_consensus.cc:606] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
W20260709 12:57:39.258623 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 56280.5us
I20260709 12:57:39.259318 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:39.260694 13508 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:39.263520 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 84644.9us
I20260709 12:57:39.185725 16914 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:39.268790 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:39.271544 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.262921 16850 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.138403 16710 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.201010 16963 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.287987 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.288669 14237 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:39.179967 16706 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.270298 16997 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:39.295058 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:39.294669 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:39.295658 13538 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:57:39.272037 16970 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:39.184535 16774 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.296239 13538 pending_rounds.cc:85] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 6
I20260709 12:57:39.316872 13538 pending_rounds.cc:107] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:57:39.317190 13538 raft_consensus.cc:2889] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:39.296706 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.319538 14240 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:39.196149 16980 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:39.325970 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:39.327272 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:39.289546 16911 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:39.289875 16914 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:39.326026 16506 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:57:39.333928 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.336025 13508 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:39.336433 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:39.268586 16878 consensus_queue.cc:1059] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:57:39.294450 17000 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:39.330365 14244 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:39.293372 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.381443 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:39.290767 16850 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.337627 16966 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:39.304765 16997 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:39.331651 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.337394 16988 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:39.339680 16911 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:39.394115 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 58177.3us
I20260709 12:57:39.299626 16963 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:39.355784 16590 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:39.398064 13582 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:39.407234 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:39.414942 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.393204 16997 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:39.401393 16911 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:39.298300 16953 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:39.422679 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 128085us
W20260709 12:57:39.430189 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:39.430974 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:39.426699 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:39.412868 16999 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:39.345947 16935 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:57:39.419816 16988 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:39.529834 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 98951.5us
W20260709 12:57:39.522722 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 334860us
I20260709 12:57:39.533779 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:39.410207 16590 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:39.410102 16555 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:57:39.420382 16985 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.522873 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:39.422420 16981 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:39.425467 17000 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:39.403237 16792 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:39.541728 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 115093us
I20260709 12:57:39.550437 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:39.538739 14211 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:39.537985 13686 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:39.423780 16803 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:39.518574 16934 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:39.431777 17021 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 12:57:39.537231 16590 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:39.542704 17000 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:39.541256 17002 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:39.631707 17005 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:39.762496 13615 raft_consensus.cc:2368] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
W20260709 12:57:39.538218 16965 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.537s	user 0.012s	sys 0.006s
I20260709 12:57:39.687500 17020 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:57:39.702010 17028 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
W20260709 12:57:39.702736 17037 log.cc:927] Time spent T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
W20260709 12:57:39.703532 17036 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.284s	user 0.000s	sys 0.000s
I20260709 12:57:39.704063 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:39.728711 17033 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
I20260709 12:57:39.730017 16975 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:39.731635 16850 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:39.632551 16985 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.751528 13618 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:57:39.535812 16765 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:57:39.783246 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 232955us
I20260709 12:57:39.762286 16930 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:39.792301 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.764413 15968 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:39.544224 16988 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.527347 16914 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:39.765684 17038 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:57:39.839236 13534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.021s	sys 0.053s
W20260709 12:57:39.895670 13534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.022s	sys 0.053s
I20260709 12:57:39.760563 16962 raft_consensus.cc:3055] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:39.766105 17032 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.787047 17000 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:39.747479 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:39.857534 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:39.784543 17005 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:39.876786 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:39.792162 17020 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:39.876858 16850 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.885438 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:39.889181 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:39.897359 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:39.923028 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:39.828822 16710 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:39.918449 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 171071us
I20260709 12:57:39.734881 17016 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.189s
W20260709 12:57:39.903410 17028 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:57:39.924916 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:39.901623 16878 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:39.907146 16970 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.017s	sys 0.030s
I20260709 12:57:39.938656 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1606) took 1047 ms. Trace:
I20260709 12:57:39.910876 17005 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:39.902135 16706 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:39.899626 16555 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:39.900343 16962 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:39.949390 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 557) took 1874 ms. Trace:
I20260709 12:57:39.961223 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1410) took 1167 ms. Trace:
I20260709 12:57:39.881860 16975 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:57:39.822439 16965 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:39.950762 13614 rpcz_store.cc:276] 0709 12:57:38.891218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.891369 (+   151us) service_pool.cc:224] Handling call
0709 12:57:38.891739 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:38.892154 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:38.892183 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:38.892642 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.322939 (+430297us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:39.322980 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.323058 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:39.938430 (+615372us) raft_consensus.cc:1685] finished
0709 12:57:39.938450 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:39.938591 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11810,"raft.queue_time_us":1012,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":12733,"thread_start_us":429888,"threads_started":1,"wal-append.queue_time_us":502228}
W20260709 12:57:39.956759 17035 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:57:39.916236 16765 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:39.958168 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.916379 17020 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:39.885128 17048 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:57:39.889740 16930 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.961253 14246 rpcz_store.cc:276] 0709 12:57:38.075121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.075362 (+   241us) service_pool.cc:224] Handling call
0709 12:57:38.089214 (+ 13852us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:38.089766 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:38.089816 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:38.156144 (+ 66328us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.144038 (+987894us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b180047eae0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:39.270371 (+126333us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:39.270424 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.270510 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:39.949035 (+678525us) raft_consensus.cc:1685] finished
0709 12:57:39.949064 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:39.949314 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8187,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":54880,"spinlock_wait_cycles":11356416,"thread_start_us":126147,"threads_started":1,"wal-append.queue_time_us":1126048}
W20260709 12:57:39.950572 16970 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.018s	sys 0.030s
I20260709 12:57:39.938663 16980 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:39.970126 13536 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:39.972971 13617 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:39.946100 16803 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:39.822808 16934 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.947630 16971 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:39.961745 13539 rpcz_store.cc:276] 0709 12:57:38.793464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.863527 (+ 70063us) service_pool.cc:224] Handling call
0709 12:57:38.872550 (+  9023us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:38.876365 (+  3815us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:38.998800 (+122435us) log.cc:844] Serialized 14 byte log entry
0709 12:57:39.190399 (+191599us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.204393 (+ 13994us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.204571 (+   178us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:39.204620 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.204705 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:39.957754 (+753049us) raft_consensus.cc:1685] finished
0709 12:57:39.957775 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:39.961146 (+  3371us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4011,"prepare.queue_time_us":229495,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":4886,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":61425,"thread_start_us":254951,"threads_started":2,"wal-append.queue_time_us":307199}
I20260709 12:57:39.949311 16774 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:39.958298 16963 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:39.935587 16850 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:39.998782 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:39.910499 17032 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:39.950970 17002 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:57:39.950820 17029 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:57:39.989734 16970 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:40.035785 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:40.041433 13539 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:39.974642 16952 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:57:40.047004 14246 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87a8c869a6b840e591423340bc32af00"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:57:40.049460 14246 raft_consensus.cc:606] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:57:40.049997 14246 raft_consensus.cc:614] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:57:40.033538 17057 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:39.973701 16555 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.033118 16792 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.104105 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.131644 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 544) took 1066 ms. Trace:
I20260709 12:57:40.168648 14246 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:57:40.178161 13617 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:40.129796 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1415) took 1224 ms. Trace:
W20260709 12:57:40.130007 16997 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.001s
W20260709 12:57:40.199263 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 260057us
I20260709 12:57:40.209092 13542 rpcz_store.cc:276] 0709 12:57:38.905611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.914267 (+  8656us) service_pool.cc:224] Handling call
0709 12:57:40.129627 (+1215360us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540007dad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:40.129738 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548768128}
I20260709 12:57:40.209998 17029 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:40.210531 17029 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.209317 16997 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.179622 14246 raft_consensus.cc:606] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
W20260709 12:57:40.126297 17046 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:57:40.212913 17029 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.097553 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1444) took 1172 ms. Trace:
I20260709 12:57:40.035037 17056 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:57:40.129209 17034 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.597s	user 0.002s	sys 0.000s
W20260709 12:57:40.136494 17040 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:57:40.125226 17048 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.166633 16985 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.122670 16988 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.205963 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1504) took 1057 ms. Trace:
I20260709 12:57:40.212227 13538 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:40.212761 14246 raft_consensus.cc:614] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:57:40.128378 17041 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:57:40.084877 16811 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.914s	user 0.014s	sys 0.000s
I20260709 12:57:40.181980 13543 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:40.202558 13539 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:40.229247 16616 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:40.229810 16616 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:40.185293 16979 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.592s	user 0.008s	sys 0.008s
W20260709 12:57:40.231607 16979 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.726s	user 0.011s	sys 0.010s
I20260709 12:57:40.133204 17032 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.142967 16984 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.232400 16984 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:40.233587 16616 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:40.234716 16984 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.235121 16811 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:40.103564 17050 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:40.213644 13684 rpcz_store.cc:276] 0709 12:57:38.925524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.926704 (+  1180us) service_pool.cc:224] Handling call
0709 12:57:38.948845 (+ 22141us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:38.988547 (+ 39702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:38.988576 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:38.998773 (+ 10197us) log.cc:844] Serialized 52 byte log entry
0709 12:57:39.270002 (+271229us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:39.270059 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.270150 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.097240 (+827090us) raft_consensus.cc:1685] finished
0709 12:57:40.097270 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.097467 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19990,"raft.queue_time_us":14274,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8229,"thread_start_us":211267,"threads_started":1,"wal-append.queue_time_us":378017}
I20260709 12:57:40.081387 17058 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:40.126251 17044 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:57:40.096663 17059 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:40.183336 16958 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:40.237253 17058 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.237983 16958 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:40.237881 17059 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.043653 16976 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:40.238746 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1493) took 1094 ms. Trace:
I20260709 12:57:40.178256 13610 rpcz_store.cc:276] 0709 12:57:39.065466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.200254 (+134788us) service_pool.cc:224] Handling call
0709 12:57:39.268832 (+ 68578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:39.291902 (+ 23070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.291949 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.316363 (+ 24414us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.827658 (+511295us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:39.827694 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.827765 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.131394 (+303629us) raft_consensus.cc:1685] finished
0709 12:57:40.131419 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.131578 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4727,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":24918,"thread_start_us":428422,"threads_started":1,"wal-append.queue_time_us":619975}
I20260709 12:57:40.180517 16930 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.226698 13540 rpcz_store.cc:276] 0709 12:57:39.148281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.256682 (+108401us) service_pool.cc:224] Handling call
0709 12:57:39.272290 (+ 15608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:39.284741 (+ 12451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.284772 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.291643 (+  6871us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.563014 (+271371us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:39.563069 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.563153 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.205672 (+642519us) raft_consensus.cc:1685] finished
0709 12:57:40.205713 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.205882 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47162,"raft.queue_time_us":18794,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4949,"thread_start_us":214242,"threads_started":1,"wal-append.queue_time_us":806923}
I20260709 12:57:40.239161 13541 rpcz_store.cc:276] 0709 12:57:39.144026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.165248 (+ 21222us) service_pool.cc:224] Handling call
0709 12:57:39.165553 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:39.184098 (+ 18545us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:57:39.184143 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:39.267782 (+ 83639us) log.cc:844] Serialized 52 byte log entry
0709 12:57:39.554676 (+286894us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:39.554743 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.554837 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.238531 (+683694us) raft_consensus.cc:1685] finished
0709 12:57:40.238551 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.238680 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11682,"raft.queue_time_us":16114,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13155,"thread_start_us":238763,"threads_started":1,"wal-append.queue_time_us":674455}
I20260709 12:57:40.053869 16970 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.221165 17029 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:40.131244 16858 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:40.240953 13540 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:57:40.240034 17039 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:57:40.257344 17055 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.258280 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1609) took 1205 ms. Trace:
I20260709 12:57:40.258759 13612 rpcz_store.cc:276] 0709 12:57:39.053203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.184998 (+131795us) service_pool.cc:224] Handling call
0709 12:57:39.265363 (+ 80365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:39.269988 (+  4625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.270022 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.284994 (+ 14972us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.539110 (+254116us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:39.539169 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.539273 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.255173 (+715900us) raft_consensus.cc:1685] finished
0709 12:57:40.255202 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.258172 (+  2970us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4211,"raft.queue_time_us":13841,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":2538,"thread_start_us":184245,"threads_started":1,"wal-append.queue_time_us":592226}
I20260709 12:57:40.258219 17055 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.263471 17055 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.264402 17067 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.037055 16962 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.246348 16965 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.239877 17058 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.177781 17057 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.235710 16811 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.265045 17067 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.284754 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1494) took 1140 ms. Trace:
I20260709 12:57:40.242020 17029 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:40.274277 16934 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.285157 13537 rpcz_store.cc:276] 0709 12:57:39.144279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.166790 (+ 22511us) service_pool.cc:224] Handling call
0709 12:57:39.167085 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:39.182926 (+ 15841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.182953 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.196685 (+ 13732us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.422390 (+225705us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:39.422433 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.422507 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.284550 (+862043us) raft_consensus.cc:1685] finished
0709 12:57:40.284576 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.284700 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26232,"raft.queue_time_us":16879,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":66977,"spinlock_wait_cycles":1949568,"thread_start_us":140488,"threads_started":1,"wal-append.queue_time_us":625694}
I20260709 12:57:40.292672 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2882) took 1147 ms. Trace:
W20260709 12:57:40.291365 16911 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.004s
I20260709 12:57:40.292984 13686 rpcz_store.cc:276] 0709 12:57:39.144798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.735270 (+590472us) service_pool.cc:224] Handling call
0709 12:57:39.785524 (+ 50254us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:39.832061 (+ 46537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.832110 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:39.882187 (+ 50077us) log.cc:844] Serialized 52 byte log entry
0709 12:57:40.042671 (+160484us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:40.042741 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.042842 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.292479 (+249637us) raft_consensus.cc:1685] finished
0709 12:57:40.292497 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.292619 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15590,"raft.queue_time_us":40931,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":39452,"thread_start_us":104356,"threads_started":1,"wal-append.queue_time_us":187458}
I20260709 12:57:40.242615 17048 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:40.291502 17067 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.253944 17050 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.310967 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 634) took 1016 ms. Trace:
I20260709 12:57:40.311389 13539 rpcz_store.cc:276] 0709 12:57:39.294277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:40.228982 (+934705us) service_pool.cc:224] Handling call
0709 12:57:40.229363 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:40.229702 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:40.229732 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:40.230199 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:57:40.235579 (+  5380us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:40.235621 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.235689 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.310744 (+ 75055us) raft_consensus.cc:1685] finished
0709 12:57:40.310771 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.310901 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1421,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":4946,"threads_started":1,"wal-append.queue_time_us":20579}
I20260709 12:57:40.315588 17029 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:40.316288 13535 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:40.316979 16811 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.300124 16911 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.318696 16911 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:40.325810 17026 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.001s	user 0.002s	sys 0.000s
I20260709 12:57:40.327136 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1448) took 1386 ms. Trace:
I20260709 12:57:40.327486 13691 rpcz_store.cc:276] 0709 12:57:38.940903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.945603 (+  4700us) service_pool.cc:224] Handling call
0709 12:57:38.945955 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:38.966418 (+ 20463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:38.966447 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:38.980707 (+ 14260us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.117530 (+136823us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:39.117588 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.117702 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.325090 (+1207388us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:40.326818 (+  1728us) raft_consensus.cc:1685] finished
0709 12:57:40.326840 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.327070 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27491,"raft.queue_time_us":20506,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":6334,"spinlock_wait_cycles":36776704,"thread_start_us":78766,"threads_started":1,"wal-append.queue_time_us":260564}
W20260709 12:57:40.329838 17054 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:57:40.304790 16997 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.335276 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:40.335774 14239 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:57:40.314093 17048 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.346581 17057 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:40.347471 17064 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:57:40.310377 17050 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.348879 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 546) took 1269 ms. Trace:
I20260709 12:57:40.349262 13619 rpcz_store.cc:276] 0709 12:57:39.079854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.908621 (+828767us) service_pool.cc:224] Handling call
0709 12:57:39.913101 (+  4480us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:39.925309 (+ 12208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.925358 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:39.936484 (+ 11126us) log.cc:844] Serialized 52 byte log entry
0709 12:57:40.036207 (+ 99723us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:40.288928 (+252721us) log.cc:844] Serialized 14 byte log entry
0709 12:57:40.289177 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.289241 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.346842 (+ 57601us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:40.348534 (+  1692us) raft_consensus.cc:1685] finished
0709 12:57:40.348651 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.348815 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":276183,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":12828,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":23094,"spinlock_wait_cycles":2979712,"thread_start_us":301960,"threads_started":2,"wal-append.queue_time_us":210643}
I20260709 12:57:40.353067 17059 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:40.353806 16590 consensus_meta.cc:287] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.002s
I20260709 12:57:40.355636 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2879) took 1266 ms. Trace:
I20260709 12:57:40.363090 16703 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:40.363505 16703 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:40.364717 17051 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.393s	user 0.002s	sys 0.000s
I20260709 12:57:40.364388 16703 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.365568 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1506) took 1168 ms. Trace:
I20260709 12:57:40.365958 13538 rpcz_store.cc:276] 0709 12:57:39.197530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.342663 (+145133us) service_pool.cc:224] Handling call
0709 12:57:39.379787 (+ 37124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:40.226866 (+847079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.365327 (+138461us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b540028d054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:40.365502 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913950464}
I20260709 12:57:40.362723 13688 rpcz_store.cc:276] 0709 12:57:39.089576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.089708 (+   132us) service_pool.cc:224] Handling call
0709 12:57:39.103701 (+ 13993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:39.200261 (+ 96560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.200292 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.200710 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.500025 (+299315us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:39.500068 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.355207 (+855139us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b54003b33d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:40.355259 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.355309 (+    50us) raft_consensus.cc:1685] finished
0709 12:57:40.355324 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.355574 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21609,"raft.queue_time_us":21698,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8752,"spinlock_wait_cycles":171444096,"thread_start_us":231442,"threads_started":1,"wal-append.queue_time_us":583920}
I20260709 12:57:40.367587 16920 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:40.356310 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:40.356456 17005 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.379465 16920 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:40.379966 14238 raft_consensus.cc:2393] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:40.380095 14240 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 12:57:40.343797 17000 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:40.381119 16920 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.382392 16920 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:40.382659 17000 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.368306 16590 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.369302 16555 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.355394 17020 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.384694 13618 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:57:40.365973 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1499) took 1220 ms. Trace:
I20260709 12:57:40.373605 16968 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:40.388871 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2699) took 1240 ms. Trace:
I20260709 12:57:40.385438 13534 rpcz_store.cc:276] 0709 12:57:39.145521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.169469 (+ 23948us) service_pool.cc:224] Handling call
0709 12:57:39.169717 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:39.191509 (+ 21792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.191553 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.203390 (+ 11837us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.896276 (+692886us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:39.896337 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.896446 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.364115 (+467669us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:40.365675 (+  1560us) raft_consensus.cc:1685] finished
0709 12:57:40.365707 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.365912 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25540,"raft.queue_time_us":25680,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":62059,"spinlock_wait_cycles":2325888,"thread_start_us":604720,"threads_started":1,"wal-append.queue_time_us":672381}
I20260709 12:57:40.374158 17029 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.378777 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.381150 17005 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.387856 17020 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.389045 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.391363 13690 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:40.389148 13615 rpcz_store.cc:276] 0709 12:57:39.148394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.161158 (+ 12764us) service_pool.cc:224] Handling call
0709 12:57:40.388729 (+1227571us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b540013c4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:40.388825 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358287616}
I20260709 12:57:40.390455 13685 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:40.396886 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.397029 13686 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:57:40.398252 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:40.397014 16797 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.396572 17029 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.398717 16797 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.400216 16968 consensus_queue.cc:1243] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Peer 0f34cda059124f839711dbef61bb2c28 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:57:40.400671 16797 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.400827 17029 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.401124 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.403488 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:40.401746 13691 raft_consensus.cc:2376] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9. Re-sending same reply.
I20260709 12:57:40.406675 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.410247 16732 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:40.410696 16732 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:40.411042 16732 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:40.411415 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.412284 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.412539 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.413375 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.413770 16797 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.413959 13686 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:40.414279 16797 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:40.415758 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:40.416002 17020 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.416488 17020 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.417421 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:40.419766 16773 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:40.421249 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.426364 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:40.429050 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.429858 13685 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:40.431367 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:40.432767 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:40.418467 17020 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:40.434770 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.435313 13686 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:57:40.435451 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.435871 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.439320 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.439700 13617 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:40.440886 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.445122 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:40.447458 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:40.448766 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.449462 13619 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:40.452127 14209 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:40.454056 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.432799 16947 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:40.455143 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.427870 16773 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:40.463472 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.456550 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.457050 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.465005 13686 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:40.465224 13610 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:40.452772 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:40.432525 16990 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:40.476140 16990 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:40.476742 16990 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:40.437384 17016 raft_consensus.cc:2764] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:57:40.449558 16744 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:40.449767 17043 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:40.452442 16911 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:40.483115 13612 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:57:40.458777 17057 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:40.491065 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.491890 13686 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:40.462846 13685 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:40.451290 17050 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:40.497953 14243 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:40.452752 17005 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:57:40.452630 16970 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:57:40.451501 16858 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:57:40.504040 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.505033 13685 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:40.505156 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:40.505198 13619 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:57:40.505841 14243 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:40.460057 16732 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:40.482805 16947 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:40.508876 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:40.498242 17050 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:40.512171 13582 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:40.518946 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.512389 13686 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:40.500841 16706 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:40.521189 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.529479 17029 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:57:40.529579 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:40.530095 17029 raft_consensus.cc:1068] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Unable to promote non-voter 0f34cda059124f839711dbef61bb2c28: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:57:40.530421 13689 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:40.530736 17068 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
I20260709 12:57:40.531329 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.531579 16616 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:40.532312 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2700) took 1273 ms. Trace:
I20260709 12:57:40.532677 13614 rpcz_store.cc:276] 0709 12:57:39.259007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.975847 (+716840us) service_pool.cc:224] Handling call
0709 12:57:39.976309 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:39.976773 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:39.976817 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:39.993647 (+ 16830us) log.cc:844] Serialized 26 byte log entry
0709 12:57:40.229053 (+235406us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:40.229100 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.229166 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.530177 (+301011us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:40.532014 (+  1837us) raft_consensus.cc:1685] finished
0709 12:57:40.532033 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.532251 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4100,"spinlock_wait_cycles":3374976,"thread_start_us":181992,"threads_started":1,"wal-append.queue_time_us":232105}
W20260709 12:57:40.534161 13616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1589) took 1963 ms (client timeout 2055 ms). Trace:
I20260709 12:57:40.533371 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:40.534605 13616 rpcz_store.cc:269] 0709 12:57:38.570914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.571120 (+   206us) service_pool.cc:224] Handling call
0709 12:57:40.533985 (+1962865us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b540020a654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:40.534106 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39287168}
I20260709 12:57:40.535236 16978 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:40.535703 16978 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:40.536125 16978 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:40.537917 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:40.538339 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:40.540012 16950 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:40.541105 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1492) took 1397 ms. Trace:
I20260709 12:57:40.541515 13535 rpcz_store.cc:276] 0709 12:57:39.143766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.144398 (+   632us) service_pool.cc:224] Handling call
0709 12:57:40.540934 (+1396536us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400088ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:40.541045 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2615522688}
I20260709 12:57:40.543557 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 549) took 1400 ms. Trace:
I20260709 12:57:40.544734 16703 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:40.546238 16590 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.491523 17043 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:40.498550 16744 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:40.796442 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:41.027832 17085 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:57:40.652129 16616 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:40.994935 17052 raft_consensus.cc:993] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:57:41.035516 17053 raft_consensus.cc:1081] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:57:40.549602 13618 rpcz_store.cc:276] 0709 12:57:39.142843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.950200 (+807357us) service_pool.cc:224] Handling call
0709 12:57:39.955065 (+  4865us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:40.385084 (+430019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.543199 (+158115us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54002fb7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:40.543494 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987467648}
I20260709 12:57:41.036456 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:41.037039 16920 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.037552 16920 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.039613 16920 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.747165 16850 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.511498 16947 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:41.040669 16947 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:41.043380 13537 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:41.044258 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.045320 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.045565 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.045941 13542 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:40.742390 17084 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:57:41.046741 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:41.046823 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.044483 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.048321 17043 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.047854 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.048838 13537 raft_consensus.cc:2393] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:40.760711 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 974) took 1473 ms. Trace:
I20260709 12:57:41.049192 13539 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:41.049678 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:41.050161 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:41.050800 14243 raft_consensus.cc:2410] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:57:41.051013 13539 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:41.052973 16920 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.053123 16950 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.034868 16616 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.053457 16920 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.053714 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:41.053656 16950 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.055388 16616 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:41.055625 16920 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.055847 16950 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:40.607191 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:41.056396 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:41.040758 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.766964 16960 raft_consensus.cc:993] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:57:41.057447 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:41.057973 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:41.058485 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:40.769193 17066 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.542s	user 0.002s	sys 0.000s
I20260709 12:57:40.884647 16817 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.175s
I20260709 12:57:41.060608 16817 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:41.060989 16817 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:40.667822 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 455) took 1380 ms. Trace:
W20260709 12:57:41.062093 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:40.517771 17050 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:41.062546 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:40.659041 17025 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.337s	user 0.002s	sys 0.002s
I20260709 12:57:40.735996 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:40.765251 16878 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:41.065114 16878 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:41.065514 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:41.065609 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 561) took 2143 ms. Trace:
I20260709 12:57:40.722232 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 627) took 1975 ms. Trace:
I20260709 12:57:41.066607 13543 rpcz_store.cc:276] 0709 12:57:38.747062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.756010 (+  8948us) service_pool.cc:224] Handling call
0709 12:57:40.721934 (+1965924us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54003c9e54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:40.722129 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3255999744}
W20260709 12:57:40.728371 13693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1447) took 1788 ms (client timeout 2136 ms). Trace:
I20260709 12:57:41.066164 14242 rpcz_store.cc:276] 0709 12:57:38.922438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.922647 (+   209us) service_pool.cc:224] Handling call
0709 12:57:38.922958 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:38.923323 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:38.923351 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:38.939886 (+ 16535us) log.cc:844] Serialized 26 byte log entry
0709 12:57:39.204956 (+265070us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:39.205007 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:39.205089 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.658445 (+1453356us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:41.065320 (+406875us) raft_consensus.cc:1685] finished
0709 12:57:41.065342 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:41.065533 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":494,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":7790,"spinlock_wait_cycles":18119040,"thread_start_us":204997,"threads_started":1,"wal-append.queue_time_us":287773}
I20260709 12:57:40.631227 16797 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:41.058560 13684 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:40.549683 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:40.668751 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:40.758004 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:41.071506 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:40.550295 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:41.098384 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:40.600943 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:40.760540 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:40.757059 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:41.100598 13693 rpcz_store.cc:269] 0709 12:57:38.940478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:38.942186 (+  1708us) service_pool.cc:224] Handling call
0709 12:57:40.728079 (+1785893us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5400048dd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:40.728293 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23724288}
I20260709 12:57:41.057662 16960 raft_consensus.cc:1081] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:57:41.114424 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:41.086359 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:40.728206 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:57:41.141690 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:41.156457 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 616572us
I20260709 12:57:41.135084 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:41.157943 14237 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:41.159078 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:41.161612 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:40.775018 17086 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:57:41.162083 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:41.159252 16616 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:41.049475 14244 rpcz_store.cc:276] 0709 12:57:39.286806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:39.286938 (+   132us) service_pool.cc:224] Handling call
0709 12:57:39.297485 (+ 10547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:39.381465 (+ 83980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.760461 (+1378996us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b54007238d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:40.760643 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74960128}
W20260709 12:57:41.132656 13690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.008s	sys 0.017s
I20260709 12:57:41.172410 13616 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:41.172609 13685 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:41.171180 14240 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:41.174086 13543 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:57:41.188956 13690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.011s	sys 0.017s
I20260709 12:57:41.177608 13541 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:40.734814 16962 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:41.100417 13692 rpcz_store.cc:276] 0709 12:57:39.287381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:40.137759 (+850378us) service_pool.cc:224] Handling call
0709 12:57:40.138166 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:40.138425 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:40.220982 (+ 82557us) log.cc:844] Serialized 14 byte log entry
0709 12:57:40.667067 (+446085us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b1800469180. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a
0709 12:57:40.667325 (+   258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:40.667395 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:40.667467 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.667560 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:40.667742 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":106701,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":107,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":14458,"spinlock_wait_cycles":4946176,"thread_start_us":131207,"threads_started":2,"wal-append.queue_time_us":85952}
I20260709 12:57:41.156780 14238 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:41.177179 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:57:41.200732 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 650513us
W20260709 12:57:40.751104 17070 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.477s	user 0.002s	sys 0.000s
I20260709 12:57:40.568711 17003 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:41.227188 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:40.653586 16990 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:40.867512 17017 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.224813 13684 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:57:40.886739 16968 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.237193 13684 pending_rounds.cc:85] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 7
I20260709 12:57:41.237599 13684 pending_rounds.cc:107] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:57:40.739987 17029 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:41.215775 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:41.237354 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.222751 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:41.233846 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:41.213955 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:41.164323 17000 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:40.739992 16966 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:41.235870 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:41.243474 13616 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:41.216531 16767 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:41.237257 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:41.132462 17063 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.003s	sys 0.019s
I20260709 12:57:41.239439 13684 raft_consensus.cc:2889] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:41.239298 13618 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:41.070595 17023 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.246439 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.216807 16920 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:40.606590 16978 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.247851 14241 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:41.254492 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:41.274101 13613 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:41.232410 16616 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.282879 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:41.244149 16966 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:41.262918 16990 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.264823 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.265318 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.232861 16803 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:57:41.264642 16732 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.272980 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1507) took 2006 ms. Trace:
I20260709 12:57:41.275959 13535 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:57:41.157418 16817 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.257872 16590 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:57:41.272904 17063 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.004s	sys 0.020s
I20260709 12:57:41.236979 16968 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.305274 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:41.306499 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:41.308403 16968 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:41.308955 16850 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:41.286798 13611 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:40.763020 16958 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.286693 13617 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:41.291177 13536 rpcz_store.cc:276] 0709 12:57:39.266258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:40.012808 (+746550us) service_pool.cc:224] Handling call
0709 12:57:40.035367 (+ 22559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:40.035694 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:40.035720 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:40.040823 (+  5103us) log.cc:844] Serialized 26 byte log entry
0709 12:57:40.139294 (+ 98471us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:40.139336 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.139402 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:40.750442 (+611040us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:41.265440 (+514998us) raft_consensus.cc:1685] finished
0709 12:57:41.265461 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:41.272894 (+  7433us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56334,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":7484,"spinlock_wait_cycles":30906752,"thread_start_us":48242,"threads_started":1,"wal-append.queue_time_us":148634}
I20260709 12:57:41.246850 17000 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.244324 17029 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:41.300450 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 143076us
I20260709 12:57:41.258102 17017 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:41.302002 13534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.011s	sys 0.009s
I20260709 12:57:41.286027 16990 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:41.320218 13689 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:41.324673 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.337157 13613 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:41.328768 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:41.331358 13534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.011s	sys 0.009s
I20260709 12:57:41.338358 13536 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:41.331732 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:41.340840 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:41.279000 17023 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.296934 16590 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:41.315002 17052 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:41.282429 16920 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.349776 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:41.353387 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.346350 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.355280 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.287873 16616 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.345618 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:41.365338 17000 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.367081 17029 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:41.348829 16732 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:57:41.399263 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:41.327683 16767 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:41.401360 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:41.403443 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53768.9us
I20260709 12:57:41.375339 17052 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:41.443169 13543 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:41.444855 13535 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:41.375110 16590 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:41.457083 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:41.459681 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 395512us
I20260709 12:57:41.470104 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:41.471983 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:41.466406 13689 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:41.489130 13534 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:57:41.426419 17020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.020s	sys 0.013s
I20260709 12:57:41.436933 16970 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:41.345041 16990 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.453222 16950 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:41.437598 17052 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:41.519104 16947 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.440263 17000 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:41.511565 16744 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:41.524714 16978 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:41.467495 16774 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:57:41.541481 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:41.505359 17020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.020s	sys 0.013s
I20260709 12:57:41.488622 16767 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:41.546679 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:57:41.549175 13534 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:57:41.512861 16970 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:41.576485 16850 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:57:41.576989 17088 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 12:57:41.577310 16850 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:41.577713 16850 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:41.578076 16850 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:41.581004 16616 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:41.587385 16878 consensus_meta.cc:287] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.000s
I20260709 12:57:41.589385 16990 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:41.590474 17017 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.592196 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1459) took 1169 ms. Trace:
I20260709 12:57:41.592589 13687 rpcz_store.cc:276] 0709 12:57:40.422911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:40.423045 (+   134us) service_pool.cc:224] Handling call
0709 12:57:40.436803 (+ 13758us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:40.437107 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:40.437132 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:40.441642 (+  4510us) log.cc:844] Serialized 26 byte log entry
0709 12:57:40.562167 (+120525us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:40.562207 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.562278 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:41.591981 (+1029703us) raft_consensus.cc:1685] finished
0709 12:57:41.592000 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:41.592133 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":170,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":12103,"thread_start_us":120204,"threads_started":1,"wal-append.queue_time_us":858080}
I20260709 12:57:41.594334 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:41.594892 13687 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:41.598336 16773 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.117s	user 0.013s	sys 0.000s
I20260709 12:57:41.598999 16773 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.604027 16878 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.604775 16850 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.605756 13542 raft_consensus.cc:2393] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:57:41.606598 17050 consensus_meta.cc:287] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.001s
I20260709 12:57:41.607079 17050 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.608852 17023 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.547943 17000 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.628944 14237 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:41.629551 13690 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: [9.7-10.8]
I20260709 12:57:41.545024 16803 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:41.633946 13537 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:41.644274 13536 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:41.664565 13687 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
W20260709 12:57:41.548000 16997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.008s	sys 0.026s
I20260709 12:57:41.608717 16878 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:41.669122 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1467) took 1209 ms. Trace:
I20260709 12:57:41.695297 13689 rpcz_store.cc:276] 0709 12:57:40.459483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:40.568560 (+109077us) service_pool.cc:224] Handling call
0709 12:57:41.668936 (+1100376us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b540012f054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:41.669056 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605054336}
I20260709 12:57:41.670208 13692 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:57:41.688711 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:41.691581 13534 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:41.612191 16970 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:41.698809 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 521656us
W20260709 12:57:41.690768 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.015s	sys 0.008s
W20260709 12:57:41.700524 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.015s	sys 0.008s
W20260709 12:57:41.668299 16997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.153s	user 0.008s	sys 0.027s
I20260709 12:57:41.665732 16773 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:41.748052 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.010s	sys 0.014s
W20260709 12:57:41.748574 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.010s	sys 0.014s
W20260709 12:57:41.748939 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 990955us
I20260709 12:57:41.749778 14211 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:41.767249 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:41.770088 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.018s	sys 0.007s
W20260709 12:57:41.770483 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.018s	sys 0.007s
W20260709 12:57:41.770803 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 556889us
W20260709 12:57:41.698350 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.006s	sys 0.018s
W20260709 12:57:41.771682 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.018s
W20260709 12:57:41.772195 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 656234us
I20260709 12:57:41.782881 13534 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:57:41.769495 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:41.783545 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:41.771556 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:41.812407 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:41.826242 14237 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:41.829780 13508 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:41.832981 13617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d73171142594388972ed22eae44f95f"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:39902
I20260709 12:57:41.833655 13617 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:57:41.834028 13617 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:41.835649 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.837517 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:41.837598 13616 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
W20260709 12:57:41.801714 17043 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.006s	sys 0.011s
I20260709 12:57:41.703292 17003 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:41.666657 16803 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:41.842693 14241 raft_consensus.cc:2393] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:41.843724 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.016s	sys 0.059s
I20260709 12:57:41.845445 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.810940 17017 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:41.811605 17000 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:41.850505 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.016s	sys 0.059s
I20260709 12:57:41.847838 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.828445 17020 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:41.827698 16878 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:57:41.842139 17043 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.006s	sys 0.012s
W20260709 12:57:41.860091 13582 connection.cc:256] RPC call timeout handler was delayed by -1.1541214179997041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
I20260709 12:57:41.868666 13617 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.844980 16850 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.856294 16920 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:41.874822 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:41.876071 13618 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:57:41.876452 13616 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:41.710644 16767 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.862843 16773 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:41.873958 17053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.009s	sys 0.029s
I20260709 12:57:41.890952 13611 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:41.868474 17000 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:41.892374 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:41.892643 13617 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:41.893613 14245 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.001s	sys 0.018s
W20260709 12:57:41.894820 13688 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.013s	sys 0.007s
W20260709 12:57:41.896132 13688 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.014s	sys 0.007s
W20260709 12:57:41.895040 14245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.001s	sys 0.018s
I20260709 12:57:41.870267 17020 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:41.878368 17063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.002s	sys 0.036s
I20260709 12:57:41.871838 17023 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:41.880033 16934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.335s	user 0.010s	sys 0.018s
W20260709 12:57:41.943226 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 471297us
W20260709 12:57:41.894662 17066 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.006s	sys 0.015s
I20260709 12:57:41.944998 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:41.876829 16990 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:41.946575 13684 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.010s	sys 0.012s
W20260709 12:57:41.898844 17053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.009s	sys 0.030s
W20260709 12:57:41.945376 16934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.400s	user 0.010s	sys 0.018s
W20260709 12:57:41.939607 17063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.002s	sys 0.036s
W20260709 12:57:41.993660 13684 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.010s	sys 0.013s
W20260709 12:57:41.994689 13613 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.007s	sys 0.017s
W20260709 12:57:41.949159 17066 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.006s	sys 0.016s
I20260709 12:57:41.948200 16947 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:42.004554 13693 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.025s
W20260709 12:57:42.005972 13693 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.009s	sys 0.025s
I20260709 12:57:41.949524 17050 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:42.001242 13613 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.007s	sys 0.018s
W20260709 12:57:41.962405 16968 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.011s	sys 0.009s
W20260709 12:57:42.008739 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 471981us
I20260709 12:57:41.948930 17023 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:42.003835 17094 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 12:57:42.017607 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:42.029019 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 245527us
W20260709 12:57:42.005752 17085 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:57:41.918601 16773 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:41.919265 17000 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:41.974218 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:42.030400 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 200673us
W20260709 12:57:42.021308 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.016s	sys 0.010s
I20260709 12:57:41.998225 16990 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:42.036806 13657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.000s	sys 0.024s
I20260709 12:57:42.015705 16947 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:41.963145 17100 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:57:42.085657 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:42.086292 13657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.113s	user 0.000s	sys 0.025s
I20260709 12:57:42.023589 17063 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:42.086694 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.016s	sys 0.010s
I20260709 12:57:42.022706 17050 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:41.958918 17003 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:42.002601 17109 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 12:57:42.123744 13543 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.005s	sys 0.004s
W20260709 12:57:42.125998 13543 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.006s	sys 0.005s
W20260709 12:57:42.126658 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 182321us
I20260709 12:57:42.127508 13509 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:57:42.091566 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 621493us
W20260709 12:57:42.041905 16590 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.005s
I20260709 12:57:42.125885 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1469) took 1659 ms. Trace:
I20260709 12:57:42.092377 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:42.104382 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 272061us
I20260709 12:57:42.106513 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1466) took 1648 ms. Trace:
I20260709 12:57:42.084899 17107 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:42.130848 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:42.131474 13686 rpcz_store.cc:276] 0709 12:57:40.457841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.134868 (+677027us) service_pool.cc:224] Handling call
0709 12:57:41.135195 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:41.135456 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:41.209789 (+ 74333us) log.cc:844] Serialized 14 byte log entry
0709 12:57:41.814112 (+604323us) log.cc:844] Serialized 14 byte log entry
0709 12:57:42.106025 (+291913us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:42.106098 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:42.106180 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:42.106261 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:42.106430 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49888,"prepare.queue_time_us":625532,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":11050,"raft.queue_time_us":376259,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":22882,"thread_start_us":782996,"threads_started":2,"wal-append.queue_time_us":807364}
I20260709 12:57:42.135954 13616 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:42.042276 17105 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:42.148687 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:42.152704 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:42.156410 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 70935.8us
I20260709 12:57:42.157333 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:42.187502 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:42.129637 13693 rpcz_store.cc:276] 0709 12:57:40.466777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.219046 (+752269us) service_pool.cc:224] Handling call
0709 12:57:41.308019 (+ 88973us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:57:41.308201 (+   182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:41.318994 (+ 10793us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:42.006458 (+687464us) log.cc:844] Serialized 14 byte log entry
0709 12:57:42.125379 (+118921us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:42.125492 (+   113us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:42.125540 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:42.125605 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:42.125827 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101261,"prepare.queue_time_us":621152,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":5278,"spinlock_wait_cycles":13676928,"thread_start_us":657708,"threads_started":2}
I20260709 12:57:42.031373 17043 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 12:57:42.091048 17063 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:42.099781 17050 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:42.194419 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:42.031080 16934 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 12:57:42.128685 17121 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:57:42.203253 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 161313us
W20260709 12:57:42.195199 17092 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.324s	user 0.001s	sys 0.000s
I20260709 12:57:42.146826 16803 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:42.197929 17099 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
I20260709 12:57:42.196026 16920 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:57:42.216225 17091 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
W20260709 12:57:42.218142 17095 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.348s	user 0.000s	sys 0.001s
I20260709 12:57:42.205907 17063 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:42.031134 17020 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:42.224756 17096 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 12:57:42.218942 17100 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:42.182668 16590 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:42.285738 17103 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:57:42.215706 17050 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:42.227285 17133 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:42.231770 16803 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:42.189445 17107 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:42.194207 17105 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:42.227840 17108 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:42.338245 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 497781us
I20260709 12:57:42.339600 13584 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:42.361747 14241 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:42.294312 17100 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:42.304513 17043 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:42.305133 17109 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:57:42.392627 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 259161us
I20260709 12:57:42.506916 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:42.393267 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 235741us
I20260709 12:57:42.504758 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 639) took 1464 ms. Trace:
I20260709 12:57:42.508174 13538 rpcz_store.cc:276] 0709 12:57:41.040415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.040540 (+   125us) service_pool.cc:224] Handling call
0709 12:57:41.040868 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:41.041200 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:41.041233 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:41.041663 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:57:41.428429 (+386766us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:41.428487 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:41.428583 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:42.504492 (+1075909us) raft_consensus.cc:1685] finished
0709 12:57:42.504517 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:42.504684 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":250,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":386323,"threads_started":1,"wal-append.queue_time_us":769528}
W20260709 12:57:42.374331 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 210948us
W20260709 12:57:42.521942 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:42.522106 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1510) took 1476 ms. Trace:
I20260709 12:57:42.525525 16744 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:42.522972 13539 rpcz_store.cc:276] 0709 12:57:41.045921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.051680 (+  5759us) service_pool.cc:224] Handling call
0709 12:57:41.051961 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:41.052288 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:41.052315 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:41.052714 (+   399us) log.cc:844] Serialized 26 byte log entry
0709 12:57:41.457848 (+405134us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:41.457886 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:41.457955 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:42.521853 (+1063898us) raft_consensus.cc:1685] finished
0709 12:57:42.521875 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:42.522043 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":615,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":144128,"thread_start_us":404726,"threads_started":1,"wal-append.queue_time_us":788630}
W20260709 12:57:42.531888 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 91976.7us
I20260709 12:57:42.535131 16850 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:42.537384 16947 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:42.538201 17067 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:42.538981 16962 consensus_meta.cc:287] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.321s	user 0.009s	sys 0.001s
I20260709 12:57:42.539584 17131 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:57:42.540557 17129 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
I20260709 12:57:42.541513 16990 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:42.663180 16703 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:42.663048 17131 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:57:42.503978 16773 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:42.544377 16958 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:42.545651 16732 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:42.548965 17121 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:42.550771 17107 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:42.338306 17133 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:42.569065 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:42.552762 17052 consensus_meta.cc:287] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.003s	user 0.011s	sys 0.000s
W20260709 12:57:42.600538 17114 log.cc:927] Time spent T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 12:57:42.384842 16803 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:42.397158 17100 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:42.570855 16978 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:57:42.628659 17023 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:42.601437 17141 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.315s	user 0.000s	sys 0.000s
I20260709 12:57:42.563740 17140 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:57:42.504276 17108 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:42.400235 17043 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:42.567185 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:42.584148 16744 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:42.385748 17111 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:57:42.519323 17103 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:57:42.661310 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1470) took 2179 ms. Trace:
I20260709 12:57:42.672840 16703 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:42.791173 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1433) took 1438 ms. Trace:
W20260709 12:57:43.037463 13509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.002s	sys 0.039s
W20260709 12:57:42.954412 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.029s	sys 0.067s
I20260709 12:57:42.755882 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:43.068554 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.029s	sys 0.078s
W20260709 12:57:43.069332 13509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.002s	sys 0.042s
W20260709 12:57:43.067276 13506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.011s	sys 0.008s
W20260709 12:57:43.070523 13507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.011s	sys 0.035s
W20260709 12:57:43.070636 13506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.012s	sys 0.008s
W20260709 12:57:43.070915 13507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.011s	sys 0.035s
I20260709 12:57:42.709784 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:43.071841 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:42.956923 13582 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.001s	sys 0.013s
I20260709 12:57:42.710361 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:42.709509 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:43.072724 13582 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.001s	sys 0.013s
W20260709 12:57:43.070834 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 563977us
I20260709 12:57:42.756057 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:43.070133 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 849300us
W20260709 12:57:43.020117 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:43.073271 13689 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:42.663115 16774 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:43.088626 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 521488us
W20260709 12:57:43.073798 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 936084us
I20260709 12:57:42.946038 16616 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:43.038479 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1511) took 1992 ms. Trace:
I20260709 12:57:42.641572 17136 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:43.096956 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.028789 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1512) took 1825 ms. Trace:
I20260709 12:57:43.212291 13539 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:43.212167 13543 rpcz_store.cc:276] 0709 12:57:41.203265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.490313 (+287048us) service_pool.cc:224] Handling call
0709 12:57:41.491009 (+   696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:42.127246 (+636237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:42.127450 (+   204us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:42.128185 (+   735us) log.cc:844] Serialized 26 byte log entry
0709 12:57:42.389977 (+261792us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:42.390042 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:42.526423 (+136381us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:43.028474 (+502051us) raft_consensus.cc:1685] finished
0709 12:57:43.028506 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:43.028708 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":384498,"spinlock_wait_cycles":17976576,"thread_start_us":824236,"threads_started":2,"wal-append.queue_time_us":799689}
I20260709 12:57:43.072259 13686 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:42.650687 16947 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:43.141814 13541 rpcz_store.cc:276] 0709 12:57:41.143190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.422378 (+279188us) service_pool.cc:224] Handling call
0709 12:57:41.440606 (+ 18228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:41.889174 (+448568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:41.889225 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:41.889851 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:57:42.186616 (+296765us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:42.186661 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:42.186737 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:43.038133 (+851396us) raft_consensus.cc:1685] finished
0709 12:57:43.038168 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:43.038394 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9774,"raft.queue_time_us":239415,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":33520,"thread_start_us":602700,"threads_started":2,"wal-append.queue_time_us":1067695}
W20260709 12:57:42.787526 17093 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.232s	user 0.021s	sys 0.011s
I20260709 12:57:43.168116 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.221515 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:43.025554 17127 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.351s	user 0.000s	sys 0.000s
W20260709 12:57:42.673537 17087 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.206s	user 0.005s	sys 0.005s
W20260709 12:57:42.673383 17149 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.057s	user 0.001s	sys 0.001s
W20260709 12:57:43.168778 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 94460.9us
I20260709 12:57:43.243674 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:42.954931 17098 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.096s	sys 0.125s
I20260709 12:57:42.935472 17146 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:42.673060 17105 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:42.808560 17000 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.207283 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.072809 13691 rpcz_store.cc:276] 0709 12:57:40.482009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.160191 (+678182us) service_pool.cc:224] Handling call
0709 12:57:41.160651 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:41.161077 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:41.161116 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:41.161710 (+   594us) log.cc:844] Serialized 52 byte log entry
0709 12:57:41.488335 (+326625us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:57:41.488393 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:41.488472 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:42.661027 (+1172555us) raft_consensus.cc:1685] finished
0709 12:57:42.661057 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:42.661239 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":241752,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":34394,"thread_start_us":326164,"threads_started":1,"wal-append.queue_time_us":538541}
I20260709 12:57:42.757505 17138 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:42.676864 16960 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.027251 16817 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:42.662925 16506 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:43.216250 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:43.067075 13542 rpcz_store.cc:276] 0709 12:57:41.352439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.787271 (+434832us) service_pool.cc:224] Handling call
0709 12:57:41.833790 (+ 46519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:41.834038 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:42.105196 (+271158us) log.cc:844] Serialized 14 byte log entry
0709 12:57:42.790551 (+685355us) spinlock_profiling.cc:243] Waited 9.34 ms on lock 0x7b18004123c0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287
0709 12:57:42.790683 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:42.790767 (+    84us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:42.790833 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:42.790917 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:42.791102 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":470103,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":63,"raft.queue_time_us":419714,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":12937,"spinlock_wait_cycles":21471616,"thread_start_us":541704,"threads_started":3,"wal-append.queue_time_us":600566}
W20260709 12:57:42.957078 16997 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.115s	user 0.010s	sys 0.005s
W20260709 12:57:43.011893 16966 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.718s	user 0.015s	sys 0.000s
I20260709 12:57:43.225755 14237 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:43.069296 17111 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:42.969658 16590 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:43.227038 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:42.656713 17150 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:43.228380 17143 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:57:43.223770 17136 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.229079 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:43.229362 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:43.229867 17137 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.232292 17067 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.233240 17023 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:43.135357 17134 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:43.101961 17157 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:43.235770 14211 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:43.234086 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:43.083761 17133 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:43.133355 16773 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:43.238903 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:43.256901 13508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.022s	sys 0.045s
W20260709 12:57:43.503001 13508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.022s	sys 0.045s
I20260709 12:57:43.503083 14240 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
W20260709 12:57:43.269111 13685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.149s	user 0.003s	sys 0.019s
W20260709 12:57:43.504284 13685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.384s	user 0.003s	sys 0.019s
I20260709 12:57:43.505590 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2900) took 2983 ms. Trace:
I20260709 12:57:43.220633 17151 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.476270 16920 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:43.329380 17150 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.509140 17151 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.506222 13685 rpcz_store.cc:276] 0709 12:57:40.522526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.191833 (+669307us) service_pool.cc:224] Handling call
0709 12:57:41.233152 (+ 41319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:41.233362 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:41.702450 (+469088us) log.cc:844] Serialized 14 byte log entry
0709 12:57:43.504914 (+1802464us) log.cc:844] Serialized 14 byte log entry
0709 12:57:43.505164 (+   250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:43.505217 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:43.505280 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:43.505372 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:43.505522 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64025,"prepare.queue_time_us":342205,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":283029,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":120679,"thread_start_us":2222007,"threads_started":3,"wal-append.queue_time_us":524183}
I20260709 12:57:43.511126 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.291142 13687 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:43.511654 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.512300 13685 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:43.512880 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:43.513103 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.513617 13685 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:43.513945 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.514386 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.298208 14239 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:43.258625 17098 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.105s	sys 0.135s
I20260709 12:57:43.293761 16960 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.433182 16947 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.516021 17151 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.474633 17136 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:43.516904 13685 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 10 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:57:43.517815 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.518761 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.476226 17111 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.519518 13685 raft_consensus.cc:2410] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:57:43.519600 17111 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:43.461416 17144 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
W20260709 12:57:43.460561 17147 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 12:57:43.472745 16590 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:57:43.302511 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:43.526091 13542 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:43.526557 17115 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.344s	user 0.006s	sys 0.007s
I20260709 12:57:43.527676 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 577) took 2496 ms. Trace:
I20260709 12:57:43.528558 17104 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:57:43.298528 16997 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.463s	user 0.013s	sys 0.005s
I20260709 12:57:43.304090 17108 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.237300 16968 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:43.175954 16616 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.535624 13542 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:43.537302 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.119671 17121 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.278281 17158 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:43.321147 13539 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:43.326051 13535 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:43.335238 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1508) took 2280 ms. Trace:
W20260709 12:57:43.316233 16970 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.610s	user 0.005s	sys 0.010s
I20260709 12:57:43.511536 17150 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.483752 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.551146 13534 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:43.288892 17146 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.306788 17160 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:43.497908 13536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1515) took 2189 ms (client timeout 2167 ms). Trace:
I20260709 12:57:43.555567 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:43.555732 13536 rpcz_store.cc:269] 0709 12:57:41.244670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.339127 (+ 94457us) service_pool.cc:224] Handling call
0709 12:57:43.432277 (+2093150us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54000aad54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:43.433653 (+  1376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642101632}
I20260709 12:57:43.470431 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1610) took 3051 ms. Trace:
I20260709 12:57:43.434212 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2902) took 2775 ms. Trace:
W20260709 12:57:43.503491 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 995053us
I20260709 12:57:43.257876 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:43.276912 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:43.328871 17138 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.509519 16920 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:43.514505 13687 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:43.077342 17052 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.176597 16774 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:43.528955 17104 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:43.240316 17043 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.528059 14245 rpcz_store.cc:276] 0709 12:57:41.031011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.031174 (+   163us) service_pool.cc:224] Handling call
0709 12:57:41.031608 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:41.032061 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:41.032102 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:41.032658 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:57:41.325477 (+292819us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b1800422440. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:41.899511 (+574034us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:41.899548 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:41.899621 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:43.526003 (+1626382us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:43.527470 (+  1467us) raft_consensus.cc:1685] finished
0709 12:57:43.527491 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:43.527616 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9143,"raft.queue_time_us":864,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":65851,"spinlock_wait_cycles":32329984,"thread_start_us":573843,"threads_started":1,"wal-append.queue_time_us":930393}
I20260709 12:57:43.296627 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:43.534200 14237 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
W20260709 12:57:43.469307 17050 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.130s	user 0.009s	sys 0.003s
I20260709 12:57:43.535823 13685 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:57:43.307909 16966 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:43.310892 13686 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:43.543375 17145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.026s	sys 0.059s
I20260709 12:57:43.402081 17157 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:43.549121 17151 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:43.569994 16590 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:43.546299 13539 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:57:43.542304 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 308321us
I20260709 12:57:43.478587 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.544575 13537 rpcz_store.cc:276] 0709 12:57:41.045444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.049448 (+  4004us) service_pool.cc:224] Handling call
0709 12:57:43.325664 (+2276216us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b54000dc354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:43.325879 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341468416}
I20260709 12:57:43.507380 17125 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:43.560689 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 302889us
I20260709 12:57:43.561370 13692 rpcz_store.cc:276] 0709 12:57:40.658791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.262934 (+604143us) service_pool.cc:224] Handling call
0709 12:57:41.669879 (+406945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:41.697858 (+ 27979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:43.432115 (+1734257us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b540012f050. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:43.434130 (+  2015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935071616}
W20260709 12:57:43.565372 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 337094us
I20260709 12:57:43.566486 13688 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:43.566888 13542 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
W20260709 12:57:43.566206 13684 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.182s	user 0.026s	sys 0.043s
I20260709 12:57:43.317948 17140 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.556985 13615 rpcz_store.cc:276] 0709 12:57:40.418570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:40.418690 (+   120us) service_pool.cc:224] Handling call
0709 12:57:40.418977 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:40.419392 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:40.419421 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:40.419878 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:57:40.623885 (+204007us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:40.623985 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:40.624092 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:42.672958 (+2048866us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:43.470006 (+797048us) raft_consensus.cc:1685] finished
0709 12:57:43.470038 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:43.470351 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3016,"raft.queue_time_us":18363,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":9173,"spinlock_wait_cycles":48488192,"thread_start_us":203506,"threads_started":1,"wal-append.queue_time_us":913730}
I20260709 12:57:43.547081 17108 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:43.313706 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.581410 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.623844 13686 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:43.624959 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:43.556243 16797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.031s	sys 0.054s
I20260709 12:57:43.404388 17137 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.501092 17155 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:43.311829 16950 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:43.571450 17150 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:43.539641 16968 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:43.604247 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.163s	user 0.046s	sys 0.032s
W20260709 12:57:43.641426 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.046s	sys 0.032s
I20260709 12:57:43.603659 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:43.598783 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:43.642194 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 1.30255e+06us
W20260709 12:57:43.643127 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 44464.4us
I20260709 12:57:43.645329 13509 leader_election.cc:400] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
I20260709 12:57:43.645685 13509 leader_election.cc:403] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:57:43.600028 13684 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.026s	sys 0.043s
I20260709 12:57:43.646812 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 466) took 2605 ms. Trace:
I20260709 12:57:43.647233 13684 rpcz_store.cc:276] 0709 12:57:41.041269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.284413 (+243144us) service_pool.cc:224] Handling call
0709 12:57:41.284970 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:41.285203 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:42.002163 (+716960us) log.cc:844] Serialized 14 byte log entry
0709 12:57:43.646382 (+1644219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:43.646453 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:43.646513 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:43.646600 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:43.646743 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23334,"prepare.queue_time_us":415337,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":405670,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":48161,"thread_start_us":2318929,"threads_started":3,"wal-append.queue_time_us":1055990}
I20260709 12:57:43.591722 17168 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.609992 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:43.664680 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:43.665336 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55425.1us
I20260709 12:57:43.573342 16803 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.609854 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:43.666646 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:43.568774 17023 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:43.591114 17145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.026s	sys 0.059s
W20260709 12:57:43.629974 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.402s	user 0.036s	sys 0.067s
I20260709 12:57:43.592898 17157 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:43.501696 17134 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.570850 17160 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.556100 16970 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.592705 17167 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:43.627072 16797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.032s	sys 0.054s
I20260709 12:57:43.588677 17131 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:43.652989 13691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:40872
I20260709 12:57:43.697459 13691 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:57:43.697934 13691 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:43.654189 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:43.595088 13659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.569s	user 0.031s	sys 0.044s
W20260709 12:57:43.698946 13659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.673s	user 0.031s	sys 0.044s
W20260709 12:57:43.699625 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 1.68196e+06us
I20260709 12:57:43.700501 13659 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:43.580652 16935 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:57:43.608860 16590 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:43.627732 16767 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.076s	user 0.007s	sys 0.008s
I20260709 12:57:43.608582 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:43.604887 16744 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.568589 17067 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:43.674582 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.447s	user 0.036s	sys 0.067s
W20260709 12:57:43.712029 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 1.5245e+06us
I20260709 12:57:43.630127 13543 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:43.585624 17050 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.681257 16968 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:43.682155 16732 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:43.685077 17146 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:43.623992 17125 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.686081 17120 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:43.686754 17052 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.687608 17169 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:43.652225 16950 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:43.690346 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 80843.9us
I20260709 12:57:43.640813 16703 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.611209 17151 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:43.698102 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 466557us
I20260709 12:57:43.667505 16506 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
I20260709 12:57:43.707376 13541 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:43.708410 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:43.668071 17164 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:43.707201 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.768977 13537 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:43.718619 14212 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:43.695097 17107 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:57:43.702905 16934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.385s	user 0.098s	sys 0.159s
I20260709 12:57:43.626551 17140 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.729204 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:43.708271 16590 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:43.652783 17155 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:43.713800 16767 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.007s	sys 0.008s
I20260709 12:57:43.762509 13538 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
W20260709 12:57:43.719542 17097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.937s	user 0.046s	sys 0.100s
I20260709 12:57:43.714841 17119 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:43.766609 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:43.768765 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:43.716183 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 62061.2us
I20260709 12:57:43.769933 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:43.770049 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:43.782768 17158 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:43.725364 17108 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:57:43.732924 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 124252us
I20260709 12:57:43.806777 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:43.807543 14245 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:43.668992 17104 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.674387 17168 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:43.773723 16934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.474s	user 0.098s	sys 0.160s
I20260709 12:57:43.731336 17173 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:43.760798 16732 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:43.731827 17130 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:43.705628 17131 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.798462 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.828941 13537 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:43.829820 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:43.780584 16978 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:43.778877 16773 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.696702 17167 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.803637 14237 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:43.772018 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 63686.9us
I20260709 12:57:43.835229 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:43.810554 13508 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:43.847630 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:43.790776 17107 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:43.799777 17097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.046s	sys 0.100s
I20260709 12:57:43.795035 17150 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.822329 13543 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:43.797073 16966 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:57:43.803471 16616 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:43.833125 14245 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:43.801105 13684 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:43.913970 16817 consensus_meta.cc:287] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.613s	user 0.011s	sys 0.000s
I20260709 12:57:43.810146 17121 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.805977 17158 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:43.817878 17166 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:43.821372 17138 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.827879 17130 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:43.998780 16920 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:43.840242 16773 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.882653 17168 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.836730 17105 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:57:43.824646 17000 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:57:43.889228 17137 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.890066 17023 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.820825 17170 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:57:43.892014 16978 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:43.850003 17052 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:57:43.894441 17178 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:43.896521 17063 consensus_meta.cc:287] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.597s	user 0.006s	sys 0.008s
I20260709 12:57:43.896180 16732 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.896627 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.830670 17104 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:43.898252 17140 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.824745 17173 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:43.835597 17029 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:57:43.900462 16947 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:57:43.900681 17177 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:57:43.900918 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:57:43.901379 17174 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:57:43.910089 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.912730 14209 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:44.055222 13537 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:44.056715 13537 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:57:44.057417 16947 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:44.057542 16773 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.057897 16947 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:43.905066 13691 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:43.849903 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:44.060495 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 210702us
I20260709 12:57:44.061244 17198 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:43.917948 17133 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.064385 17133 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:43.919284 17160 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:43.920406 14211 connection.cc:256] RPC call timeout handler was delayed by -1.7576490900000863s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 12:57:44.066689 14211 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.65:39209 timed out after 1.586s (SENT) [suppressed 7 similar messages]
I20260709 12:57:43.920519 17155 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.924278 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2913) took 1214 ms. Trace:
I20260709 12:57:44.067703 13688 rpcz_store.cc:276] 0709 12:57:42.709685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:42.709939 (+   254us) service_pool.cc:224] Handling call
0709 12:57:43.923752 (+1213813us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54002977d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:43.923864 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131840384}
I20260709 12:57:43.926508 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:43.930114 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 211565us
W20260709 12:57:43.927791 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 80279.5us
I20260709 12:57:44.070041 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.070168 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:43.844507 17020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.000s	sys 0.017s
W20260709 12:57:43.958329 16958 consensus_meta.cc:287] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.875s	user 0.012s	sys 0.000s
I20260709 12:57:44.072382 17197 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:43.962323 17053 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:44.073629 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 160999us
I20260709 12:57:44.073386 17053 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.074615 13540 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:43.967607 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:43.954994 17163 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:57:43.972512 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:44.075977 14242 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:57:43.923637 16970 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.999564 17130 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:44.000609 17151 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:44.001458 17107 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:44.002744 17166 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.004035 17134 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:44.005924 17100 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.869s	user 0.005s	sys 0.006s
I20260709 12:57:44.125331 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.008805 17157 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.011670 16774 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:43.969205 17050 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.015671 17003 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:44.017089 17158 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:44.141356 17158 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:44.141876 16774 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:44.018375 16590 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:44.018913 13614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 557) took 2754 ms (client timeout 1838 ms). Trace:
I20260709 12:57:43.957589 17150 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.020625 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:44.022956 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:43.810882 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 137776us
I20260709 12:57:44.025578 17167 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.031668 17125 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:44.036120 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 206370us
W20260709 12:57:44.039424 17017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.168s	user 0.015s	sys 0.016s
I20260709 12:57:44.046578 13542 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:44.060562 13691 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:43.915637 17131 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:44.072861 17197 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:44.082222 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.088173 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:44.161231 13617 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:44.079484 14212 connection.cc:256] RPC call timeout handler was delayed by -2.1783028429999831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
I20260709 12:57:44.089722 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.162029 13613 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:44.091610 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:44.091660 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:44.131733 16616 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:44.045473 17178 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:44.094485 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:44.096086 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:44.090622 16744 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:44.053550 17173 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:44.100421 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.107438 16978 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:44.120569 17166 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.122310 16950 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.126801 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.129323 13618 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:44.164860 17197 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.139715 17192 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:44.146096 13614 rpcz_store.cc:269] 0709 12:57:41.264096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.264567 (+   471us) service_pool.cc:224] Handling call
0709 12:57:44.018694 (+2754127us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b540030acd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:44.018842 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6138234112}
I20260709 12:57:44.198813 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:44.199487 13542 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.130296 16947 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.151207 16590 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:44.153614 13534 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:44.205490 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:44.206198 13534 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:44.152886 13541 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:44.167400 16744 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:44.165637 16616 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:44.155589 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:44.159129 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:44.159085 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:44.159646 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 92984.3us
I20260709 12:57:44.163870 14237 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:44.092348 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.166066 14245 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:44.168159 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 72157.2us
I20260709 12:57:44.170265 13619 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:44.145547 17134 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:57:44.161443 17017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.292s	user 0.016s	sys 0.016s
I20260709 12:57:44.097787 13536 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:44.174861 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1429) took 2929 ms. Trace:
W20260709 12:57:44.173718 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46991.1us
I20260709 12:57:44.224776 13535 rpcz_store.cc:276] 0709 12:57:41.245158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.446686 (+201528us) service_pool.cc:224] Handling call
0709 12:57:43.325843 (+1879157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:43.544163 (+218320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:44.174594 (+630431us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54000dc350. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:44.174773 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4320702720}
I20260709 12:57:44.208787 13541 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:44.049085 17063 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:44.169289 17178 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:44.169536 17125 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:44.128830 17100 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:44.224177 13617 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:57:44.226614 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.218063 13616 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:44.220191 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.175453 17163 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:44.216704 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 57713.1us
W20260709 12:57:44.215337 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 59841.4us
I20260709 12:57:44.218257 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:44.219370 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:44.259819 13541 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:44.261998 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:44.261315 14211 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:44.264292 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 126628us
I20260709 12:57:44.264976 13542 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:44.216761 13543 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
W20260709 12:57:44.268174 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 48921.9us
I20260709 12:57:44.267479 13584 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:44.270669 14240 raft_consensus.cc:2410] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:57:44.222680 16744 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:44.274051 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:44.294909 13689 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:44.219990 17131 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:44.285336 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:44.222854 16616 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.222555 17108 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:57:44.301643 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 81767us
W20260709 12:57:44.301576 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 75062.2us
I20260709 12:57:44.315152 13686 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:44.223873 17017 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:44.315116 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:44.283280 17134 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:44.348614 14212 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:44.396796 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 48260.5us
I20260709 12:57:44.282816 16590 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:44.332693 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.320012 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:44.356050 13540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 643) took 3312 ms (client timeout 1586 ms). Trace:
W20260709 12:57:44.384902 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 69855.1us
W20260709 12:57:44.409205 13540 rpcz_store.cc:269] 0709 12:57:41.043397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.044013 (+   616us) service_pool.cc:224] Handling call
0709 12:57:44.355803 (+3311790us) spinlock_profiling.cc:243] Waited 3.03 s on lock 0x7b54000dbe54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:44.355960 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6967673216}
I20260709 12:57:44.410661 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:44.377343 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1437) took 1224 ms. Trace:
I20260709 12:57:44.411837 13538 rpcz_store.cc:276] 0709 12:57:43.153067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:43.198530 (+ 45463us) service_pool.cc:224] Handling call
0709 12:57:44.377113 (+1178583us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54003b7c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:44.377258 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082084224}
I20260709 12:57:44.389745 13541 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:44.413228 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 80624.7us
I20260709 12:57:44.390697 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:44.414143 13543 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:44.400748 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:44.347673 16935 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:44.331382 17020 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:44.229557 16978 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 12:57:44.419596 16590 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:44.471972 16590 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:44.472649 17157 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:44.428006 17029 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:44.308677 17104 consensus_queue.cc:1059] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.165s
I20260709 12:57:44.473964 17029 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.474488 17104 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:44.474956 17104 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:44.476102 17137 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:44.414202 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:44.463141 16935 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.212065 17192 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.477315 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.213148 17199 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.223s
I20260709 12:57:44.477362 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1479) took 1217 ms. Trace:
I20260709 12:57:44.481572 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:44.470937 13535 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:44.415781 14242 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 649) took 3270 ms (client timeout 1960 ms). Trace:
I20260709 12:57:44.470280 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.483284 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:44.414870 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 94967.8us
I20260709 12:57:44.505383 14240 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:44.504834 14242 rpcz_store.cc:269] 0709 12:57:41.113847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:41.113989 (+   142us) service_pool.cc:224] Handling call
0709 12:57:44.356645 (+3242656us) spinlock_profiling.cc:243] Waited 2.9 s on lock 0x7b5400541354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:44.384107 (+ 27462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6677367680}
I20260709 12:57:44.424645 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:44.488293 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 74212.3us
I20260709 12:57:44.486951 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:44.506071 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 81520us
I20260709 12:57:44.471189 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1528) took 1251 ms. Trace:
I20260709 12:57:44.506238 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:44.487730 13508 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:44.312240 17170 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:44.312462 16968 raft_consensus.cc:2764] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:57:44.302920 17003 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:44.374328 16797 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:57:44.306610 17119 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:44.419337 16817 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:44.358611 17101 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:44.351286 17043 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:44.473311 17157 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:44.506202 13618 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:57:44.418856 13536 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:44.502662 13692 rpcz_store.cc:276] 0709 12:57:43.261862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:43.625653 (+363791us) service_pool.cc:224] Handling call
0709 12:57:43.641609 (+ 15956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:43.800472 (+158863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:43.800500 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:43.800544 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:43.800577 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:44.477128 (+676551us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400159d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:44.477188 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:44.477307 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364870016}
I20260709 12:57:44.510732 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:44.364336 16960 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:44.379284 16950 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.354658 17053 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:44.506613 13539 rpcz_store.cc:276] 0709 12:57:43.219620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:43.597046 (+377426us) service_pool.cc:224] Handling call
0709 12:57:43.597412 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:43.790089 (+192677us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:43.800429 (+ 10340us) log.cc:844] Serialized 14 byte log entry
0709 12:57:43.957547 (+157118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:43.957600 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:43.957732 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:44.469900 (+512168us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b180041cf20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6932a6 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419
0709 12:57:44.470956 (+  1056us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54000b7cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:44.471003 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:44.471130 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12312,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":84,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":10248,"spinlock_wait_cycles":451574400,"thread_start_us":156712,"threads_started":1,"wal-append.queue_time_us":248458}
I20260709 12:57:44.522109 17120 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:44.517474 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:44.516039 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 45831.8us
I20260709 12:57:44.522595 13619 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:44.526822 16732 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:44.344254 17067 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.407879 17136 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:44.525286 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.478652 17104 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.416075 17134 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:44.468235 17108 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:44.529167 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:44.331632 17125 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.170s
I20260709 12:57:44.331509 17105 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.186s
I20260709 12:57:44.320219 17155 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.208s
I20260709 12:57:44.318044 17193 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:44.331147 16773 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.188s
I20260709 12:57:44.349712 17111 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.172s
W20260709 12:57:44.558264 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 47609.1us
I20260709 12:57:44.336148 17000 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.185s
I20260709 12:57:44.514276 17137 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:44.501793 17029 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.520407 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.642558 13688 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:44.572029 13534 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:44.526038 17003 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:44.524927 17119 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:44.573639 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 257955us
I20260709 12:57:44.531687 16817 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:44.597321 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:44.592039 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:44.601437 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 96040.4us
I20260709 12:57:44.564603 17194 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:44.536460 17101 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:44.539664 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:44.537487 17166 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:57:44.573863 17120 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:44.598140 17134 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.565685 16990 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:44.527086 16962 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:44.786180 13687 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:57:44.608752 17133 consensus_queue.cc:1175] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 8 index: 9 } }
I20260709 12:57:44.648183 17157 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:44.649545 17000 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:44.650095 17003 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:44.641777 17137 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:44.612054 17193 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:44.664502 17107 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:44.665593 17150 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.667510 17192 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.600824 17108 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:44.538179 17170 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:44.703792 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.704772 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.622928 17104 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.705312 17119 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:44.784626 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.785198 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.647380 17185 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:44.701032 17100 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:44.707011 16817 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:44.788203 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.789434 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.599819 17017 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.792678 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:44.872445 17158 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.000s
I20260709 12:57:44.720813 16774 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:44.721231 16732 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:44.794698 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.890033 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.733155 17120 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:44.821823 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.733294 17053 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 12:57:44.834146 13689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1474) took 1578 ms (client timeout 1598 ms). Trace:
W20260709 12:57:44.895320 13689 rpcz_store.cc:269] 0709 12:57:43.256156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:43.478236 (+222080us) service_pool.cc:224] Handling call
0709 12:57:44.833947 (+1355711us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54000488d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:44.834082 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1602799360}
I20260709 12:57:44.896520 13613 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:44.787467 17063 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:44.838295 17157 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:44.842293 16590 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.800921 16962 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:44.844498 16947 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.844002 16797 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.837255 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:44.791739 17194 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:44.821192 17193 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:44.855083 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:44.859881 13616 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:44.794636 17000 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:44.818874 17137 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.860456 17108 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:44.632529 13536 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:44.877792 17178 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.006s
I20260709 12:57:44.861435 16935 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:44.794729 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.889475 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.936355 13690 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:44.892243 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.937865 13685 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:44.893220 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.898746 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.878270 17185 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:44.902143 13613 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:44.943588 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:44.890587 16616 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
I20260709 12:57:44.904374 13615 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:57:44.914078 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:44.923267 13687 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:44.895234 16732 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:44.940088 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 300562us
I20260709 12:57:44.927165 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.953135 13616 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.895414 16774 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.947566 13613 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:44.953183 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:44.960687 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:44.961938 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:44.962738 13690 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:44.990325 16774 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:44.990209 17194 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.979595 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.979880 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.993023 13613 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:44.993567 13612 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:44.981101 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:44.985869 17210 log.cc:927] Time spent T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 12:57:44.995051 13614 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:44.984637 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:44.996045 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.996546 13614 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:44.915544 17157 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:44.999119 13687 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:44.986378 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:45.001561 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:45.004328 16744 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.000s
I20260709 12:57:44.988376 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:44.964860 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 389042us
I20260709 12:57:44.984165 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:45.009963 14211 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:44.979970 13616 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:45.005492 13685 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:45.014727 13685 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:44.984231 16935 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:45.016872 16935 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:45.017351 16935 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:45.015743 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.019156 13536 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
W20260709 12:57:45.007345 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 501155us
I20260709 12:57:45.006979 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:45.022413 13614 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:44.924477 17170 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.026304 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:44.942451 17178 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.028213 13509 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:45.039358 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:45.021278 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:44.965608 17020 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:45.032148 13616 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:45.045979 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:45.046088 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:44.958047 16616 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.050357 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:45.041733 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:45.033525 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:45.035564 13685 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:57:45.053087 16732 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:45.051695 17150 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:45.056847 13688 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:45.057585 13613 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:45.057823 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:45.061223 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:45.063410 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:45.066519 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:45.067155 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:45.048627 16947 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:45.088174 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.085649 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:45.153860 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 63729.6us
W20260709 12:57:45.149500 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 83048.3us
I20260709 12:57:45.155242 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:45.153867 13542 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:45.158593 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:45.071857 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.160753 13536 raft_consensus.cc:2393] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:45.164789 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.014885 16744 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:45.151022 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 80649.7us
I20260709 12:57:45.057396 17158 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:45.204315 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.208117 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:45.210547 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.068572 14241 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:45.084200 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.056826 17167 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:45.187567 13509 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:45.193351 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.064779 16774 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:44.996203 17000 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:45.218820 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 212005us
I20260709 12:57:45.220350 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:45.235541 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:45.238055 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:45.238600 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.224650 13534 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:45.090459 16966 raft_consensus.cc:2764] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:57:45.005178 16962 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.314108 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.066725 16920 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:45.316278 16920 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:45.316885 16920 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:45.318084 16920 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:57:45.150110 17197 consensus_meta.cc:287] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.938s	user 0.011s	sys 0.000s
I20260709 12:57:45.320359 17197 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.321806 14246 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:45.067777 17215 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:45.225082 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:45.060770 17219 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:45.100488 16732 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:45.068042 16990 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:45.307188 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:45.191010 17131 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:45.223382 17063 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:45.027258 17185 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.218358 17017 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:45.090575 16803 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.238s
I20260709 12:57:45.191380 17050 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:45.230012 17105 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.154s
W20260709 12:57:45.220064 17173 consensus_meta.cc:287] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.045s	user 0.008s	sys 0.004s
I20260709 12:57:45.191658 17067 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:45.352634 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:45.359741 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:45.490566 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:45.325016 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:45.231120 17029 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:45.490064 13542 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:45.505990 14241 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:45.230476 17125 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:45.219183 17167 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:45.540093 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:45.308487 17192 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:45.398435 17131 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:45.395002 17218 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:57:45.349160 17214 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
W20260709 12:57:45.612128 14210 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.022s	sys 0.062s
W20260709 12:57:45.613044 14210 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.022s	sys 0.063s
I20260709 12:57:45.194288 17111 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.209s
W20260709 12:57:45.603461 13690 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.018s	sys 0.027s
I20260709 12:57:45.486788 17150 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:45.392848 17157 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:45.393150 17215 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:45.323167 17197 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:45.386375 16616 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:45.352519 16744 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:45.333392 17219 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:45.368247 16590 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:45.519860 17000 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:45.594036 17023 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:45.594589 17167 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:45.601631 16935 consensus_meta.cc:287] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.001s
I20260709 12:57:45.592226 17067 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:45.600912 17131 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:45.367893 17120 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:45.538691 17029 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.373889 17119 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.360405 17020 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:45.490041 17173 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.591361 17216 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.149s
I20260709 12:57:45.598366 17125 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:45.520401 17063 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.627792 17222 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:57:45.642366 17163 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.356s	user 0.011s	sys 0.000s
I20260709 12:57:45.644119 17023 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:45.644323 17219 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:45.631630 17150 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:45.685992 17178 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:45.702726 17136 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:45.703444 17136 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:45.707242 17104 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.004s
W20260709 12:57:45.708323 17130 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.579s	user 0.009s	sys 0.005s
I20260709 12:57:45.708881 17130 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.681053 17163 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:45.737151 17226 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:57:45.742139 17211 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.139s	user 0.006s	sys 0.018s
W20260709 12:57:45.748039 17225 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.398s	user 0.002s	sys 0.000s
W20260709 12:57:45.750175 17223 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.382s	user 0.001s	sys 0.000s
I20260709 12:57:45.697067 17215 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:45.752290 17200 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.429s	user 0.007s	sys 0.008s
I20260709 12:57:45.753628 17220 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:45.754060 17220 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:45.754829 17220 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:45.758047 17209 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.868s	user 0.000s	sys 0.022s
W20260709 12:57:45.758617 17193 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.818s	user 0.009s	sys 0.000s
I20260709 12:57:45.759169 17220 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:45.763221 17219 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:45.795709 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1531) took 1970 ms. Trace:
W20260709 12:57:45.796326 13509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.006s	sys 0.009s
I20260709 12:57:45.801545 13535 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:57:45.802029 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.001s	sys 0.025s
W20260709 12:57:45.992380 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.002s	sys 0.026s
W20260709 12:57:45.992859 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 756976us
W20260709 12:57:45.804073 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:45.805905 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 573) took 1334 ms. Trace:
W20260709 12:57:45.807420 13659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.003s	sys 0.003s
W20260709 12:57:45.991987 13509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.007s	sys 0.009s
I20260709 12:57:45.808990 13685 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:45.531423 13688 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:45.809983 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2710) took 1107 ms. Trace:
W20260709 12:57:45.810582 13657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.002s	sys 0.015s
I20260709 12:57:45.811425 13689 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
W20260709 12:57:45.622702 13690 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.019s	sys 0.027s
W20260709 12:57:45.812240 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.001s	sys 0.011s
I20260709 12:57:45.815126 13684 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
W20260709 12:57:45.614944 13687 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.022s	sys 0.022s
I20260709 12:57:46.030444 13618 rpcz_store.cc:276] 0709 12:57:44.702580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.702742 (+   162us) service_pool.cc:224] Handling call
0709 12:57:44.706747 (+  4005us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:44.821744 (+114997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:44.821776 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:44.832231 (+ 10455us) log.cc:844] Serialized 26 byte log entry
0709 12:57:45.065357 (+233126us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:57:45.065395 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:45.749605 (+684210us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:45.809236 (+ 59631us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b54002b5554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:45.809336 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:45.809439 (+   103us) raft_consensus.cc:1685] finished
0709 12:57:45.809456 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:45.809800 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25632,"raft.queue_time_us":29914,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":25831,"spinlock_wait_cycles":284151424,"thread_start_us":227514,"threads_started":1,"wal-append.queue_time_us":487743}
W20260709 12:57:46.052672 13687 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.023s	sys 0.022s
I20260709 12:57:46.052551 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:46.053671 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 485) took 1001 ms. Trace:
I20260709 12:57:45.996896 13611 rpcz_store.cc:276] 0709 12:57:44.470991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.498589 (+ 27598us) service_pool.cc:224] Handling call
0709 12:57:44.524897 (+ 26308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:44.532326 (+  7429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:44.532368 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:44.532912 (+   544us) log.cc:844] Serialized 52 byte log entry
0709 12:57:44.933293 (+400381us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:44.933340 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:44.933419 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:45.805222 (+871803us) raft_consensus.cc:1685] finished
0709 12:57:45.805284 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:45.805607 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7017,"raft.queue_time_us":79871,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6698,"thread_start_us":339902,"threads_started":1,"wal-append.queue_time_us":501826}
I20260709 12:57:45.815412 13692 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
W20260709 12:57:46.054556 17170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.000s	sys 0.015s
W20260709 12:57:46.055050 17170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.000s	sys 0.015s
I20260709 12:57:46.054805 17136 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:45.815858 13693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.000s	sys 0.013s
I20260709 12:57:46.056452 17136 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:46.057833 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1486) took 1358 ms. Trace:
W20260709 12:57:45.817607 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.040s	sys 0.084s
W20260709 12:57:46.059619 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.040s	sys 0.084s
W20260709 12:57:46.062222 17158 consensus_meta.cc:287] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.838s	user 0.007s	sys 0.011s
I20260709 12:57:46.058496 13693 rpcz_store.cc:276] 0709 12:57:44.699683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.699800 (+   117us) service_pool.cc:224] Handling call
0709 12:57:44.700115 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:44.732713 (+ 32598us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:45.060667 (+327954us) log.cc:844] Serialized 14 byte log entry
0709 12:57:46.056491 (+995824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:46.056603 (+   112us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:46.056807 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:46.057329 (+   522us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:46.057613 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5406,"prepare.queue_time_us":839693,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":21267,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":29844,"thread_start_us":987192,"threads_started":2,"wal-append.queue_time_us":529463}
I20260709 12:57:46.062886 17158 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:57:45.817626 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.003s	sys 0.019s
W20260709 12:57:46.064189 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.003s	sys 0.019s
I20260709 12:57:46.064222 13693 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:46.065011 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 486) took 1012 ms. Trace:
I20260709 12:57:46.065461 13693 rpcz_store.cc:276] 0709 12:57:45.052869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.063380 (+1010511us) service_pool.cc:224] Handling call
0709 12:57:46.063829 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:46.064776 (+   947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:46.064957 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:46.066640 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:46.067467 13693 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:57:46.069967 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:46.070768 13543 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:46.071192 17245 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:45.821133 14241 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 997) took 1729 ms (client timeout 2039 ms). Trace:
I20260709 12:57:45.826220 17017 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:46.072086 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2916) took 2419 ms. Trace:
I20260709 12:57:46.072570 13686 rpcz_store.cc:276] 0709 12:57:43.606898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:43.637065 (+ 30167us) service_pool.cc:224] Handling call
0709 12:57:43.672282 (+ 35217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:44.315613 (+643331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:45.956692 (+1641079us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400244854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:46.026180 (+ 69488us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478028672}
I20260709 12:57:46.073254 17017 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.074335 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:46.074671 13686 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:57:46.075132 13613 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:45.826826 17134 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.038s	user 0.011s	sys 0.003s
I20260709 12:57:46.076207 17134 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.078898 17134 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:45.830776 17029 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:45.839581 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 999) took 1695 ms. Trace:
I20260709 12:57:45.896797 17130 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:45.906425 17222 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:46.079933 14246 rpcz_store.cc:276] 0709 12:57:44.143948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.144160 (+   212us) service_pool.cc:224] Handling call
0709 12:57:44.144541 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:45.322366 (+1177825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:45.838816 (+516450us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400601654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:45.839291 (+   475us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2706780416}
I20260709 12:57:46.081497 17222 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:46.072134 14241 rpcz_store.cc:269] 0709 12:57:44.091105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.091271 (+   166us) service_pool.cc:224] Handling call
0709 12:57:45.820393 (+1729122us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54005f0354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:45.820798 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2082381184}
I20260709 12:57:45.908353 16950 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:45.909353 16732 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.008s
I20260709 12:57:45.912685 17067 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.084252 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 480) took 1077 ms. Trace:
I20260709 12:57:46.084653 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:46.084411 16950 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:46.085808 13611 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:45.759058 17193 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.088538 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:46.089569 13618 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:46.090237 17193 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:45.915462 17137 consensus_meta.cc:287] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.973s	user 0.007s	sys 0.007s
I20260709 12:57:45.929623 17173 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:45.930469 17235 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:46.091150 17137 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.091969 17235 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.094107 17137 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.094316 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:45.932029 17163 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:46.095481 13615 raft_consensus.cc:2368] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
I20260709 12:57:46.095383 17163 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:46.096978 17163 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.098546 17163 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:45.933473 17023 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.101090 17023 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:45.934458 17125 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:45.942893 17185 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:46.104791 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.105758 14240 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:45.656322 16935 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:45.951234 17100 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.013s	sys 0.000s
I20260709 12:57:46.054073 13684 rpcz_store.cc:276] 0709 12:57:45.052564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.052391 (+999827us) service_pool.cc:224] Handling call
0709 12:57:46.052842 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:46.053237 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:46.053273 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:46.053328 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:46.053393 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:46.053451 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:46.053589 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:45.952092 17063 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:45.970134 17150 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:45.971623 16947 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:45.972102 17119 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:45.972558 16774 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:46.047549 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 860018us
I20260709 12:57:45.991043 17219 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:45.808444 13614 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.018s	sys 0.016s
W20260709 12:57:46.009789 13659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.003s	sys 0.003s
W20260709 12:57:46.045130 13657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.003s	sys 0.015s
W20260709 12:57:46.046208 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.002s	sys 0.011s
I20260709 12:57:46.049558 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:46.051155 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:46.132597 13541 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:46.134481 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:46.051983 17194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.024s	sys 0.043s
I20260709 12:57:46.084430 16732 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:57:46.136240 17194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.024s	sys 0.044s
I20260709 12:57:46.109386 13543 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:46.108355 13688 rpcz_store.cc:276] 0709 12:57:45.007052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:45.007632 (+   580us) service_pool.cc:224] Handling call
0709 12:57:46.084041 (+1076409us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:46.115234 17100 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.014s	sys 0.000s
I20260709 12:57:45.966393 13537 rpcz_store.cc:276] 0709 12:57:43.825397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.057474 (+232077us) service_pool.cc:224] Handling call
0709 12:57:44.057808 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:44.058169 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:44.058199 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:44.058727 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 12:57:44.169399 (+110672us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:44.169446 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:44.169522 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:45.751673 (+1582151us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:45.794395 (+ 42722us) raft_consensus.cc:1685] finished
0709 12:57:45.794501 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:45.795283 (+   782us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":32949632,"thread_start_us":109904,"threads_started":1,"wal-append.queue_time_us":154677}
I20260709 12:57:46.140051 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:46.087827 16950 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:46.144134 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.144510 17217 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.138s
I20260709 12:57:46.144423 17131 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.146837 14212 leader_election.cc:400] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
I20260709 12:57:46.147240 14212 leader_election.cc:403] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:57:46.140652 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:46.094851 17235 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.147503 17131 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.152693 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.160516 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:46.161130 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:46.162920 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:46.172962 13614 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.019s	sys 0.017s
I20260709 12:57:46.179445 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.180071 17197 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:46.180675 17197 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:46.181039 17197 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:46.181253 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.184376 17053 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:46.184832 17053 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:46.185135 17053 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:46.188334 17238 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
W20260709 12:57:46.189306 17167 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 12:57:46.189852 17167 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:46.170825 13584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.010s	sys 0.027s
W20260709 12:57:46.150097 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 926606us
I20260709 12:57:46.193457 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:46.197268 13611 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:46.201660 13613 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
W20260709 12:57:46.202129 13584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.010s	sys 0.027s
W20260709 12:57:46.210290 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 47417.6us
I20260709 12:57:46.211109 13618 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:57:46.217934 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.164237 17178 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:46.190037 17101 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.228333 17101 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:46.228840 17050 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.709s	user 0.015s	sys 0.004s
I20260709 12:57:46.229427 17050 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:57:46.232124 17043 consensus_meta.cc:287] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.628s	user 0.006s	sys 0.017s
I20260709 12:57:46.232231 17053 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.233364 17052 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:46.233777 17052 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:46.233877 14239 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:46.234175 17052 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:46.236167 17229 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:46.236485 13584 connection.cc:256] RPC call timeout handler was delayed by -2.5535501319991454s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
I20260709 12:57:46.215865 16774 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.237067 17229 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.128348 17245 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.239579 16774 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:46.236331 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:46.240309 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:46.241107 17063 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:46.241987 17043 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:46.242177 17063 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.243000 17043 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:46.241065 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:46.227700 14237 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:46.241180 17245 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.243770 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:46.243748 13535 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
W20260709 12:57:46.248309 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:46.240846 14212 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:46.226562 17003 consensus_meta.cc:287] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.272s	user 0.015s	sys 0.002s
I20260709 12:57:46.272609 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:46.275821 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1490) took 1574 ms. Trace:
I20260709 12:57:46.276342 13690 rpcz_store.cc:276] 0709 12:57:44.701485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.963532 (+262047us) service_pool.cc:224] Handling call
0709 12:57:44.997557 (+ 34025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:44.997761 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:57:46.046320 (+1048559us) log.cc:844] Serialized 14 byte log entry
0709 12:57:46.274814 (+228494us) log.cc:844] Serialized 14 byte log entry
0709 12:57:46.275084 (+   270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:46.275201 (+   117us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:46.275312 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:46.275421 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:46.275632 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":666145,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":2731,"raft.queue_time_us":4536,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":25068,"thread_start_us":1086977,"threads_started":2}
I20260709 12:57:46.279309 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:46.280236 13542 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:46.280510 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:46.281160 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:46.282088 13615 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:46.282967 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:46.284338 13684 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:57:46.284492 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:46.286899 17017 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:46.287711 17017 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:46.288160 17017 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:46.288658 17108 consensus_meta.cc:287] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 1.343s	user 0.014s	sys 0.004s
I20260709 12:57:46.288400 17043 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.290388 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:46.288580 17017 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:46.248212 17219 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:46.289812 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1496) took 1586 ms. Trace:
W20260709 12:57:46.294169 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53482.7us
I20260709 12:57:46.306922 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.293677 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.309818 17219 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:46.310467 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 84060.9us
I20260709 12:57:46.312299 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.294180 13687 rpcz_store.cc:276] 0709 12:57:44.703365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:45.005602 (+302237us) service_pool.cc:224] Handling call
0709 12:57:45.005893 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:45.006081 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:46.053366 (+1047285us) log.cc:844] Serialized 14 byte log entry
0709 12:57:46.165821 (+112455us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:46.195456 (+ 29635us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:46.209811 (+ 14355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:46.226980 (+ 17169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:46.289754 (+ 62774us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21089,"prepare.queue_time_us":681804,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":113,"raft.queue_time_us":25963,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":130,"thread_start_us":1075343,"threads_started":2,"wal-append.queue_time_us":172368}
W20260709 12:57:46.318248 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61910.2us
I20260709 12:57:46.319993 17003 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.258809 17178 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.321473 13617 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:46.322171 17003 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.324235 17003 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:46.324324 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:46.325089 14242 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:46.324877 17003 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.327977 16935 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:46.296197 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:46.320614 14241 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:46.361928 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.207715 16947 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.355365 13542 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
W20260709 12:57:46.364588 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52554.4us
I20260709 12:57:46.289269 17108 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.312340 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:46.369911 14240 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:46.372952 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:46.376840 17120 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.380506 13693 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:57:46.320016 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:46.380312 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:46.404578 13543 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:46.404865 17050 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:46.404732 17067 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:46.405782 17067 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:46.405587 17050 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:46.405774 13538 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:46.406464 17067 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.407466 13538 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:46.407721 17131 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:57:46.408327 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 96120.7us
I20260709 12:57:46.413833 14242 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:57:46.408166 17131 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:46.293633 17053 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.474203 13610 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:46.420400 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:46.964224 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 672136us
W20260709 12:57:46.966043 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:46.968200 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:46.714502 13542 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:57:46.971202 13535 consensus_meta.cc:287] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.696s	user 0.013s	sys 0.017s
I20260709 12:57:46.419085 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:46.971547 13535 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:46.761046 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:46.418354 13538 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:46.972901 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:46.324357 17245 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:57:46.982388 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:46.716321 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 336080us
W20260709 12:57:46.954021 17017 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.000s
I20260709 12:57:46.988212 17017 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:46.989653 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 585399us
I20260709 12:57:46.990981 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:46.486135 17197 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.992493 17017 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:46.805085 17136 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:46.988595 17131 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:46.951753 17052 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.717s	user 0.007s	sys 0.003s
I20260709 12:57:46.997740 17052 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.998934 14245 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:57:46.837610 17222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.017s	sys 0.056s
W20260709 12:57:46.999719 17222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.018s	sys 0.062s
I20260709 12:57:46.987242 14240 raft_consensus.cc:2393] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:47.002022 17217 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:46.995829 14243 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:46.996279 17067 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.002759 17217 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.003283 17067 raft_consensus.cc:2764] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:57:46.997215 17131 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:46.418213 17173 raft_consensus.cc:2764] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:57:47.004763 17173 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:47.005151 17173 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:47.005555 17173 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:47.000965 17157 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:46.996990 17197 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:47.001082 17023 consensus_meta.cc:287] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.899s	user 0.009s	sys 0.001s
I20260709 12:57:47.008256 17219 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.008306 17157 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:47.009603 17003 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.009398 17136 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.010203 13543 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:47.009936 17157 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:46.969127 16947 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.011729 14244 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:47.001477 17052 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:47.008975 17101 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.002s
I20260709 12:57:46.483040 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 477) took 1590 ms. Trace:
I20260709 12:57:47.003137 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.013705 14237 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:47.014075 14240 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:47.013463 13691 rpcz_store.cc:276] 0709 12:57:44.892553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:45.813306 (+920753us) service_pool.cc:224] Handling call
0709 12:57:45.814190 (+   884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:45.936247 (+122057us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:45.936326 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:45.937091 (+   765us) log.cc:844] Serialized 26 byte log entry
0709 12:57:46.043357 (+106266us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:46.043414 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:46.045772 (+  2358us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:46.482629 (+436857us) raft_consensus.cc:1685] finished
0709 12:57:46.482654 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:46.482826 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121108,"raft.queue_time_us":126285,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":235,"spinlock_wait_cycles":14592,"thread_start_us":105685,"threads_started":1,"wal-append.queue_time_us":543627}
I20260709 12:57:47.016700 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.014549 17172 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:47.018105 17172 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:47.017192 17217 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.018810 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:47.019598 14246 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:47.019994 14246 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:47.019629 17172 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.021212 17172 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:47.020943 14246 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.022099 14246 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:46.418213 16817 consensus_meta.cc:287] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.528s	user 0.019s	sys 0.004s
W20260709 12:57:46.483057 16990 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.992s	user 0.007s	sys 0.007s
I20260709 12:57:47.026692 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.028244 13615 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
W20260709 12:57:46.476914 17000 consensus_meta.cc:287] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.829s	user 0.007s	sys 0.006s
W20260709 12:57:46.705158 13617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2712) took 2001 ms (client timeout 2191 ms). Trace:
I20260709 12:57:47.011600 17136 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:47.029675 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 709676us
I20260709 12:57:46.476120 17167 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.031040 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2730) took 2071 ms. Trace:
I20260709 12:57:46.484546 17120 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:46.973968 17178 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:47.030764 13617 rpcz_store.cc:269] 0709 12:57:44.703335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.703468 (+   133us) service_pool.cc:224] Handling call
0709 12:57:46.704756 (+2001288us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540012a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:46.705047 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3361331456}
I20260709 12:57:46.623389 16774 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:47.035668 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:47.037767 17131 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.042469 17199 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.043237 16947 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.043020 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.032811 17245 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:47.052758 13611 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:47.050691 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:47.056303 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:47.057722 17101 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:47.059664 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:47.064349 13540 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:47.064947 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.065611 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:47.065897 13617 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:47.034931 13614 rpcz_store.cc:276] 0709 12:57:44.959364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:45.026894 (+ 67530us) service_pool.cc:224] Handling call
0709 12:57:45.053959 (+ 27065us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:45.054411 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:45.054450 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:45.055061 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:57:46.243029 (+1187968us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:46.693141 (+450112us) log.cc:844] Serialized 14 byte log entry
0709 12:57:46.729463 (+ 36322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:46.986804 (+257341us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:46.986861 (+    57us) raft_consensus.cc:1685] finished
0709 12:57:46.986871 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:47.030985 (+ 44114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4880,"raft.queue_time_us":48858,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":102399,"thread_start_us":1528004,"threads_started":2,"wal-append.queue_time_us":628625}
I20260709 12:57:47.067358 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.047888 17178 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:47.078261 17178 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:47.079027 17178 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:47.080124 14240 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:47.041678 16990 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:47.045351 17108 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.044888 17050 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.084210 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.039529 17131 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.039662 16817 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.098290 17173 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.100514 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.102185 14238 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:47.073261 17222 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:47.087520 13542 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 6.
I20260709 12:57:47.106894 14246 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
W20260709 12:57:47.106904 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61679.1us
I20260709 12:57:47.043013 17199 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.109440 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.110653 13611 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:47.120704 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:47.127714 13614 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:47.100088 16947 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.131532 13611 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:47.115842 17222 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:47.135807 13618 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.040s	sys 0.047s
I20260709 12:57:47.125995 17053 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:47.124073 17050 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:47.137774 13618 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.040s	sys 0.047s
I20260709 12:57:47.142288 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.144739 13614 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:47.144775 13616 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:57:47.145951 13616 pending_rounds.cc:85] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 5
I20260709 12:57:47.140904 17053 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:47.146251 13616 pending_rounds.cc:107] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:57:47.168593 13616 raft_consensus.cc:2889] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:47.145288 16817 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.141258 17167 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:47.143266 17173 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:47.142051 17050 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.186666 17053 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:47.191012 17108 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:47.207952 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:47.211702 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:47.214213 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.143410 17245 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.201745 17167 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:47.215862 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.232515 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.213646 17108 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.230113 17131 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.244299 13611 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:47.248071 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.231086 17173 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:47.253118 13617 raft_consensus.cc:2368] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:57:47.218189 17222 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.309595 13617 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 12:57:47.314514 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 346412us
W20260709 12:57:47.326121 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 1.02991e+06us
I20260709 12:57:47.310931 17131 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:47.330909 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:47.331471 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.299739 17173 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.347539 13611 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
W20260709 12:57:47.348826 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 203152us
W20260709 12:57:47.350863 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 157012us
I20260709 12:57:47.365658 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 586) took 1143 ms. Trace:
I20260709 12:57:47.365453 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:47.368806 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 377934us
I20260709 12:57:47.342429 17101 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:47.320057 17167 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:47.380715 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.336853 17131 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.371418 17101 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:47.389842 17108 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.390192 17101 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:47.378846 16947 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.433796 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 671) took 1106 ms. Trace:
I20260709 12:57:47.434773 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 667) took 1216 ms. Trace:
W20260709 12:57:47.435709 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 53743us
I20260709 12:57:47.381652 17050 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:47.438424 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.440909 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:47.441695 13613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 585) took 1238 ms (client timeout 1565 ms). Trace:
W20260709 12:57:47.442029 13613 rpcz_store.cc:269] 0709 12:57:46.203648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:47.440623 (+1236975us) service_pool.cc:224] Handling call
0709 12:57:47.441618 (+   995us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:47.443014 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:47.448086 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.450049 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:47.451529 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59313.4us
I20260709 12:57:47.452242 17271 raft_consensus.cc:2764] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:57:47.452430 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:47.452715 13534 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:57:47.453521 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:47.453701 17108 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.454340 17108 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.391996 17157 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.455251 17274 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:47.455832 17274 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:47.456262 17165 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:47.456468 17274 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:47.456761 17165 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:47.457253 17165 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:47.455833 17157 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.460127 13611 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:47.461300 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.462077 13611 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:47.463579 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 670) took 1136 ms. Trace:
I20260709 12:57:47.463747 17271 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:47.464363 14244 rpcz_store.cc:276] 0709 12:57:46.326701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.326944 (+   243us) service_pool.cc:224] Handling call
0709 12:57:47.011415 (+684471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:47.012200 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:47.463397 (+451197us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b54007102d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:47.463521 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38868480}
I20260709 12:57:47.464880 17271 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:47.434342 14243 rpcz_store.cc:276] 0709 12:57:46.327066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.327281 (+   215us) service_pool.cc:224] Handling call
0709 12:57:46.987756 (+660475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:47.002713 (+ 14957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:47.403426 (+400713us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b540005fd54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:47.433725 (+ 30299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11692288}
I20260709 12:57:47.461372 17157 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:47.436208 14245 rpcz_store.cc:276] 0709 12:57:46.224203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.245029 (+ 20826us) service_pool.cc:224] Handling call
0709 12:57:46.257831 (+ 12802us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:46.999280 (+741449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:47.402973 (+403693us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b5400631354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:47.434717 (+ 31744us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703757568}
W20260709 12:57:47.466550 13685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 483) took 2414 ms (client timeout 3145 ms). Trace:
W20260709 12:57:47.470840 13689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1502) took 2637 ms (client timeout 2236 ms). Trace:
W20260709 12:57:47.471335 13685 rpcz_store.cc:269] 0709 12:57:45.051888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:45.539732 (+487844us) service_pool.cc:224] Handling call
0709 12:57:47.408513 (+1868781us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5400122fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:47.466491 (+ 57978us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614671104}
I20260709 12:57:47.473587 13438 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 6, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 6 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:57:47.462741 13692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2930) took 2896 ms (client timeout 2245 ms). Trace:
I20260709 12:57:47.476346 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.474812 17108 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.477823 17050 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:47.478232 14239 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 665) took 1374 ms. Trace:
I20260709 12:57:47.482707 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:47.477933 17277 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
W20260709 12:57:47.472160 13689 rpcz_store.cc:269] 0709 12:57:44.834613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.957178 (+122565us) service_pool.cc:224] Handling call
0709 12:57:47.405256 (+2448078us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54002ed6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:47.470778 (+ 65522us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860312320}
W20260709 12:57:47.476507 13692 rpcz_store.cc:269] 0709 12:57:44.566404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.566606 (+   202us) service_pool.cc:224] Handling call
0709 12:57:47.462507 (+2895901us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003a0a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:47.462655 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563184896}
I20260709 12:57:47.470489 17271 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:47.480605 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:47.483433 13534 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:47.482851 14239 rpcz_store.cc:276] 0709 12:57:46.103759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.151149 (+ 47390us) service_pool.cc:224] Handling call
0709 12:57:46.223540 (+ 72391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:46.234268 (+ 10728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:47.477997 (+1243729us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b54006536d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:47.478169 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23052672}
I20260709 12:57:47.489125 13543 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:47.494308 13611 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:57:47.484566 17050 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.502655 17050 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.503988 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:47.505244 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.506093 14246 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:47.506736 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:47.507117 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:47.507756 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:47.508337 14238 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEARNER]: Advancing to term 9
I20260709 12:57:47.508576 14240 raft_consensus.cc:2376] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9. Re-sending same reply.
I20260709 12:57:47.509704 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.511021 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.512105 17023 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:57:47.507743 14246 raft_consensus.cc:2376] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11. Re-sending same reply.
I20260709 12:57:47.511679 14240 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:47.498835 14211 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:47.521836 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.523059 13613 raft_consensus.cc:2368] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:57:47.526505 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.527274 13611 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:47.496882 17067 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:47.529022 17067 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:47.530280 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.531030 17043 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:57:47.532251 17278 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:57:47.532410 17043 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:47.532919 17043 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:47.532061 17273 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.128s	user 0.000s	sys 0.005s
I20260709 12:57:47.537225 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1628) took 1296 ms. Trace:
I20260709 12:57:47.538825 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.541239 13508 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:47.512392 17120 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:57:47.539885 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.543046 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.546581 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.547320 13611 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:47.531404 17275 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.521078 17023 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:47.494125 16947 consensus_queue.cc:1059] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:57:47.549675 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.527349 16774 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.528056 17050 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:47.550812 13509 connection.cc:256] RPC call timeout handler was delayed by -1.1585855989997071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260709 12:57:47.531160 17067 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.553251 13534 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:47.536819 17131 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.565280 13658 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:47.554960 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.564618 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:47.605413 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.696501 14240 raft_consensus.cc:2393] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:47.696803 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 160755us
I20260709 12:57:47.583304 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.697996 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:47.759660 17272 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.107s	user 0.001s	sys 0.003s
I20260709 12:57:47.760416 16817 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:47.763269 17288 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:57:47.765539 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 211710us
W20260709 12:57:47.765796 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 237048us
I20260709 12:57:47.768743 13611 raft_consensus.cc:2393] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:47.764536 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 224612us
I20260709 12:57:47.772743 13543 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:57:47.554469 17173 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.560411 17178 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:47.775075 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 77198us
W20260709 12:57:47.788450 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:47.769281 17172 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.748s	user 0.011s	sys 0.004s
I20260709 12:57:47.794857 14238 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:47.778672 17290 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:47.804318 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.806131 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.806957 13611 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:47.784075 17050 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.812305 16978 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:47.813113 16978 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.791307 17271 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.818020 17271 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:47.820490 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.822794 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.828198 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:47.833483 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.837191 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:47.811609 13659 connection.cc:256] RPC call timeout handler was delayed by -1.556159600000683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
W20260709 12:57:47.837898 13659 connection.cc:256] RPC call timeout handler was delayed by -1.4848244229997363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
I20260709 12:57:47.838361 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 1002) took 1552 ms. Trace:
I20260709 12:57:47.796216 17290 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:47.838698 14237 rpcz_store.cc:276] 0709 12:57:46.285969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.290363 (+  4394us) service_pool.cc:224] Handling call
0709 12:57:47.838176 (+1547813us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b54005cf3d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:47.838300 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1497272704}
I20260709 12:57:47.839731 17290 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:47.843107 17050 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.854501 17101 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.788159 17199 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.858366 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.859542 17276 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:47.860069 16616 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.860589 16616 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.860925 17276 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:47.861461 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:47.862006 17276 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:47.862396 17276 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:47.862819 16616 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:47.822651 17173 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:47.876693 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:47.877808 13614 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:47.864074 16990 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:47.863799 17155 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:47.878252 16990 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:47.878430 17155 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:47.863312 17276 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.669875 16774 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.878840 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.879228 16990 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.568929 17275 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.880563 16990 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:47.880751 16774 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.774047 17023 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:47.865499 17290 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.881529 17023 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:47.882871 17290 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.859903 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.884737 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.885804 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:47.887153 13617 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:57:47.869527 14240 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:57:47.870157 17270 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.548s	user 0.004s	sys 0.011s
W20260709 12:57:47.871469 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:47.876497 16978 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.891302 13617 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:47.892726 13617 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:57:47.877272 17173 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.918045 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.920208 17173 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:47.920873 13617 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:47.881812 17275 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.867601 17178 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:47.841100 17053 consensus_meta.cc:287] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.002s
I20260709 12:57:47.866418 17063 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:47.920848 16616 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:47.879642 13614 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:47.923496 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 86820.4us
I20260709 12:57:47.887621 17276 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:47.878855 17155 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:47.923158 17157 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:47.883860 17023 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:47.953612 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:47.973002 14244 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:47.953740 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.944473 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:47.887501 17165 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:47.970206 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:47.984189 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:47.997740 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 76900.8us
W20260709 12:57:48.003720 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 304309us
I20260709 12:57:48.026736 17288 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:47.942966 17053 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:48.090900 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 299078us
W20260709 12:57:48.119768 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.176370 14244 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:48.137303 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.167371 17178 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.168252 16616 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:48.170111 17155 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.171324 14240 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:48.203383 14238 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
W20260709 12:57:48.171587 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.013111 17289 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:57:47.944500 17067 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 12:57:48.179891 13659 connection.cc:256] RPC call timeout handler was delayed by -1.4146244230005323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 12:57:48.180091 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 237620us
I20260709 12:57:48.180537 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:48.202284 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 256654us
I20260709 12:57:48.180500 17290 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:57:48.183820 17053 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.208088 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.211007 13614 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:48.189647 17274 consensus_meta.cc:287] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.732s	user 0.016s	sys 0.001s
I20260709 12:57:48.210961 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:48.213747 13611 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:48.182194 17165 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:48.216349 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:48.218372 13659 connection.cc:256] RPC call timeout handler was delayed by -1.1262964759998795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
I20260709 12:57:48.224696 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.225385 13613 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
W20260709 12:57:48.226857 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46440.7us
I20260709 12:57:48.228132 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:48.206182 17222 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.230726 17222 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.231678 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.232892 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.233592 13614 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:48.234306 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.176047 17157 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:48.236948 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.234370 17229 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:47.541062 13615 rpcz_store.cc:276] 0709 12:57:46.238506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:47.076590 (+838084us) service_pool.cc:224] Handling call
0709 12:57:47.116629 (+ 40039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:47.138283 (+ 21654us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:47.372284 (+234001us) log.cc:844] Serialized 14 byte log entry
0709 12:57:47.394138 (+ 21854us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b1800448c60. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:47.526015 (+131877us) log.cc:844] Serialized 14 byte log entry
0709 12:57:47.534330 (+  8315us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:47.534392 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:47.534453 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:47.534534 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:47.534667 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40799,"prepare.queue_time_us":88040,"prepare.run_cpu_time_us":14806,"prepare.run_wall_time_us":41608,"raft.queue_time_us":9338,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":5347,"spinlock_wait_cycles":16602240,"thread_start_us":191829,"threads_started":2,"wal-append.queue_time_us":58506}
I20260709 12:57:48.241581 16616 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.242651 17157 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:48.243669 17101 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.216117 17274 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.205458 17178 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.250236 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.251140 13538 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:48.251297 17274 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:48.251572 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 681) took 1192 ms. Trace:
I20260709 12:57:48.252066 17178 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:48.252265 17173 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.250229 17140 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:48.252796 17173 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.253368 17140 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:48.253724 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.253978 17140 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.255239 17173 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.245344 17289 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:48.256779 17289 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:48.257103 13508 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:48.257685 17289 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.259227 17296 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:48.259613 17296 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:48.241619 17155 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:48.259966 17296 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:48.260378 17288 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.252012 13538 rpcz_store.cc:276] 0709 12:57:47.058910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:48.249931 (+1191021us) service_pool.cc:224] Handling call
0709 12:57:48.251511 (+  1580us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:48.270280 17043 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.722s	user 0.013s	sys 0.003s
I20260709 12:57:48.268879 17140 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.272743 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:48.274245 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:48.277596 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 65363.8us
W20260709 12:57:48.280707 14241 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 659) took 2269 ms (client timeout 1918 ms). Trace:
W20260709 12:57:48.281129 14241 rpcz_store.cc:269] 0709 12:57:46.101308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.117389 (+ 16081us) service_pool.cc:224] Handling call
0709 12:57:48.276308 (+2158919us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001c57d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:48.276408 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325349760}
I20260709 12:57:48.276012 17140 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.289510 14212 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:48.270730 17043 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.251981 17274 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.294298 17274 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:48.297446 17286 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.805s	user 0.002s	sys 0.003s
W20260709 12:57:48.302357 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.276580 17229 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:48.284363 17290 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:48.303731 17290 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:48.304150 17290 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:48.292482 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51822.7us
I20260709 12:57:48.293795 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2734) took 3291 ms. Trace:
I20260709 12:57:48.308132 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2718) took 3497 ms. Trace:
I20260709 12:57:48.309075 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:48.307399 13616 rpcz_store.cc:276] 0709 12:57:45.002617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:45.051109 (+ 48492us) service_pool.cc:224] Handling call
0709 12:57:45.051425 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:47.193413 (+2141988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:48.282019 (+1088606us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b540030a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:48.293707 (+ 11688us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4773257600}
I20260709 12:57:48.311498 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:48.309852 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:48.281098 17275 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:48.308495 13610 rpcz_store.cc:276] 0709 12:57:44.795875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.897740 (+101865us) service_pool.cc:224] Handling call
0709 12:57:44.905593 (+  7853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:47.029565 (+2123972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:48.249335 (+1219770us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001298d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:48.292762 (+ 43427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3536388352}
I20260709 12:57:48.322626 17274 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.316579 17101 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:57:48.324020 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.324651 13538 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:48.326242 13619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2711) took 3623 ms (client timeout 1970 ms). Trace:
I20260709 12:57:48.327023 13612 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:48.327382 13612 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:48.328555 13537 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:57:48.328958 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.329401 17131 raft_consensus.cc:3055] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:48.329772 17131 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:48.330317 17157 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:57:48.326659 13619 rpcz_store.cc:269] 0709 12:57:44.702985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:44.704192 (+  1207us) service_pool.cc:224] Handling call
0709 12:57:48.325798 (+3621606us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001469d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:48.325970 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2684376960}
I20260709 12:57:48.330667 17131 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.331024 17157 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:48.326454 17194 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:57:48.332545 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:48.333343 13539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 679) took 1306 ms (client timeout 1589 ms). Trace:
W20260709 12:57:48.333693 13539 rpcz_store.cc:269] 0709 12:57:47.026813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:48.332177 (+1305364us) service_pool.cc:224] Handling call
0709 12:57:48.333285 (+  1108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:47.366278 13618 rpcz_store.cc:276] 0709 12:57:46.222105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:46.247384 (+ 25279us) service_pool.cc:224] Handling call
0709 12:57:46.293917 (+ 46533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:46.444092 (+150175us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:47.142368 (+698276us) log.cc:844] Serialized 14 byte log entry
0709 12:57:47.357529 (+215161us) log.cc:844] Serialized 14 byte log entry
0709 12:57:47.364846 (+  7317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:47.364906 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:47.365382 (+   476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:47.365462 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:47.365597 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6719,"prepare.queue_time_us":722371,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":15544,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":6446,"thread_start_us":826539,"threads_started":2}
I20260709 12:57:48.317560 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:48.336441 14246 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:57:48.348413 13584 connection.cc:256] RPC call timeout handler was delayed by -1.4164363070003674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 12:57:48.349001 13584 connection.cc:256] RPC call timeout handler was delayed by -1.4047363070003485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 12:57:48.349316 13584 connection.cc:256] RPC call timeout handler was delayed by -1.2796363070010557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 12:57:48.349668 13584 connection.cc:256] RPC call timeout handler was delayed by -1.0636363070007064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 12:57:48.350032 13584 connection.cc:256] RPC call timeout handler was delayed by -1.0141363070006264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
W20260709 12:57:48.350379 13584 connection.cc:256] RPC call timeout handler was delayed by -1.011436307000622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
I20260709 12:57:48.352305 13618 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:48.354128 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:48.352344 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.355149 13539 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:57:48.292249 17178 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:57:48.334489 17101 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:48.371634 17173 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:57:48.372869 17173 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:48.372557 17101 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.374238 17285 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:48.375180 17285 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:48.375494 17285 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:48.376071 17295 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:48.376626 17295 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:48.376346 17285 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.377415 17274 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:48.377800 17285 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.342191 16774 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:48.378571 16774 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:48.378942 16774 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.340525 16990 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.379582 16990 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.380299 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.380456 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:48.353224 17000 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.381330 17000 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.381886 16990 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:48.364939 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 77878.4us
I20260709 12:57:48.369592 17140 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.382907 17140 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.357872 17131 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.384408 17136 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:48.384972 17136 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:48.385133 17140 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.385938 17136 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.387735 17173 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:48.388161 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.388892 13541 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:48.389084 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.371451 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:48.377851 17274 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:48.388200 17173 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:48.357740 16978 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:57:48.394306 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.394845 16978 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:48.395047 13690 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:48.395371 16978 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:48.396591 17199 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:48.396325 16978 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:48.397006 17199 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:48.397445 17199 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.397567 16978 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:48.372632 17271 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.005s
I20260709 12:57:48.376998 17295 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.400324 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:48.401906 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.403075 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:48.368513 17043 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:48.404214 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.404783 16990 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:48.365901 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.405198 17298 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:57:48.383692 17000 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.388917 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.408056 13691 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:48.389739 13687 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:57:48.388087 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:48.409446 13689 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:48.371474 13659 leader_election.cc:400] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:57:48.392526 17140 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:48.409994 13659 leader_election.cc:403] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:57:48.409782 17292 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:57:48.411504 17271 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:48.411811 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.412935 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.393467 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:48.415177 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.415951 13616 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:48.416992 13616 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:57:48.416610 17173 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.418232 17173 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:48.418936 17120 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:48.419333 17120 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:48.419653 17120 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:48.419869 17216 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:48.422819 17290 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.423179 17067 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:48.423789 17067 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.424942 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.425550 13687 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:48.425971 17274 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.427211 17274 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:48.393685 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.428256 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.353471 17275 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:48.428859 17275 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:48.429235 17275 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:48.429417 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.430562 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.431571 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:48.432217 13615 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:48.432301 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 60977.4us
I20260709 12:57:48.418237 13538 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:48.399760 16774 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:48.435635 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.403668 13539 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:48.437350 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.437429 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:48.441551 13617 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:48.440109 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:48.469394 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:48.473007 13616 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:48.480808 13685 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:48.485373 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:48.497676 16978 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:48.498456 16978 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.499807 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.500229 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:48.499898 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.501230 13692 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:48.501380 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:48.502899 13687 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:48.518934 13687 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:48.519057 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.520742 13684 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
W20260709 12:57:48.522511 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.522565 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:48.509727 13690 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:57:48.523278 13693 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
W20260709 12:57:48.471683 17276 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.000s
I20260709 12:57:48.522047 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:48.528208 13617 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:48.472339 17298 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:48.533975 13614 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:48.500993 17222 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.543035 14242 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 12:57:48.528810 16817 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:48.502247 17303 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:48.555222 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.555666 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:48.555902 13541 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.556481 14246 raft_consensus.cc:2410] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:57:48.532176 17298 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:48.561825 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.560259 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:48.561960 14240 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEARNER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:48.563251 14239 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:48.566645 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 129325us
I20260709 12:57:48.567351 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:48.544203 17289 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.570647 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.578123 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:48.558123 17067 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.616745 14242 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:48.558748 16817 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:48.621631 16960 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:48.628196 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:48.624733 17304 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:57:48.628794 16978 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:48.568495 17298 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.622332 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:48.644086 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.644308 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.646624 13617 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:48.651886 13534 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:48.658354 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:48.661100 17216 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:48.671393 17216 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:48.672551 17216 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.674099 17216 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.661545 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:48.662479 17305 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:57:48.664628 13687 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.666589 16817 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.667407 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:48.670680 17307 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:57:48.670478 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.665172 17199 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.661109 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:48.676070 13536 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:48.680150 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.687403 13687 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.710975 13617 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:57:48.712133 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.659073 17217 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:48.658979 17290 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.746531 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:48.734622 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 73637.2us
I20260709 12:57:48.736979 13506 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:48.659061 16960 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:48.615573 17289 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.748423 13506 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:48.751402 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.735086 13612 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.755371 13612 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.760221 13619 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:48.767858 13506 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:48.777299 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.760689 17303 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:48.780460 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:48.780987 16960 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:48.784461 17308 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:57:48.784998 17275 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.787338 17275 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.780489 17303 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:48.789090 17199 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.789974 17303 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.790879 17289 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:48.792068 17120 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.793332 13537 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:48.794039 17293 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:48.794493 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.795354 13538 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:57:48.795274 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:48.796298 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.795024 17120 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.797755 13538 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:48.798991 13538 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:48.796308 13534 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:48.800858 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 688) took 1245 ms. Trace:
W20260709 12:57:48.801338 17296 consensus_meta.cc:287] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.001s
I20260709 12:57:48.801268 13535 rpcz_store.cc:276] 0709 12:57:47.555976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:47.556104 (+   128us) service_pool.cc:224] Handling call
0709 12:57:47.556459 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:47.922252 (+365793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:47.922306 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:47.942771 (+ 20465us) log.cc:844] Serialized 26 byte log entry
0709 12:57:48.316318 (+373547us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:48.316378 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:48.316449 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:48.800674 (+484225us) raft_consensus.cc:1685] finished
0709 12:57:48.800695 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:48.800808 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21909,"raft.queue_time_us":218266,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":3279,"thread_start_us":317636,"threads_started":2,"wal-append.queue_time_us":183956}
I20260709 12:57:48.802009 17296 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.751878 17276 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:48.803850 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:48.804576 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.794680 17293 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:48.805352 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.711884 17043 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:57:48.806080 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:48.800844 13538 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:48.750797 17288 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:48.838246 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 57890.6us
W20260709 12:57:48.794054 17313 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 12:57:48.813750 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.815191 13612 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:57:48.706948 17295 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.825559 13689 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:48.855105 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:48.855981 14242 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:48.858135 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:48.832844 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.809020 13541 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:48.845005 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:48.860854 13506 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:48.847124 13612 pending_rounds.cc:85] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 4
I20260709 12:57:48.861944 13612 pending_rounds.cc:107] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:57:48.862326 13612 raft_consensus.cc:2889] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:48.849756 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:48.864802 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:48.866250 13540 raft_consensus.cc:2393] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:57:48.867516 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.865320 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:48.871134 14246 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:48.871884 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:48.872551 14237 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:48.778443 16774 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:48.875167 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:48.868297 13658 connection.cc:256] RPC call timeout handler was delayed by -1.8427968110008806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 12:57:48.876547 13658 connection.cc:256] RPC call timeout handler was delayed by -1.6717968110006041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
W20260709 12:57:48.877043 13658 connection.cc:256] RPC call timeout handler was delayed by -1.5268968110003698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
W20260709 12:57:48.877439 13658 connection.cc:256] RPC call timeout handler was delayed by -1.4647968110002694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
I20260709 12:57:48.879711 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:48.880160 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:48.880646 13543 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:48.826573 17000 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:48.826525 17120 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.882807 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:48.886873 14243 raft_consensus.cc:2393] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:48.885705 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:48.879046 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 156820us
I20260709 12:57:48.894100 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:48.837606 17288 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:48.867136 17067 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:48.886159 17289 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:48.909340 17131 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.000s
W20260709 12:57:48.937121 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 43591.3us
I20260709 12:57:48.890202 17271 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
W20260709 12:57:48.937403 13615 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.000s
W20260709 12:57:48.932106 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 241052us
W20260709 12:57:48.938585 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 64934.3us
I20260709 12:57:48.949699 13615 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:48.952028 14212 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:48.953518 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:48.956552 13543 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:48.908905 17288 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:48.961241 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 73851us
I20260709 12:57:48.918733 17276 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:48.958431 17309 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:57:48.948915 17000 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:48.938643 16774 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:48.978051 13584 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:48.854852 17293 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:48.963958 17312 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:48.991556 17173 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
I20260709 12:57:48.991839 14241 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:57:48.992239 17172 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:49.000099 17310 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:49.018579 17310 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:49.019236 17293 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:49.003260 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:48.980928 17275 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:57:49.020265 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.008661 17296 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:49.020874 17275 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:57:49.009021 17311 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
W20260709 12:57:49.010159 17317 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:57:49.021330 17173 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:48.995241 17300 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:57:49.012455 13438 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 6, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 6 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.022718 17300 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:49.022163 17173 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.023169 17300 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:49.023293 13438 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:57:49.023947 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 45935.8us
I20260709 12:57:49.024825 13509 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:49.024890 13438 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 6, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 6 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:49.025727 17173 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:49.026540 13438 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 6, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 6 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:49.024137 17300 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.019510 17310 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.029948 17300 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:49.031136 13437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49880 (request call id 328) took 23516 ms (client timeout 15000 ms). Trace:
I20260709 12:57:49.031761 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:49.032406 13691 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:48.978929 16774 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:49.032931 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2736) took 4025 ms. Trace:
I20260709 12:57:49.033339 13612 rpcz_store.cc:276] 0709 12:57:45.019068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:45.230933 (+211865us) service_pool.cc:224] Handling call
0709 12:57:48.326786 (+3095853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:48.718877 (+392091us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b4c001ee440. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02ca7ea20 00007fa02ca8fa29 00007fa02ca55644 00007fa02ca7cf2d 00007fa02ca7cb93 00007fa02e7ad18b 00007fa02e7adcb5 00007fa02e7a8f30 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416
0709 12:57:48.862898 (+144021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:49.032705 (+169807us) spinlock_profiling.cc:243] Waited 2.98 s on lock 0x7b54001469d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:49.032871 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6882456704}
I20260709 12:57:48.976639 17276 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.035655 17063 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:49.036245 17063 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:49.036953 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 58980.8us
I20260709 12:57:49.036430 13438 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.037912 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:49.038642 17063 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:49.038780 17217 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:49.039577 17217 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:49.026151 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:49.041224 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:49.042593 17217 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:49.038509 13438 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.044179 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:49.046005 13691 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:49.013557 17295 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:49.045178 13438 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 7, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 7 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:49.065085 14242 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:49.058449 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.051944 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.070439 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:49.090570 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:49.094977 13536 raft_consensus.cc:2410] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:57:49.097584 13687 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:57:49.098132 13687 pending_rounds.cc:85] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 7
I20260709 12:57:49.098906 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:49.100026 13543 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:49.101135 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:49.101795 14238 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:49.106930 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:49.108210 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 64109.5us
I20260709 12:57:49.021283 17275 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:49.113090 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:49.113893 14244 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:49.113957 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:49.069841 14209 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.688s (SENT) [suppressed 32 similar messages]
I20260709 12:57:49.114900 13536 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:49.113323 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 87236.2us
W20260709 12:57:49.115088 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45290.5us
I20260709 12:57:49.116575 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:49.093365 17274 consensus_meta.cc:287] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.000s
I20260709 12:57:49.036443 17276 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:49.032963 16774 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:49.117604 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.117858 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:49.119427 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 258666us
I20260709 12:57:49.122488 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:49.122462 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:49.067521 17217 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:49.089685 17134 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:49.125787 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:49.096424 17292 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:49.095479 17324 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:49.098434 13687 pending_rounds.cc:107] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.8
I20260709 12:57:49.136138 13687 raft_consensus.cc:2917] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Aborting config change with OpId 8.8: Aborted: Op aborted by new leader
I20260709 12:57:49.136754 13687 raft_consensus.cc:2889] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:49.114900 17293 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:49.135253 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:49.118770 17173 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:49.144528 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:49.145272 13543 raft_consensus.cc:2393] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:49.143698 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:49.148466 14244 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:49.092224 17296 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:57:49.063851 17312 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.130192 16978 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:49.156108 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 246431us
I20260709 12:57:49.133723 17295 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:57:49.158638 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 205217us
I20260709 12:57:49.161201 13536 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:49.163969 13438 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.166385 13543 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:49.126600 17131 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:57:49.166496 14212 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:49.152012 17100 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:49.126413 17332 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:57:49.187762 14238 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:49.202332 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:49.206532 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 88739.2us
I20260709 12:57:49.133876 17285 consensus_queue.cc:1059] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:57:49.147465 17271 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:57:49.161257 17324 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:49.162529 17067 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:57:49.212249 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 68633.8us
I20260709 12:57:49.212431 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:49.213722 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:49.161921 17293 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:49.216856 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:49.163902 17295 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:49.225395 13438 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 2 to 8, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 8 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.138530 17325 consensus_queue.cc:1059] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:57:49.258005 13536 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:49.210191 17157 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:49.203994 17100 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:49.263180 13543 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:49.264220 13438 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.265961 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.272053 13692 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:49.282361 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.283134 13692 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:49.301994 13536 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:49.303205 13686 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:49.168874 17334 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 12:57:49.214435 17343 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:49.257598 17295 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:49.310454 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:49.031585 13437 rpcz_store.cc:269] 0709 12:57:25.534694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:25.535056 (+   362us) service_pool.cc:224] Handling call
0709 12:57:49.030790 (+23495734us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:57:38.192023 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:57:38.192776 (+   753us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:57:38.192829 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:57:38.192849 (+    20us) tablet.cc:662] Decoding operations
0709 12:57:38.194572 (+  1723us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:57:38.194664 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:57:38.194697 (+    33us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:57:38.456469 (+261772us) tablet.cc:701] Row locks acquired
0709 12:57:38.456526 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:57:38.456725 (+   199us) write_op.cc:270] Start()
0709 12:57:38.459795 (+  3070us) write_op.cc:276] Timestamp: P: 1783601858456644 usec, L: 0
0709 12:57:38.476511 (+ 16716us) op_driver.cc:348] REPLICATION: starting
0709 12:57:38.505252 (+ 28741us) log.cc:844] Serialized 7052 byte log entry
0709 12:57:40.034455 (+1529203us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02b6a80ac 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b
0709 12:57:40.346143 (+311688us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b180030ea40. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:40.767456 (+421313us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:42.933725 (+2166269us) op_driver.cc:464] REPLICATION: finished
0709 12:57:43.640558 (+706833us) write_op.cc:301] APPLY: starting
0709 12:57:43.640686 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 12:57:43.679978 (+ 39292us) tablet.cc:1370] finished BulkCheckPresence
0709 12:57:43.680016 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:57:43.682245 (+  2229us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:57:43.682700 (+   455us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 12:57:43.682755 (+    55us) write_op.cc:312] APPLY: finished
0709 12:57:43.683223 (+   468us) log.cc:844] Serialized 125 byte log entry
0709 12:57:44.024625 (+341402us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:44.024822 (+   197us) write_op.cc:454] Released schema lock
0709 12:57:44.024911 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 12:57:44.027059 (+  2148us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:44.027091 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":136157,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":625220,"apply.run_cpu_time_us":31813,"apply.run_wall_time_us":400069,"num_ops":18,"prepare.queue_time_us":294910,"prepare.run_cpu_time_us":28718,"prepare.run_wall_time_us":2072577,"raft.queue_time_us":1452963,"raft.run_cpu_time_us":24298,"raft.run_wall_time_us":581374,"replication_time_us":4297201,"spinlock_wait_cycles":249721728,"thread_start_us":3271369,"threads_started":5,"wal-append.queue_time_us":2120617,"wal-append.run_cpu_time_us":37113,"wal-append.run_wall_time_us":3854042}]]}
I20260709 12:57:49.265254 17276 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:49.310887 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:49.312973 13687 raft_consensus.cc:2410] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:57:49.257033 17271 consensus_queue.cc:1243] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer 0f34cda059124f839711dbef61bb2c28 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:57:49.313517 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 99877us
I20260709 12:57:49.171053 17275 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:49.279340 17100 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:49.318881 13438 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 7, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.307860 17290 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:49.312813 17343 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:49.358474 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.322327 13438 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 7, leader changed from <none> to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:57:49.357415 13536 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:49.362515 13686 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:49.324052 17298 consensus_meta.cc:287] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.658s	user 0.005s	sys 0.005s
I20260709 12:57:49.359148 13692 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:57:49.376307 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.377857 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.327368 17349 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:49.363456 17120 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:49.381062 13687 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:49.391240 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:49.403218 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.409375 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.410038 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 193276us
I20260709 12:57:49.369020 17217 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:49.363757 17343 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:49.416826 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:49.433071 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:49.357980 17290 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.440546 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.444427 13618 raft_consensus.cc:2410] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:57:49.445870 13614 raft_consensus.cc:2393] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:49.449743 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:49.453370 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.455761 13692 raft_consensus.cc:2393] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:49.454551 13506 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:49.375295 17298 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:49.450675 17326 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:57:49.468154 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.468839 13687 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:57:49.467988 17348 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:49.472370 16947 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:49.475503 17063 raft_consensus.cc:3055] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:49.475977 17063 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:49.476842 17063 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.478116 17312 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:49.478127 17063 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:49.479444 17341 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:57:49.481350 17222 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:49.481751 17222 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:49.482692 17222 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:49.483592 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.494946 13506 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:49.459991 17333 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:57:49.523486 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 697) took 1076 ms. Trace:
I20260709 12:57:49.524097 13537 rpcz_store.cc:276] 0709 12:57:48.433900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:48.434023 (+   123us) service_pool.cc:224] Handling call
0709 12:57:49.510109 (+1076086us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540028c154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:49.510321 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816469376}
I20260709 12:57:49.427055 17217 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.530900 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.531822 13687 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:49.534358 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:49.403689 17293 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.533623 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:49.542366 13540 raft_consensus.cc:2393] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:49.545786 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:49.568189 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.572450 17345 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 12:57:49.554112 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 59303.3us
I20260709 12:57:49.525195 17157 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:49.588379 13692 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:57:49.593943 17366 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:49.614984 17366 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:49.594791 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.625674 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:49.652772 13508 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:49.654340 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 74746.9us
I20260709 12:57:49.655397 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:49.669449 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.678013 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 299863us
W20260709 12:57:49.678727 14209 leader_election.cc:336] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.804s (SENT)
I20260709 12:57:49.699015 14209 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:49.711180 13438 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 7, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.696111 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:49.613466 17000 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.000s
I20260709 12:57:49.712785 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:49.712960 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 139054us
W20260709 12:57:49.716459 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.717316 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.710354 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:49.587980 17356 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:49.718855 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:49.696333 17289 consensus_meta.cc:287] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.001s
I20260709 12:57:49.677547 17349 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:49.722151 13614 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:49.724658 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:49.729888 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.763670 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:49.763484 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.768476 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.769047 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.693271 17347 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:49.769826 13616 raft_consensus.cc:1217] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[6.7-6.7]   Dedup: 6.7->[]
I20260709 12:57:49.776225 13616 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:49.770124 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:49.772048 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 452235us
I20260709 12:57:49.769871 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.774821 13610 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:49.777226 13684 raft_consensus.cc:2410] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:57:49.774132 13438 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 7, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:57:49.774331 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 95623.6us
I20260709 12:57:49.770196 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.770272 13613 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:49.771293 13617 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:49.798120 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.798902 13618 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8.
I20260709 12:57:49.784723 13690 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:49.772393 13619 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:49.803426 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.725745 17000 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:49.805214 13438 catalog_manager.cc:5697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 7, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 7 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:57:49.729657 17289 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:49.725443 17356 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:49.807200 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.814822 13615 raft_consensus.cc:2410] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:57:49.808190 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:49.802476 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.810720 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:49.811545 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.817919 14209 leader_election.cc:336] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.867s (SENT)
W20260709 12:57:49.818962 14209 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.870s (SENT)
I20260709 12:57:49.811863 13438 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 3 to 8, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 8 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.769096 17101 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:49.829636 13693 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:49.830078 13693 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:49.827014 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.781002 17347 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:49.833952 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:49.834762 13686 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:49.837524 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:49.831871 13610 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:49.844383 14209 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.077s (SENT)
I20260709 12:57:49.852738 13438 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 8, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 8 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:57:49.850466 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:49.868058 13688 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:49.869756 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:49.870548 13535 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:49.871093 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.771119 17353 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 12:57:49.809175 17194 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:57:49.851519 13693 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.823311 17356 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:49.881214 13659 leader_election.cc:336] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.611s (SENT)
I20260709 12:57:49.842288 17290 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:49.839844 17347 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:49.851486 17289 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:49.905403 13538 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:49.908205 13539 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:49.904091 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:49.881093 17350 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.340s	user 0.001s	sys 0.001s
I20260709 12:57:49.881999 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:49.929719 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 85390.3us
I20260709 12:57:49.881615 17298 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:49.939107 17300 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:57:49.939426 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 58247.2us
I20260709 12:57:49.935376 17354 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:57:49.957396 16774 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.827s	user 0.012s	sys 0.000s
I20260709 12:57:49.941368 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 702) took 1268 ms. Trace:
I20260709 12:57:49.928598 17289 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.943459 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.968523 13619 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:49.928913 17360 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:49.958259 17194 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:49.944813 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.945765 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:49.952380 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:49.955870 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.957564 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:49.833323 17157 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:49.959879 13693 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:49.960616 14209 leader_election.cc:336] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.083s (SENT)
W20260709 12:57:49.965562 13659 leader_election.cc:336] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.651s (SENT)
I20260709 12:57:50.016767 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:49.966205 17376 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:50.019500 17363 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:57:49.925290 17358 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:57:49.969463 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:49.944185 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:50.023852 13615 raft_consensus.cc:2368] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:57:49.960197 17271 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.012709 13618 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 5.
I20260709 12:57:49.964439 17217 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.022969 13619 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:50.030093 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:50.038367 14238 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:57:50.038851 14238 pending_rounds.cc:85] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 7
I20260709 12:57:49.971776 17360 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:50.061664 14238 pending_rounds.cc:107] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 10.8
W20260709 12:57:50.062177 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:49.968163 13542 rpcz_store.cc:276] 0709 12:57:48.673230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:48.866854 (+193624us) service_pool.cc:224] Handling call
0709 12:57:48.890486 (+ 23632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:48.890727 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:57:49.140399 (+249672us) log.cc:844] Serialized 14 byte log entry
0709 12:57:49.940930 (+800531us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b180039e380. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287
0709 12:57:49.941028 (+    98us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:49.941073 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:57:49.941121 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:49.941184 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:49.941295 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63956,"prepare.queue_time_us":67242,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2494,"raft.queue_time_us":64116,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":7916,"spinlock_wait_cycles":90746624,"thread_start_us":273312,"threads_started":2}
I20260709 12:57:50.064361 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:50.066979 13693 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260709 12:57:50.067539 13693 pending_rounds.cc:85] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 9
I20260709 12:57:50.067850 13693 pending_rounds.cc:107] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 12:57:50.062155 14238 raft_consensus.cc:2889] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:50.068359 13693 raft_consensus.cc:2889] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:49.971591 17194 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:50.061725 13616 raft_consensus.cc:2376] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9. Re-sending same reply.
I20260709 12:57:50.073774 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:50.077111 14242 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:50.012425 17349 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:50.079331 13619 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:50.079301 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:50.081288 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.083115 14244 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
W20260709 12:57:50.038921 17337 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
W20260709 12:57:50.085222 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 119679us
I20260709 12:57:50.011224 17354 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:50.087621 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1023) took 1305 ms. Trace:
I20260709 12:57:50.090113 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:50.087612 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.090780 13615 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:50.091411 13693 raft_consensus.cc:2368] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
I20260709 12:57:50.038610 17298 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.098034 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.098678 13693 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:50.090026 14245 rpcz_store.cc:276] 0709 12:57:48.782235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:48.826417 (+ 44182us) service_pool.cc:224] Handling call
0709 12:57:48.853619 (+ 27202us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:49.157272 (+303653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:49.157299 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:49.163866 (+  6567us) log.cc:844] Serialized 26 byte log entry
0709 12:57:49.220694 (+ 56828us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:49.220737 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:49.220802 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:50.087345 (+866543us) raft_consensus.cc:1685] finished
0709 12:57:50.087382 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:50.087550 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18413,"raft.queue_time_us":134724,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":341229,"threads_started":2,"wal-append.queue_time_us":284344}
W20260709 12:57:50.078483 17336 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:57:50.075300 17101 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.106119 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.106685 13693 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:49.971932 16774 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.070118 17217 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:50.107735 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 94046.6us
I20260709 12:57:50.084432 17063 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:50.078940 17379 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.070683 17360 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:50.115159 13691 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
W20260709 12:57:50.117360 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 156754us
W20260709 12:57:50.119869 17374 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:57:50.119614 17217 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:50.120528 17364 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 12:57:50.121337 17290 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:57:50.201857 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 46944.5us
I20260709 12:57:50.122756 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:50.202382 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:50.124621 17357 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 12:57:50.125075 17382 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:57:50.125453 17194 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.216936 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:50.129060 17289 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.103130 17377 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:50.105309 17367 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 12:57:50.146085 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.217680 14245 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:50.177047 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.224081 13693 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:50.196280 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.121266 17354 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.103578 17373 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:50.071759 17271 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:50.240921 17271 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:50.039577 17376 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.242998 17376 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.110298 17366 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.245064 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.125411 17349 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.252049 17373 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.254273 17373 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.255033 17194 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:50.261641 17343 consensus_meta.cc:287] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.818s	user 0.009s	sys 0.000s
W20260709 12:57:50.262177 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.203322 17379 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:50.263724 16978 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 1.099s	user 0.010s	sys 0.001s
I20260709 12:57:50.264150 16978 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:50.264685 17379 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.264849 16978 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.265383 16978 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.231634 13684 raft_consensus.cc:2368] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
I20260709 12:57:50.290709 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:50.232326 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 109692us
I20260709 12:57:50.230973 17300 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 12:57:50.235525 17100 consensus_meta.cc:287] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.907s	user 0.010s	sys 0.002s
I20260709 12:57:50.235436 17289 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:50.270874 17271 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.262221 17343 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.104935 17298 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.340858 13535 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:57:50.229970 13691 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
W20260709 12:57:50.286407 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.299224 13687 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:50.116083 17101 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.330055 13688 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:50.345801 13542 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:50.326753 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 273918us
W20260709 12:57:50.332639 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.002s	sys 0.006s
I20260709 12:57:50.261742 17217 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:50.336457 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 113556us
I20260709 12:57:50.345805 13686 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 9. Ops: []
W20260709 12:57:50.349151 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.002s	sys 0.007s
W20260709 12:57:50.351619 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 624959us
I20260709 12:57:50.333704 13693 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
W20260709 12:57:50.328915 17364 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:57:50.349869 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:50.372606 13543 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:50.350550 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.350085 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.375244 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.376257 13535 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:50.377981 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.378825 13693 raft_consensus.cc:2376] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7. Re-sending same reply.
I20260709 12:57:50.378890 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.383282 13540 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:50.375002 14211 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:50.387933 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.388664 13693 raft_consensus.cc:2376] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12. Re-sending same reply.
I20260709 12:57:50.389523 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.391080 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:50.394349 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.397130 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.397691 13691 raft_consensus.cc:2376] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9. Re-sending same reply.
I20260709 12:57:50.393271 13659 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:50.228890 17377 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.403677 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.460987 13693 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 9
W20260709 12:57:50.414690 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.007s	sys 0.019s
I20260709 12:57:50.329555 17100 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.402122 13687 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:50.373867 17289 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.455411 13543 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:57:50.457136 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 63987.2us
I20260709 12:57:50.344352 17271 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.459631 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:50.398499 17334 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:57:50.450330 17327 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.472s	user 0.004s	sys 0.000s
W20260709 12:57:50.461541 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.007s	sys 0.019s
I20260709 12:57:50.462258 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 610) took 1257 ms. Trace:
I20260709 12:57:50.464561 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.466207 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.466378 13537 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:50.399799 17365 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 12:57:50.459759 17357 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:57:50.491468 13693 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:50.471498 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:50.460438 17288 consensus_meta.cc:287] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.485s	user 0.010s	sys 0.003s
I20260709 12:57:50.477399 13688 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:50.508234 17288 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.474696 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:50.480275 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.478793 13612 rpcz_store.cc:276] 0709 12:57:49.204760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:49.211935 (+  7175us) service_pool.cc:224] Handling call
0709 12:57:49.212308 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:49.212644 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:49.212672 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:49.223604 (+ 10932us) log.cc:844] Serialized 52 byte log entry
0709 12:57:49.723045 (+499441us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:57:50.277169 (+554124us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b1800465be0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf
0709 12:57:50.277330 (+   161us) log.cc:844] Serialized 14 byte log entry
0709 12:57:50.277730 (+   400us) log.cc:844] Serialized 14 byte log entry
0709 12:57:50.400549 (+122819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:50.400628 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:50.461061 (+ 60433us) raft_consensus.cc:1685] finished
0709 12:57:50.461084 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:50.462143 (+  1059us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":149545,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":51039,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":76812,"spinlock_wait_cycles":7265152,"thread_start_us":670281,"threads_started":3,"wal-append.queue_time_us":642783}
I20260709 12:57:50.484061 13690 raft_consensus.cc:2393] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:50.493304 16774 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.508841 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.519696 17393 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:50.520167 13688 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:50.520795 17393 consensus_queue.cc:765] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Error trying to read ahead of the log while preparing peer request: Incomplete: Op with index 8 is ahead of the local log (next sequential op: 8). Destination peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.290s
I20260709 12:57:50.521251 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.521821 13688 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:50.521927 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.522691 13693 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:50.522751 17271 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:50.524719 17298 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.525444 17217 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.493702 17399 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.495441 16978 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.502099 17100 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.512365 13685 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:50.468643 17334 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:50.528441 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.541610 13687 raft_consensus.cc:2376] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9. Re-sending same reply.
I20260709 12:57:50.537773 13688 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:57:50.528311 17217 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.540314 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.476804 17289 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.551712 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:50.543705 13684 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:50.484467 17063 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:57:50.560490 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.561779 13542 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
W20260709 12:57:50.560967 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 86347.3us
I20260709 12:57:50.549033 17334 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:50.565444 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:50.567190 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.569300 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:50.567250 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1503) took 1414 ms. Trace:
I20260709 12:57:50.555048 16978 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:57:50.563747 17295 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 1.243s	user 0.011s	sys 0.002s
I20260709 12:57:50.576382 13691 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:57:50.570015 13689 rpcz_store.cc:276] 0709 12:57:49.152900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:49.156655 (+  3755us) service_pool.cc:224] Handling call
0709 12:57:49.170464 (+ 13809us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:49.198857 (+ 28393us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:49.432956 (+234099us) log.cc:844] Serialized 14 byte log entry
0709 12:57:49.780560 (+347604us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:49.829570 (+ 49010us) log.cc:844] Serialized 52 byte log entry
0709 12:57:49.829735 (+   165us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:49.829938 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 12:57:49.830109 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:49.830180 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:50.567059 (+736879us) raft_consensus.cc:1685] finished
0709 12:57:50.567080 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:50.567195 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":265638,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":8062,"raft.queue_time_us":53863,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":24493,"thread_start_us":425270,"threads_started":2,"wal-append.queue_time_us":404413}
I20260709 12:57:50.577590 17339 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:50.581652 17275 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:50.583808 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:50.592996 17356 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.684s	user 0.003s	sys 0.009s
I20260709 12:57:50.597811 17289 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.604379 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.611382 13612 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:57:50.612798 13688 raft_consensus.cc:2368] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
I20260709 12:57:50.611073 17378 consensus_queue.cc:1059] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:57:50.642732 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:50.642911 17378 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:50.642954 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.643445 17378 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:50.643776 13688 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:50.643496 13611 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:50.649813 17398 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.598296 17343 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.654865 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:50.602665 17339 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:50.547379 17399 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.653498 17398 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.660219 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.644549 17404 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 12:57:50.664031 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.668233 13688 raft_consensus.cc:2376] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9. Re-sending same reply.
I20260709 12:57:50.609696 17101 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:50.666086 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.655391 17157 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:57:50.609915 17295 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.664647 17339 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:50.681654 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.688757 13688 raft_consensus.cc:2376] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7. Re-sending same reply.
I20260709 12:57:50.698120 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.698881 13689 raft_consensus.cc:2376] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9. Re-sending same reply.
I20260709 12:57:50.703702 13543 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:50.706364 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.708276 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.707108 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.709551 13689 raft_consensus.cc:2368] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
W20260709 12:57:50.699546 17385 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:57:50.706844 17399 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.708359 17398 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.647364 17356 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:50.719189 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.720134 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:50.725736 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.727568 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:50.730010 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.733126 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:50.732856 13582 connection.cc:256] RPC call timeout handler was delayed by -1.2281351820001873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24180000000000001s for a response
W20260709 12:57:50.733400 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 78610.9us
W20260709 12:57:50.736209 13582 connection.cc:256] RPC call timeout handler was delayed by -1.1447790909996911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
I20260709 12:57:50.736523 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:50.710242 17271 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:50.739076 17271 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.741256 17271 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:50.734217 17063 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:50.742588 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.675040 17378 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.744004 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:50.744521 17347 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.821s	user 0.001s	sys 0.010s
I20260709 12:57:50.745141 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.738435 13658 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:50.747236 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:50.718992 17377 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.750176 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:50.751411 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:50.731092 17157 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:50.726590 17398 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:50.729071 16978 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.734215 17402 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.757673 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.745788 13542 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:50.735502 17217 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.735421 17288 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.747866 17063 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:50.789799 13690 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:50.792224 13690 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:50.791622 13537 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:50.763266 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:50.760319 13540 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:50.753264 17157 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:50.759879 17271 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:57:50.802083 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.788849 17295 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.789047 17217 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:50.744937 17347 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.796151 17398 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.792961 13690 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.810439 13690 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:50.792830 17276 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:50.761266 16978 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.783526 17402 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.809144 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.802259 17157 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:50.822728 17393 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:57:50.845928 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 94596.9us
I20260709 12:57:50.853653 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:50.839040 13542 raft_consensus.cc:2410] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:57:50.850813 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.854076 13688 raft_consensus.cc:2410] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:57:50.745342 17100 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:50.841293 17408 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.812378 17217 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.840363 17413 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:50.828152 17276 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.822543 17398 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.867880 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:50.870157 13689 raft_consensus.cc:2393] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:50.870666 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 134204us
I20260709 12:57:50.867177 17393 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:50.874822 17063 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:50.875728 17063 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:50.877696 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:50.871402 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:50.876569 17063 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:50.881495 17063 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:50.881647 17393 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:50.882460 17063 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.882619 17393 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.883704 17063 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:50.885891 17419 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:50.886320 17419 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:50.887203 17419 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.888460 17419 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:50.865885 17408 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:50.893744 17408 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.893959 17410 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:50.894804 17410 consensus_queue.cc:1243] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:57:50.894994 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 614) took 1094 ms. Trace:
I20260709 12:57:50.895402 13610 rpcz_store.cc:276] 0709 12:57:49.800790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:49.802130 (+  1340us) service_pool.cc:224] Handling call
0709 12:57:50.894824 (+1092694us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b54002fc6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:50.894936 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34720000}
I20260709 12:57:50.901036 17245 raft_consensus.cc:3055] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:50.901473 17276 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.901640 17245 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:50.902081 17276 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:57:50.902515 17276 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:50.902910 17245 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:50.905330 17402 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.906996 17398 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:50.908751 13537 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:57:50.909157 17372 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:50.909689 17372 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:50.911377 17360 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.000s
I20260709 12:57:50.912787 17372 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.913192 17217 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:50.914134 17383 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.441s	user 0.002s	sys 0.000s
I20260709 12:57:50.914901 13690 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:50.907073 13610 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:57:50.916282 13684 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:50.917098 17217 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:50.917430 16978 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:50.917578 17217 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.918140 17400 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:50.918617 17400 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:50.918965 17400 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:50.919732 17274 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:57:50.920022 17217 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:50.922132 17340 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:57:50.925689 17222 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:50.917500 13684 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:57:50.929705 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 52164.3us
I20260709 12:57:50.930465 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:50.931607 13542 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:50.871716 17356 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:50.928301 17413 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:50.903345 17276 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.956687 13542 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:50.956764 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:50.958132 13617 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:50.907047 17000 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 12:57:50.959956 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.961961 17413 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:50.963617 17347 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:50.964161 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:50.964351 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:50.964982 13615 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:50.965929 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:50.933876 17100 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:50.985121 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:50.986009 13618 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:50.997779 13537 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:50.997833 13542 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:50.956002 17419 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:50.911787 17360 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.004257 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:51.006520 13659 leader_election.cc:336] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.758s (SENT)
I20260709 12:57:51.039816 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.006894 13618 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:51.040867 13537 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
W20260709 12:57:51.009732 13539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1565) took 2113 ms (client timeout 1946 ms). Trace:
W20260709 12:57:51.042127 13539 rpcz_store.cc:269] 0709 12:57:48.896672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:49.523276 (+626604us) service_pool.cc:224] Handling call
0709 12:57:51.009420 (+1486144us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b540007e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:51.009592 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669518720}
I20260709 12:57:51.015785 13538 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
W20260709 12:57:51.025725 14209 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.785s (SENT)
W20260709 12:57:51.048985 14209 leader_election.cc:336] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.156s (SENT)
I20260709 12:57:51.028805 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:51.035742 13685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1505) took 1871 ms (client timeout 2039 ms). Trace:
W20260709 12:57:51.051453 13685 rpcz_store.cc:269] 0709 12:57:49.164166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:49.164408 (+   242us) service_pool.cc:224] Handling call
0709 12:57:51.035474 (+1871066us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001eebd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:51.035654 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2444942592}
I20260709 12:57:51.053580 13685 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:51.065085 13542 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:57:51.067276 16947 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:51.067668 17419 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:51.068781 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.069504 13613 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:51.069341 17400 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:51.080478 17412 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 12:57:51.080708 14245 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:57:51.081969 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:51.082509 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 76017.8us
W20260709 12:57:51.083575 13659 leader_election.cc:336] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.128s (SENT)
I20260709 12:57:51.084040 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:51.085217 13659 leader_election.cc:336] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.133s (SENT)
W20260709 12:57:51.086127 13659 leader_election.cc:336] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.206s (SENT)
W20260709 12:57:51.086705 17334 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
W20260709 12:57:51.088661 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 84461us
I20260709 12:57:51.089870 17419 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:51.090651 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.091799 17408 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:51.093176 17408 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:51.094038 17408 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:51.094429 17408 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:51.095125 13685 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:57:51.067850 16947 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.087610 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.083583 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.098837 13612 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:51.099771 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:51.101505 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:51.097791 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 152786us
I20260709 12:57:51.103946 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.105515 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.091703 17398 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 12:57:51.108273 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:51.108686 14242 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:51.087275 17334 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.109194 13685 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:57:51.108603 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:51.111167 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:51.111589 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:51.111773 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.090741 17217 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:51.093463 17360 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.101938 16947 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.107542 17419 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.096797 17404 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.132673 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.103571 17399 raft_consensus.cc:2764] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:57:51.110190 17000 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:51.132700 17400 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.137964 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:51.130975 17275 consensus_meta.cc:287] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 12:57:51.138164 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.139324 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:51.108866 17356 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.115697 17372 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:51.115113 17100 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.144012 13509 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.142624 13685 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:51.145237 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.098685 17157 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.147588 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:51.147658 13534 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:51.138897 17419 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.149094 17063 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:51.161042 13690 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:57:51.150450 17422 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:57:51.143052 17295 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:51.150785 17100 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:51.152701 13541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1559) took 2304 ms (client timeout 1805 ms). Trace:
I20260709 12:57:51.155179 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.156136 16978 raft_consensus.cc:2764] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:57:51.138298 17408 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.158231 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:51.149706 17424 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:57:51.115387 17409 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:57:51.141963 17000 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.166862 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:51.169333 17289 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:57:51.169991 17414 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:57:51.169931 17429 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:57:51.193095 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.161026 17419 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.176038 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.203716 14245 raft_consensus.cc:2368] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
I20260709 12:57:51.203903 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.205837 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.132072 17381 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:57:51.183048 13685 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:51.185451 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.194271 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:51.200649 13536 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:51.209265 13692 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:51.209506 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.201303 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.210125 13685 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.210429 13539 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 13 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:57:51.173934 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.211501 13688 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:51.206445 13542 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
W20260709 12:57:51.175401 13541 rpcz_store.cc:269] 0709 12:57:48.848198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.152497 (+2304299us) service_pool.cc:211] Skipping call since client already timed out
0709 12:57:51.152614 (+   117us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:57:51.180524 17372 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:51.213622 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.213984 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.214291 13692 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:51.180297 17353 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.209867 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.142275 17404 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.217206 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.179991 17347 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:51.187268 17400 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:57:51.273532 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:51.278658 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.276257 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.220906 14246 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:57:51.267939 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:51.161461 17356 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.284216 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.192837 17000 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:51.220850 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.202667 17134 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:51.192299 16774 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:51.277385 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.277782 13537 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:57:51.294086 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 91310us
I20260709 12:57:51.294557 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2974) took 1224 ms. Trace:
I20260709 12:57:51.287258 13685 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:51.295425 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.218324 17271 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:51.287947 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.201057 17414 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:51.282275 17382 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:57:51.281956 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:51.207404 17289 raft_consensus.cc:1068] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Unable to promote non-voter 0f34cda059124f839711dbef61bb2c28: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:57:51.208282 17360 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.285226 13689 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:51.201624 17108 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:57:51.292101 17296 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.494s	user 0.028s	sys 0.010s
I20260709 12:57:51.205660 17354 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:51.294940 13687 rpcz_store.cc:276] 0709 12:57:50.070683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:50.784721 (+714038us) service_pool.cc:224] Handling call
0709 12:57:50.797814 (+ 13093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:50.803123 (+  5309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:50.803153 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:50.806217 (+  3064us) log.cc:844] Serialized 26 byte log entry
0709 12:57:50.951613 (+145396us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:50.952112 (+   499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:50.952210 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:51.294341 (+342131us) raft_consensus.cc:1685] finished
0709 12:57:51.294363 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:51.294494 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":428,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":3216,"thread_start_us":122577,"threads_started":1,"wal-append.queue_time_us":189392}
I20260709 12:57:51.221990 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.222568 17409 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:51.328467 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.329314 13537 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:51.296686 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.288432 13618 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:51.333137 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.276700 17217 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:57:51.221158 16978 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.298420 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.279420 17358 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.222535 17300 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:57:51.344657 17384 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.102s	user 0.005s	sys 0.000s
I20260709 12:57:51.344975 13611 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:57:51.276031 17063 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:51.280359 17408 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.345810 17063 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.346223 17063 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.297546 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.348812 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.318845 13688 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:51.319022 13619 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:51.351516 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.206835 17398 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 12:57:51.352206 13619 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:51.275846 17400 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:51.352917 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.353638 13619 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:51.353482 17358 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.287173 17134 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.289705 17353 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.359557 17353 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.333387 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.336911 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.362205 13689 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:51.363581 13688 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:51.341209 16960 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:51.341384 17378 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.386416 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.389904 14239 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:57:51.389212 17339 consensus_meta.cc:287] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.689s	user 0.011s	sys 0.000s
I20260709 12:57:51.392690 17339 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.393949 14242 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:51.393467 17134 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.396000 17134 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:51.396286 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:51.396916 13542 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:51.397303 17339 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.293311 17432 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:57:51.391206 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.344568 17300 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:51.353005 17400 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:51.364524 17435 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:57:51.364995 17416 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:57:51.416800 17417 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:57:51.417054 17416 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:51.418179 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2971) took 1508 ms. Trace:
I20260709 12:57:51.418386 17416 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.418592 13693 rpcz_store.cc:276] 0709 12:57:49.909388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:50.523232 (+613844us) service_pool.cc:224] Handling call
0709 12:57:50.523517 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:50.524081 (+   564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:50.524110 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:50.525502 (+  1392us) log.cc:844] Serialized 26 byte log entry
0709 12:57:50.951157 (+425655us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:50.951209 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:50.951312 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:51.417946 (+466634us) raft_consensus.cc:1685] finished
0709 12:57:51.417966 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:51.418116 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":425212,"threads_started":1,"wal-append.queue_time_us":634716}
I20260709 12:57:51.317204 17342 raft_consensus.cc:2764] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:57:51.420480 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.366679 17157 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:51.333923 13616 raft_consensus.cc:2393] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 7: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:51.368098 17334 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:51.421665 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.368959 13509 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:51.422467 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53579.4us
I20260709 12:57:51.422938 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:57:51.426116 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 492) took 1460 ms. Trace:
I20260709 12:57:51.426548 13691 rpcz_store.cc:276] 0709 12:57:49.965402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:50.407486 (+442084us) service_pool.cc:224] Handling call
0709 12:57:51.425956 (+1018470us) spinlock_profiling.cc:243] Waited 99.1 ms on lock 0x7b5400159d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:51.426051 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227850624}
I20260709 12:57:51.369120 17108 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:51.428233 17108 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:51.430323 17425 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 12:57:51.430778 17108 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:51.431011 17353 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.431416 17353 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.369572 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2978) took 1023 ms. Trace:
W20260709 12:57:51.432972 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:51.434499 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.434898 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.296017 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.435669 17274 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.371172 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.297406 16774 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.415031 17300 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.373580 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.466571 13687 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:51.467592 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.373811 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.288516 17347 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.468076 13687 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 5.
I20260709 12:57:51.379309 13689 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:57:51.469123 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.380247 13692 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:51.470453 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:51.326541 17296 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.546s	user 0.033s	sys 0.010s
W20260709 12:57:51.385111 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:51.385105 13617 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:51.343516 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45181.8us
W20260709 12:57:51.404670 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 127358us
I20260709 12:57:51.339815 16978 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:51.378336 17358 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:51.413610 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:51.415979 17433 log.cc:927] Time spent T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:57:51.368145 17409 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:51.427591 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.434551 17178 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.480634 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.480800 17178 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.481793 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:51.482362 13687 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:51.482951 17178 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.436167 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.483582 13685 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:51.431761 17353 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.440371 17271 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.440887 17411 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:51.370223 17100 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:51.390712 16960 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:51.432649 13690 rpcz_store.cc:276] 0709 12:57:50.346130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:50.915609 (+569479us) service_pool.cc:224] Handling call
0709 12:57:51.157979 (+242370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:51.161566 (+  3587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:51.369367 (+207801us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54003b33d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:51.369508 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556746624}
I20260709 12:57:51.495481 13690 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:51.446738 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.446861 13693 raft_consensus.cc:2376] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9. Re-sending same reply.
I20260709 12:57:51.477584 16978 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.450598 13542 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:51.451390 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.451481 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:51.452178 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:51.459371 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.373186 17356 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:51.463272 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.500759 14237 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:51.465020 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:51.465719 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.501509 14246 raft_consensus.cc:2368] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9.
I20260709 12:57:51.471212 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.473356 14209 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.473680 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.475155 13537 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.502789 13687 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:51.476172 13543 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:51.476516 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.479172 13691 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:51.489156 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.505667 13692 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:51.458463 17108 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:51.509821 17100 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:51.509733 17108 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:51.510389 17423 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.427s	user 0.002s	sys 0.000s
I20260709 12:57:51.512740 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 488) took 1679 ms. Trace:
I20260709 12:57:51.513109 13686 rpcz_store.cc:276] 0709 12:57:49.833901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:49.843400 (+  9499us) service_pool.cc:224] Handling call
0709 12:57:50.345644 (+502244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:50.374223 (+ 28579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:51.512537 (+1138314us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54003a0a50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:51.512683 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076021632}
I20260709 12:57:51.515486 17404 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.450223 17295 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:57:51.436070 17274 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.517187 17274 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.518461 17329 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:51.519003 17329 raft_consensus.cc:697] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:51.519804 17329 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.503448 16960 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.510428 17100 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:51.524818 17100 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:51.528121 17415 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:51.528467 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.529101 13617 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:51.472468 17416 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:51.499709 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.481809 17347 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:51.506872 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.506909 14237 raft_consensus.cc:1217] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Deduplicated request from leader. Original: 5.6->[6.7-6.7]   Dedup: 6.7->[]
I20260709 12:57:51.507855 13687 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:51.509930 13693 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:57:51.509759 14241 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:51.490692 17411 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:51.547631 14237 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:57:51.404546 17414 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.533931 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.536039 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.541635 14240 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:51.545804 17217 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:57:51.536181 17413 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.559351 17178 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:51.540059 17356 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:51.560004 17399 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:51.564363 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.565698 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.596988 13619 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.567513 13537 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:51.581962 13684 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:51.598388 17393 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:57:51.599225 17438 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:57:51.599354 17393 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:51.599779 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.592890 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:51.601235 14240 raft_consensus.cc:2393] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:51.599321 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 519) took 1004 ms. Trace:
I20260709 12:57:51.602301 13584 leader_election.cc:400] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9.
I20260709 12:57:51.602077 13684 rpcz_store.cc:276] 0709 12:57:50.594501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:50.989044 (+394543us) service_pool.cc:224] Handling call
0709 12:57:51.039908 (+ 50864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:51.598090 (+558182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:51.599083 (+   993us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b540001c054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:51.599270 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245945600}
I20260709 12:57:51.583124 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:51.559350 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52530us
I20260709 12:57:51.603813 14237 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:51.604131 14209 leader_election.cc:400] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
I20260709 12:57:51.604470 14209 leader_election.cc:403] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:57:51.606050 13615 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:57:51.608754 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.610939 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.585650 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:51.556557 17000 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.587275 17063 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.633844 17178 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:51.635736 17063 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:51.640311 13613 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:57:51.594007 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.536309 17108 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.595260 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.579054 17347 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.612299 17100 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.612779 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.606374 17409 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.618096 17360 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:51.621661 17000 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.623090 16978 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.624147 17329 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9.
I20260709 12:57:51.606262 17356 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:51.629417 17402 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:51.761445 17402 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:51.629274 17419 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.762439 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:51.541719 17415 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:51.764202 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 372097us
I20260709 12:57:51.764609 17415 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.764919 13506 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:51.765377 13688 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:51.765669 17377 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.763659 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:51.766047 17377 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:51.766471 13536 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.632829 17400 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.768600 17276 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:51.633601 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.642956 14237 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:57:51.611817 13687 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:57:51.644624 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:51.648943 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 53775.7us
I20260709 12:57:51.671175 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:51.672545 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.599792 17393 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:51.744853 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:51.748728 13534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1562) took 2807 ms (client timeout 1886 ms). Trace:
I20260709 12:57:51.751019 13540 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:57:51.651659 17295 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:51.778764 13534 rpcz_store.cc:269] 0709 12:57:48.882072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:49.373940 (+491868us) service_pool.cc:224] Handling call
0709 12:57:51.678389 (+2304449us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400380254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:51.689249 (+ 10860us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3968327040}
I20260709 12:57:51.649879 17426 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:51.753247 13541 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:57:51.754920 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.756100 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.791074 13611 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.757879 17434 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.758868 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.760365 17356 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.771699 13615 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:51.790508 17426 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:51.809736 17426 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.772487 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:51.810756 17426 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.811641 13582 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.812573 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.813371 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:51.813575 17415 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:51.814200 17415 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:51.593355 17413 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:51.816185 17360 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.817298 17360 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.671298 17432 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.817822 17360 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.786325 17400 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:51.753036 17053 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:57:51.818502 17413 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:57:51.808811 17390 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:51.816115 17217 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:51.820802 17100 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:51.821027 17217 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.767609 17238 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:57:51.775034 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:51.671926 17347 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:51.774138 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:51.822052 14240 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:51.822563 17238 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:57:51.822619 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 50722.6us
I20260709 12:57:51.823364 17238 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:51.823532 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:51.825001 17000 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:51.825356 17000 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:51.825680 17000 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.788930 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:51.791771 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:51.772301 13542 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:51.828424 13612 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:57:51.808322 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 53519.3us
I20260709 12:57:51.766476 17377 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.652108 17063 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 12:57:51.821453 17217 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.609941 17300 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:51.808060 17393 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.828161 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:52.719303 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 891133us
I20260709 12:57:51.785461 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:52.726336 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 940891us
I20260709 12:57:52.727272 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:51.801846 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:52.781010 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 979175us
I20260709 12:57:52.781865 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:51.833057 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:52.418946 13619 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.812s	user 0.018s	sys 0.005s
I20260709 12:57:52.784425 13619 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
W20260709 12:57:52.784914 13619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 629) took 1223 ms (client timeout 1626 ms). Trace:
W20260709 12:57:52.785219 13619 rpcz_store.cc:269] 0709 12:57:51.561207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.564089 (+  2882us) service_pool.cc:224] Handling call
0709 12:57:52.784858 (+1220769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:52.785547 17472 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:52.492877 13692 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:52.511730 17465 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:52.530542 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:52.786348 17471 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:52.786779 17471 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:52.787099 17471 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:52.537648 13657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.005s	user 0.024s	sys 0.075s
I20260709 12:57:52.787266 16817 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:57:52.787835 13657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.256s	user 0.025s	sys 0.076s
I20260709 12:57:52.787860 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1530) took 1592 ms. Trace:
W20260709 12:57:52.548120 13539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.025s	sys 0.055s
I20260709 12:57:52.788372 13692 rpcz_store.cc:276] 0709 12:57:51.195972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.506210 (+310238us) service_pool.cc:224] Handling call
0709 12:57:51.567962 (+ 61752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:52.786160 (+1218198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:52.787619 (+  1459us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b54002963d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:52.787802 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2126700800}
W20260709 12:57:52.789083 13539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.025s	sys 0.055s
I20260709 12:57:52.788847 16817 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:57:52.790283 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 717) took 1104 ms. Trace:
I20260709 12:57:52.790642 13539 rpcz_store.cc:276] 0709 12:57:51.685626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.687645 (+  2019us) service_pool.cc:224] Handling call
0709 12:57:51.759621 (+ 71976us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:57:51.760105 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:51.760139 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:57:51.760736 (+   597us) log.cc:844] Serialized 52 byte log entry
0709 12:57:52.789471 (+1028735us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:52.789811 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 12:57:52.789993 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:52.790061 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:52.790101 (+    40us) raft_consensus.cc:1685] finished
0709 12:57:52.790113 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:52.790227 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":163,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":656,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":1028337,"threads_started":1,"wal-append.queue_time_us":483217}
I20260709 12:57:52.792938 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:52.793504 13539 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:52.794199 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:52.566476 16774 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:51.946463 17000 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.905794 17238 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:52.490489 17271 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:51.941051 17300 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:52.301469 17377 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:51.923553 16947 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.784s	user 0.007s	sys 0.004s
I20260709 12:57:52.989048 16947 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.749058 17043 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 12:57:51.849143 17063 consensus_queue.cc:1243] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:57:51.845810 17067 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:57:52.295771 17217 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:52.477696 17245 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:52.257416 17274 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.740s	user 0.007s	sys 0.004s
W20260709 12:57:52.298581 17276 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 12:57:51.780445 17277 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:57:53.022814 17217 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.023401 17390 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:51.933638 17322 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.189s
I20260709 12:57:51.822022 17347 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:51.963472 17351 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:51.864738 17353 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:52.294664 17360 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:51.896647 17393 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:51.818547 17400 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:51.687891 17411 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:51.815166 17415 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:52.248333 17419 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:52.257838 17432 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:51.831212 17434 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:52.573999 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.064697 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 724) took 1307 ms. Trace:
I20260709 12:57:53.065083 13535 rpcz_store.cc:276] 0709 12:57:51.757751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:52.573693 (+815942us) service_pool.cc:224] Handling call
0709 12:57:53.064619 (+490926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:52.479310 17437 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:53.068151 17437 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:53.069820 17437 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:52.282596 17416 consensus_meta.cc:287] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.736s	user 0.009s	sys 0.000s
W20260709 12:57:52.594596 13612 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.000s
I20260709 12:57:53.073431 13612 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:53.073994 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 630) took 1512 ms. Trace:
I20260709 12:57:53.074328 13612 rpcz_store.cc:276] 0709 12:57:51.561473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.753693 (+192220us) service_pool.cc:224] Handling call
0709 12:57:53.073925 (+1320232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:52.607836 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:53.075976 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 468192us
I20260709 12:57:53.076810 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:52.622903 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.078830 17354 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:57:53.079458 13537 raft_consensus.cc:2376] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11. Re-sending same reply.
I20260709 12:57:53.080401 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1589) took 1283 ms. Trace:
I20260709 12:57:53.080708 17354 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.081107 13537 rpcz_store.cc:276] 0709 12:57:51.797002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:52.622589 (+825587us) service_pool.cc:224] Handling call
0709 12:57:53.080334 (+457745us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:52.631807 13536 consensus_meta.cc:287] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.865s	user 0.015s	sys 0.006s
I20260709 12:57:53.083391 13536 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
W20260709 12:57:53.083891 13536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 725) took 1325 ms (client timeout 1688 ms). Trace:
W20260709 12:57:53.084249 13536 rpcz_store.cc:269] 0709 12:57:51.758098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.763426 (+  5328us) service_pool.cc:224] Handling call
0709 12:57:53.083836 (+1320410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:53.085999 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1590) took 1287 ms. Trace:
I20260709 12:57:52.635181 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.087031 13538 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:53.087431 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 720) took 1400 ms. Trace:
I20260709 12:57:53.087728 13538 rpcz_store.cc:276] 0709 12:57:51.686587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:52.634906 (+948319us) service_pool.cc:224] Handling call
0709 12:57:53.087376 (+452470us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:53.089161 13538 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:53.090548 13538 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:52.640241 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:52.662031 13611 consensus_meta.cc:287] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.870s	user 0.016s	sys 0.004s
I20260709 12:57:53.092188 13611 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
W20260709 12:57:53.092794 13611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1689) took 1450 ms (client timeout 1596 ms). Trace:
W20260709 12:57:53.093134 13611 rpcz_store.cc:269] 0709 12:57:51.642202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.654627 (+ 12425us) service_pool.cc:224] Handling call
0709 12:57:53.092732 (+1438105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:57:52.677371 14239 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 675) took 2648 ms (client timeout 1754 ms). Trace:
I20260709 12:57:52.680152 17466 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:53.094038 14239 rpcz_store.cc:269] 0709 12:57:50.028927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:50.029051 (+   124us) service_pool.cc:224] Handling call
0709 12:57:52.677157 (+2648106us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004c2a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:52.677295 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3011800960}
W20260709 12:57:52.683306 13584 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.017s	sys 0.028s
I20260709 12:57:52.678684 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.094360 17466 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:53.095479 13584 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.389s	user 0.018s	sys 0.029s
W20260709 12:57:53.095636 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 417000us
I20260709 12:57:53.095836 17245 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:57:52.713291 17453 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.793s	user 0.004s	sys 0.000s
I20260709 12:57:53.096788 17245 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:52.693598 13543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1485) took 1400 ms (client timeout 1578 ms). Trace:
I20260709 12:57:52.735702 13506 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:52.739871 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.097575 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:53.098320 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1587) took 1322 ms. Trace:
I20260709 12:57:53.098696 13541 rpcz_store.cc:276] 0709 12:57:51.776295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:52.729319 (+953024us) service_pool.cc:224] Handling call
0709 12:57:53.098260 (+368941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:52.728863 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:52.736666 17296 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:53.099884 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 371062us
I20260709 12:57:53.100230 17296 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:53.100534 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:53.100843 17322 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:53.101410 17411 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.101984 17322 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.102145 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:53.102705 17466 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.103144 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2790) took 1570 ms. Trace:
I20260709 12:57:53.102921 17322 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:52.485561 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:53.104467 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.103471 13617 rpcz_store.cc:276] 0709 12:57:51.532247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.533046 (+   799us) service_pool.cc:224] Handling call
0709 12:57:51.533340 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:51.584725 (+ 51385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:51.584761 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:51.585202 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:57:51.836665 (+251463us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:57:52.394418 (+557753us) log.cc:844] Serialized 14 byte log entry
0709 12:57:52.394749 (+   331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:52.394825 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:52.712760 (+317935us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:53.102843 (+390083us) raft_consensus.cc:1685] finished
0709 12:57:53.102945 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:53.103083 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":608277,"prepare.queue_time_us":557503,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":60179,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":14995,"spinlock_wait_cycles":35391232,"thread_start_us":209199,"threads_started":1,"wal-append.queue_time_us":314038}
W20260709 12:57:53.104956 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:52.756959 17408 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:52.786377 17465 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.105614 17408 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:53.106048 17408 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:53.100450 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.107288 17465 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:52.816030 13539 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:52.779323 17381 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:53.107684 17351 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:52.796118 17472 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.108285 17351 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:52.824354 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:52.826406 14244 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:52.832857 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2770) took 1614 ms. Trace:
I20260709 12:57:52.811149 17271 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.110165 13613 rpcz_store.cc:276] 0709 12:57:51.219234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.276106 (+ 56872us) service_pool.cc:224] Handling call
0709 12:57:51.276487 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:51.642800 (+366313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:52.832606 (+1189806us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400333bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:52.832796 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836142848}
I20260709 12:57:52.830358 17000 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.109186 17351 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:52.855000 14241 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:52.887250 17300 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:53.112481 17217 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:52.911736 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 129935us
I20260709 12:57:53.112856 17000 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:53.113003 17217 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:53.113441 17000 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:53.113620 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:53.114353 17000 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:52.841717 17355 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:52.917891 17474 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:53.116194 17355 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.117504 17300 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:53.117919 17416 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:52.841996 16774 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.118350 17300 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:53.113543 17217 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:53.118697 16774 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.118898 17300 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:52.937321 17477 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.120306 17477 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:52.918336 17475 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:52.947387 17409 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:53.121727 17475 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.122238 17409 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:52.948702 17471 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:52.963594 13430 maintenance_manager.cc:419] P 7c7c6d1a2ec946878b513e7eb3a47a90: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033447
W20260709 12:57:52.976104 14242 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 680) took 1542 ms (client timeout 1676 ms). Trace:
I20260709 12:57:53.124715 17472 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:53.124958 14242 rpcz_store.cc:269] 0709 12:57:51.433964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.434086 (+   122us) service_pool.cc:224] Handling call
0709 12:57:52.975857 (+1541771us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540005fd54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:52.976028 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2993469696}
I20260709 12:57:53.125628 17471 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:52.950577 13542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:53.126612 13542 raft_consensus.cc:606] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:57:53.126694 13619 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:53.127171 13542 raft_consensus.cc:614] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:57:53.127858 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.128654 13612 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:52.974681 17377 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:52.978852 17414 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:53.132773 17434 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.134104 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:53.135484 13658 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:52.986011 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 530) took 1870 ms. Trace:
I20260709 12:57:53.118357 17416 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:53.136502 17416 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.136077 13685 rpcz_store.cc:276] 0709 12:57:51.115822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.484248 (+368426us) service_pool.cc:224] Handling call
0709 12:57:51.504986 (+ 20738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:51.505187 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:52.700519 (+1195332us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b18003e7b80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7c55e5
0709 12:57:52.700671 (+   152us) log.cc:844] Serialized 14 byte log entry
0709 12:57:52.728491 (+ 27820us) log.cc:844] Serialized 14 byte log entry
0709 12:57:52.958972 (+230481us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b18003e7b80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287
0709 12:57:52.959092 (+   120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:52.959145 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:52.959201 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:52.959286 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:52.975159 (+ 15873us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b18003e7b80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:52.985811 (+ 10652us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b54002968d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:52.985943 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":468776,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":75,"raft.queue_time_us":11802,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":5789,"spinlock_wait_cycles":159292800,"thread_start_us":232894,"threads_started":3,"wal-append.queue_time_us":43709}
I20260709 12:57:52.992678 16947 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.019593 17274 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:52.993940 16978 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.138844 16978 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:53.138970 17238 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:53.002430 17329 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:53.021122 17473 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.140051 17172 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:53.140434 17172 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.140599 17329 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:57:53.140252 17473 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.141206 17329 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.141746 17172 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.142354 17329 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.143216 17172 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:53.143527 17329 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:53.004000 17402 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.144094 16978 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.144464 17402 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:53.144680 16978 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.147462 16978 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:51.792372 17295 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.086338 13536 rpcz_store.cc:276] 0709 12:57:51.798691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.084999 (+1286308us) service_pool.cc:224] Handling call
0709 12:57:53.085391 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:53.085683 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:53.085711 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:53.085752 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:53.085788 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:53.085841 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:53.085950 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:53.150977 13537 raft_consensus.cc:1240] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:53.151458 17295 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.066912 14211 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.152181 13689 raft_consensus.cc:2368] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
I20260709 12:57:53.049774 14237 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 12:57:53.096208 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 1.38996e+06us
I20260709 12:57:53.153739 17172 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:53.092234 13534 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:53.154539 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1588) took 1358 ms. Trace:
I20260709 12:57:53.154887 13534 rpcz_store.cc:276] 0709 12:57:51.796132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.832321 (+ 36189us) service_pool.cc:224] Handling call
0709 12:57:53.154478 (+1322157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:52.688603 17348 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:53.156205 17274 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.156246 17348 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.097563 17245 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:53.158300 13689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2989) took 1986 ms (client timeout 2150 ms). Trace:
W20260709 12:57:53.158703 13689 rpcz_store.cc:269] 0709 12:57:51.171980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.468898 (+296918us) service_pool.cc:224] Handling call
0709 12:57:53.158115 (+1689217us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400101b54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:53.158242 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3869084416}
I20260709 12:57:53.159196 17348 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:52.774910 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:52.846060 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.116272 17474 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:53.097847 13543 rpcz_store.cc:269] 0709 12:57:51.293441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.296464 (+  3023us) service_pool.cc:224] Handling call
0709 12:57:52.693383 (+1396919us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000b90d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:52.693527 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330335488}
I20260709 12:57:53.137444 17043 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:53.162112 17474 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:52.925729 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.144294 13614 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:57:53.036309 13438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41440 (request call id 208) took 16001 ms (client timeout 15000 ms). Trace:
W20260709 12:57:53.164098 13438 rpcz_store.cc:269] 0709 12:57:37.034877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.037381 (+  2504us) service_pool.cc:224] Handling call
0709 12:57:53.035840 (+15998459us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:57:50.594194 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:57:50.595530 (+  1336us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:57:50.595609 (+    79us) write_op.cc:435] Acquired schema lock
0709 12:57:50.595624 (+    15us) tablet.cc:662] Decoding operations
0709 12:57:50.759995 (+164371us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:57:50.791661 (+ 31666us) write_op.cc:641] Partition lock acquired for write op
0709 12:57:50.795715 (+  4054us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:57:51.172796 (+377081us) tablet.cc:701] Row locks acquired
0709 12:57:51.172846 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:57:51.173015 (+   169us) write_op.cc:270] Start()
0709 12:57:51.173161 (+   146us) write_op.cc:276] Timestamp: P: 1783601871172946 usec, L: 0
0709 12:57:51.173192 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:57:51.173989 (+   797us) log.cc:844] Serialized 7056 byte log entry
0709 12:57:51.601311 (+427322us) op_driver.cc:464] REPLICATION: finished
0709 12:57:52.227972 (+626661us) write_op.cc:301] APPLY: starting
0709 12:57:52.228090 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:57:52.228554 (+   464us) tablet.cc:1370] finished BulkCheckPresence
0709 12:57:52.228789 (+   235us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:57:52.241618 (+ 12829us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:57:52.245244 (+  3626us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 12:57:52.245313 (+    69us) write_op.cc:312] APPLY: finished
0709 12:57:52.260412 (+ 15099us) log.cc:844] Serialized 125 byte log entry
0709 12:57:52.266757 (+  6345us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:52.271635 (+  4878us) write_op.cc:454] Released schema lock
0709 12:57:52.275316 (+  3681us) write_op.cc:341] FINISH: Updating metrics
0709 12:57:52.276463 (+  1147us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:52.276498 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4132754,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":626192,"apply.run_cpu_time_us":6819,"apply.run_wall_time_us":48506,"num_ops":18,"prepare.queue_time_us":134985,"prepare.run_cpu_time_us":60150,"prepare.run_wall_time_us":747995,"raft.queue_time_us":170177,"raft.run_cpu_time_us":15562,"raft.run_wall_time_us":243508,"replication_time_us":427942,"spinlock_wait_cycles":37120,"thread_start_us":553535,"threads_started":4,"wal-append.queue_time_us":230440}]]}
I20260709 12:57:53.167225 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:53.052364 17400 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:53.167802 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:53.100718 17296 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:53.168562 17277 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.168962 14241 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:57:53.169116 17277 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:53.169000 13614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2778) took 1942 ms (client timeout 1926 ms). Trace:
W20260709 12:57:53.170351 13614 rpcz_store.cc:269] 0709 12:57:51.226959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.343147 (+116188us) service_pool.cc:224] Handling call
0709 12:57:53.163874 (+1820727us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540020f654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:53.168931 (+  5057us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3854066944}
I20260709 12:57:53.170750 17477 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.171289 17277 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.168133 17400 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:53.172005 17465 consensus_queue.cc:1175] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ad2fdb1517c542439654bd3f7c72dad0" responder_term: 9 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:57:53.172703 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:53.173745 14246 raft_consensus.cc:2410] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:57:53.174729 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.175379 14244 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:53.167091 16817 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:53.152505 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 85724us
I20260709 12:57:53.176115 16817 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:53.177435 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 75372.3us
I20260709 12:57:53.178503 16817 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.178471 13582 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:53.177459 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.180604 17300 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.180976 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:53.181200 17271 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.181792 17043 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:53.182574 17043 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.183485 17390 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:53.184948 17390 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:53.185309 17390 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.186033 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.186695 13543 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:53.186245 17390 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.184978 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.187721 17390 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.078903 13508 leader_election.cc:400] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
W20260709 12:57:53.189157 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 110393us
I20260709 12:57:53.189960 17355 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.189168 17475 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.192302 17409 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:53.192636 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.193593 17360 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.198316 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 726) took 1439 ms. Trace:
I20260709 12:57:53.183423 17043 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.199008 13537 rpcz_store.cc:276] 0709 12:57:51.758404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.082134 (+1323730us) service_pool.cc:224] Handling call
0709 12:57:53.082422 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:53.151431 (+ 69009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:53.197808 (+ 46377us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b540028cb54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:53.198059 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157204224}
I20260709 12:57:53.203030 16774 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.189985 17473 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.205693 17477 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.206298 17466 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.206480 17471 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.207314 17478 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:53.206880 17466 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.204694 17437 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:53.208372 17465 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:53.207883 17478 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.208881 17465 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.209633 17466 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.210230 17466 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.210561 17478 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.210680 17466 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.207007 17471 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.213253 17471 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.206344 17477 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.206756 17063 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.214635 17063 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.215992 17477 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.207631 17402 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.050784 17353 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.216760 17402 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:53.217221 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 81805us
I20260709 12:57:53.217543 17063 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.218156 17063 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:53.218696 17063 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:53.218801 17402 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.210960 17465 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.218242 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:53.224124 16978 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.224906 16978 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.225529 17419 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:53.226106 17419 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:53.226326 13610 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:53.226548 17426 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:53.227464 13610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2783) took 1924 ms (client timeout 1657 ms). Trace:
I20260709 12:57:53.226984 17419 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:53.227895 13610 rpcz_store.cc:269] 0709 12:57:51.303479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.322591 (+ 19112us) service_pool.cc:224] Handling call
0709 12:57:53.227276 (+1904685us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b540013b0d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:53.227391 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4287477632}
I20260709 12:57:53.227054 17426 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.229864 13616 raft_consensus.cc:2410] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:57:53.230938 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39902 (request call id 628) took 1670 ms. Trace:
I20260709 12:57:53.231312 17426 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.232087 17296 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.231323 13616 rpcz_store.cc:276] 0709 12:57:51.560905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.567507 (+  6602us) service_pool.cc:224] Handling call
0709 12:57:53.230724 (+1663217us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540020a154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:53.230878 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3649515264}
W20260709 12:57:53.237267 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 501630us
I20260709 12:57:53.238260 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.238629 17275 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:53.238814 13506 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.239220 17275 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:53.240321 13689 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:53.240160 17275 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.241045 17289 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:57:53.241537 17289 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:53.242372 17289 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:53.243503 17416 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.245666 17416 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.248847 17354 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:53.250213 13540 raft_consensus.cc:2410] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:57:53.251327 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 722) took 1564 ms. Trace:
I20260709 12:57:53.251623 13540 rpcz_store.cc:276] 0709 12:57:51.687154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:52.774604 (+1087450us) service_pool.cc:224] Handling call
0709 12:57:53.251128 (+476524us) spinlock_profiling.cc:243] Waited 89.8 ms on lock 0x7b540028bc54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:53.251274 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206438784}
I20260709 12:57:53.253090 17354 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:57:53.255909 13614 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:53.259112 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.259361 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:53.259586 17348 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.259723 13614 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:53.259968 13689 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:53.260161 17348 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.262349 17348 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.263676 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:53.265120 13685 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:57:53.265278 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:53.265547 13685 pending_rounds.cc:85] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 5
I20260709 12:57:53.264431 13540 raft_consensus.cc:606] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.265846 13685 pending_rounds.cc:107] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:57:53.266033 13690 raft_consensus.cc:2393] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:53.266139 13685 raft_consensus.cc:2889] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:53.266068 13540 raft_consensus.cc:614] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:57:53.265134 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:53.267593 13692 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:53.268340 17381 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:53.269304 17381 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.269941 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85eb0ac09bf7466899637997aae42111"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:53.270512 13540 raft_consensus.cc:606] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:57:53.273502 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:53.274327 13540 raft_consensus.cc:606] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.274741 13540 raft_consensus.cc:614] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:57:53.276222 17353 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.276830 17353 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:53.280275 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 90329.7us
I20260709 12:57:53.281751 17353 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.282521 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:53.281068 17472 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:53.285184 17067 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:53.286820 17295 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:57:53.287253 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:57:53.288056 13540 raft_consensus.cc:606] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.288458 13540 raft_consensus.cc:614] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:57:53.289038 14212 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.289515 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:57:53.281805 17245 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.289835 17390 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:53.290599 17277 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:53.290869 17390 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:53.291057 17277 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:53.291306 17390 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.291496 17277 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:53.291837 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:57:53.292264 17390 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.293367 17390 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.297531 17474 raft_consensus.cc:2764] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:57:53.297739 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.298321 17474 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.298743 17474 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.300119 17432 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:57:53.301461 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:53.219691 17063 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.303146 17474 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.308145 17277 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.309451 17351 consensus_queue.cc:1059] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:53.310662 17355 raft_consensus.cc:3055] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:53.311074 17355 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.310864 17277 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.311818 17466 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:53.312129 17355 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.314127 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.314940 17277 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:53.316381 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.317207 13534 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:53.319813 13689 raft_consensus.cc:2410] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:57:53.321022 17245 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.235306 16978 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:53.335950 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.378687 13534 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:57:53.384421 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:53.353672 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 109471us
I20260709 12:57:53.268380 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:53.434154 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 165884us
I20260709 12:57:53.434810 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:53.436484 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.437383 13618 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:53.438527 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:53.439567 13618 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:57:53.355830 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 72031.5us
I20260709 12:57:53.432520 14211 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:57:53.406322 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 227909us
I20260709 12:57:53.457626 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:53.447059 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:53.449097 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:53.461354 13615 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:53.448294 13506 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:53.463490 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:53.464269 13619 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.466514 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:53.467458 13614 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:57:53.472108 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.473263 13618 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:53.474668 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.476086 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:53.494171 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.494561 13612 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:53.494208 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:53.427243 17378 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:53.463304 17466 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.446950 17419 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:53.501113 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:53.335430 17296 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.392728 17043 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.403491 17355 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:53.490410 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.457295 16978 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:53.491636 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:53.514113 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.492425 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.494652 13618 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:53.337131 17217 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.468616 17329 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:57:53.496980 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 48014.6us
W20260709 12:57:53.498602 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.498775 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.499854 13619 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
W20260709 12:57:53.558009 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.569167 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.566718 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.577106 13537 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:53.478163 17063 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:53.430714 17360 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.328220 17400 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:53.503090 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 57157.6us
I20260709 12:57:53.561954 14241 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:53.580540 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.504711 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:53.507597 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 59432.5us
I20260709 12:57:53.511363 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:53.405962 17408 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.478613 17275 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:57:53.485591 17356 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:57:53.614976 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.516806 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.552223 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.500285 14246 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:53.576584 13543 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:53.577775 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.566808 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.567088 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.547284 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.551580 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.515432 17466 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.587743 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.588503 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:53.568245 17419 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:57:53.589258 14267 debug-util.cc:398] Leaking SignalData structure 0x7b08002e87e0 after lost signal to thread 12412
I20260709 12:57:53.615949 13537 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:53.624279 14267 debug-util.cc:398] Leaking SignalData structure 0x7b08002e8f20 after lost signal to thread 13475
I20260709 12:57:53.555680 17043 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:53.589993 17490 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 12:57:53.611968 17488 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:57:53.555133 17502 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:57:53.592530 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:53.593250 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.593392 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.672436 13538 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.673141 13536 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:53.516062 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.581776 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.599124 13612 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:57:53.594641 16978 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:53.613336 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:53.555542 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:53.619462 13619 raft_consensus.cc:2410] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:57:53.617394 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:53.676545 13618 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.612819 17329 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:53.620803 13611 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:53.684271 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:53.623399 17419 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:53.684983 13685 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
W20260709 12:57:53.625542 14267 debug-util.cc:398] Leaking SignalData structure 0x7b0800411520 after lost signal to thread 13568
W20260709 12:57:53.685856 14267 debug-util.cc:398] Leaking SignalData structure 0x7b08004115a0 after lost signal to thread 13569
I20260709 12:57:53.627939 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.686919 13539 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:57:53.687037 14267 debug-util.cc:398] Leaking SignalData structure 0x7b08002b7a20 after lost signal to thread 13643
W20260709 12:57:53.687366 14267 debug-util.cc:398] Leaking SignalData structure 0x7b080034eb60 after lost signal to thread 13644
I20260709 12:57:53.626588 13610 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:53.629593 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54065.2us
I20260709 12:57:53.630010 13543 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:53.688319 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.688699 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.689498 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.689844 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.554818 13441 catalog_manager.cc:5697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 2 to 7, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:53.516053 17289 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:53.506333 17426 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:53.696058 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.696583 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.615307 17491 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:53.651103 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 159325us
I20260709 12:57:53.638217 17474 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.580303 17063 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:53.641346 13506 leader_election.cc:400] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
W20260709 12:57:53.675503 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 93810us
I20260709 12:57:53.619728 13541 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:53.617013 17437 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.682651 14244 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:53.675036 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:53.683866 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 167863us
I20260709 12:57:53.730001 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.730897 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:53.630831 17498 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:57:53.691422 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.732235 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.623214 17466 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.623497 16960 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:53.692619 14267 debug-util.cc:398] Leaking SignalData structure 0x7b080040cd80 after lost signal to thread 13717
W20260709 12:57:53.737802 14267 debug-util.cc:398] Leaking SignalData structure 0x7b08003882c0 after lost signal to thread 14270
I20260709 12:57:53.693323 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:53.700313 17322 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.546981 17378 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:53.546854 17480 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.164s
W20260709 12:57:53.612305 17503 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:57:53.700289 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.668155 17296 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:53.515635 17497 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.176s
I20260709 12:57:53.668916 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:53.706432 13537 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:53.674357 17274 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.711908 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:53.714056 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.714699 13686 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:57:53.715456 13543 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:53.615108 17360 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.670432 17393 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:53.719905 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.734030 17289 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.721114 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.690312 17475 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:53.678453 17414 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:53.723757 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:53.800897 13542 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:53.803952 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:53.804850 13689 raft_consensus.cc:2410] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 12:57:53.700997 16978 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.724481 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:53.808670 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 84298.1us
I20260709 12:57:53.809378 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.673573 17465 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:53.634344 17353 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:57:53.733121 13508 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:53.694315 17419 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.723649 17474 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.740887 13692 raft_consensus.cc:2410] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:57:53.695245 14239 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:53.747054 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:53.813079 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 80033.1us
I20260709 12:57:53.834947 13685 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.833837 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:53.696177 17426 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:53.835873 13686 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:53.836238 17426 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:53.754441 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.837224 13686 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:53.755455 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.723400 17063 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.780607 13540 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:53.768666 13538 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
W20260709 12:57:53.754961 17347 consensus_meta.cc:287] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.723s	user 0.013s	sys 0.000s
I20260709 12:57:53.781239 13441 catalog_manager.cc:5697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 5 to 8, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 8 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:53.720527 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.793074 13618 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:53.854887 13685 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:53.855087 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.761266 17322 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.794759 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.860566 13619 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:53.796519 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:53.861595 14242 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:53.797905 14241 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:53.753728 17271 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:53.724370 13506 leader_election.cc:403] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:57:53.800436 17475 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:53.697196 17402 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:53.753777 17466 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:53.757493 17296 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:53.721608 17491 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:53.726521 17437 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.752566 16960 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:53.865075 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 234266us
I20260709 12:57:53.838016 17067 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:53.870507 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:53.905664 13618 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:57:53.795186 17274 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.753443 17329 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.876729 13441 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 5 to 10, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:53.911089 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:53.813138 17411 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:57:53.879571 14238 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:53.881099 17504 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.190s	user 0.000s	sys 0.002s
I20260709 12:57:53.881191 17296 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.882807 13689 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:57:53.806583 17393 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.891575 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:53.894886 13614 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:57:53.922075 13441 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:53.870126 17475 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.838824 17494 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:57:53.873543 17402 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:53.833632 17465 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:53.913847 13684 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:53.914063 13686 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:53.865692 17377 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10.
W20260709 12:57:53.906848 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 97559.2us
I20260709 12:57:53.882711 17217 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.916133 13618 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:53.915452 17067 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:53.810472 17289 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.862960 17347 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:53.883459 17271 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:53.833807 17414 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:53.926337 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 195508us
I20260709 12:57:53.936306 14238 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:53.940529 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:54.041162 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:54.041858 13692 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:53.942361 13441 catalog_manager.cc:5697] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 5 to 6, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 6 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:57:53.920310 14246 pending_rounds.cc:85] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 8
I20260709 12:57:54.045257 14246 pending_rounds.cc:107] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:57:54.045661 14246 raft_consensus.cc:2889] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:53.949702 17437 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:53.953441 17217 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:53.955590 13618 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:57:53.953518 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:53.957158 17507 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:57:53.927554 17411 consensus_queue.cc:1243] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:57:53.965924 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:53.967207 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.056689 13686 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.938648 17400 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.942934 17402 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:54.087860 13441 catalog_manager.cc:5697] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 4 to 5, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:57:53.960180 17296 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:53.954937 16978 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.047674 13541 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:57:54.051177 13535 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:54.044586 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:53.927151 17512 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:54.049420 17217 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:53.959371 17515 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:53.969941 17043 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:53.875135 17378 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.089964 13614 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:57:54.090456 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:54.117136 13614 pending_rounds.cc:85] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 7
I20260709 12:57:54.117448 13614 pending_rounds.cc:107] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:57:54.117726 13614 raft_consensus.cc:2889] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:54.097862 13692 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:54.120281 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:54.121121 13692 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:54.105947 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61441.1us
I20260709 12:57:54.055653 13684 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:57:54.113090 13441 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 8 to 9, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:54.122193 13659 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:54.124159 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:54.114758 17465 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:54.115850 17217 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:54.176218 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:54.176975 14244 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:54.103662 17474 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:54.183422 17063 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.179641 13686 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:54.188139 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:54.188351 17274 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:54.112609 17437 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:57:54.189484 17437 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:54.140049 17067 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.193306 17347 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.116377 17512 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.196130 17289 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.197506 17512 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.197747 17414 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.146469 17295 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.198495 17414 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:54.148592 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:54.125263 17515 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:54.211683 13441 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 1 to 4, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 4 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:54.164592 17480 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:57:54.176640 16960 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:54.245715 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.247139 13616 raft_consensus.cc:2410] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:57:54.200487 17360 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.201220 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:54.252561 13692 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:57:54.110208 17400 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
W20260709 12:57:54.204633 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.188464 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:54.207208 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:54.225041 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 101101us
W20260709 12:57:54.156674 17518 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:57:54.254356 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.255824 13441 catalog_manager.cc:5697] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 3 to 6, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 6 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:54.268071 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.268697 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:54.257113 14211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.001s	sys 0.021s
W20260709 12:57:54.258553 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.002s	sys 0.011s
I20260709 12:57:54.251771 13618 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:57:54.269531 13536 raft_consensus.cc:2393] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:54.261366 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:54.269359 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.270630 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.270833 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:54.271423 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:54.271878 13536 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:54.272802 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:54.273387 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.245771 17274 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:57:54.276268 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:54.276540 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.002s	sys 0.011s
I20260709 12:57:54.261194 13686 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:57:54.269400 14211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.001s	sys 0.021s
I20260709 12:57:54.282050 13540 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:57:54.285147 13542 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:54.284251 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.287982 13536 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:54.252931 17322 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:54.291909 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 561804us
I20260709 12:57:54.301807 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:54.303664 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:54.304426 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.304597 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:54.305176 13616 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:57:54.262953 16978 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:57:54.282789 17474 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:54.309978 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.312633 13618 raft_consensus.cc:2393] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:54.314411 17519 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:54.315100 14244 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:54.227319 17271 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:57:54.322268 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:54.323633 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.282855 17477 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:54.326411 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.327039 13543 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:57:54.191443 17437 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.327891 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:54.307968 13441 catalog_manager.cc:5697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 5 to 6, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 6 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:57:54.329999 17530 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:54.330693 17400 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:57:54.331025 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.332193 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:54.333240 17402 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.334309 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:57:54.335398 17063 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.335587 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:54.336093 17477 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.336243 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.336493 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.337690 17390 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:54.338171 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.338696 17515 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.397629 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.398746 13539 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:54.339740 17393 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:54.339668 17510 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:54.340310 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:54.342593 17514 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:57:54.336087 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.347736 17474 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:54.311846 16817 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:54.352447 17416 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:54.353682 17172 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:57:54.358462 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56766.1us
I20260709 12:57:54.289532 17465 raft_consensus.cc:2764] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:57:54.286151 17274 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:54.359910 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.411593 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:54.411816 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:54.412669 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.413775 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:54.384003 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.382191 13613 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:54.388131 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:54.389881 13582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.031s	sys 0.039s
I20260709 12:57:54.390192 13536 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:54.383322 13441 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:57:54.415827 13582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.031s	sys 0.039s
W20260709 12:57:54.382365 13693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.017s	sys 0.036s
I20260709 12:57:54.393889 13542 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:57:54.395222 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 59735.2us
I20260709 12:57:54.399698 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.402299 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.404011 13691 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:54.405732 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.405892 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:54.359089 17519 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:54.415661 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:54.490576 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 76926.9us
I20260709 12:57:54.492101 13613 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:57:54.490296 17063 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
W20260709 12:57:54.493249 17522 log.cc:927] Time spent T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:57:54.493513 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.398142 17530 consensus_queue.cc:1243] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer 0f34cda059124f839711dbef61bb2c28 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:57:54.494192 13613 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:57:54.496784 13618 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:54.494686 13534 raft_consensus.cc:2376] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10. Re-sending same reply.
I20260709 12:57:54.492476 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:54.497514 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:54.416576 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 693888us
I20260709 12:57:54.498811 13613 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:54.499173 13658 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:54.499387 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:54.362016 17516 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:57:54.495375 13693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.018s	sys 0.043s
W20260709 12:57:54.448263 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.395483 17477 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:54.492165 17355 consensus_meta.cc:287] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.937s	user 0.011s	sys 0.002s
I20260709 12:57:54.505010 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:54.409344 17419 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.546s	user 0.001s	sys 0.014s
W20260709 12:57:54.490869 17426 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.654s	user 0.004s	sys 0.008s
I20260709 12:57:54.512250 14240 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:54.402037 17067 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:54.407140 17416 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:54.502190 17063 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:54.523340 17296 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:54.491530 17474 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.546098 17525 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:54.511715 17465 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:57:54.507229 17378 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:57:54.410131 16817 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.480460 17402 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:54.417631 17274 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:54.504618 17516 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.545037 17296 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:54.523128 17000 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:54.541976 17393 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:54.557364 13692 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:57:54.544884 17217 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.509920 17480 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 12:57:54.573956 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 70807us
I20260709 12:57:54.582065 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:54.583178 13506 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:54.584211 13506 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:54.546643 17525 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.573069 17516 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:54.507503 17530 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:57:54.537684 17400 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:54.589591 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:54.541024 17414 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:54.539666 17475 consensus_meta.cc:287] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.598s	user 0.007s	sys 0.005s
I20260709 12:57:54.522637 17426 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.567811 17296 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:54.570804 17000 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:54.521108 17419 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.590118 17347 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:54.623004 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 721) took 2936 ms. Trace:
I20260709 12:57:54.634209 13535 rpcz_store.cc:276] 0709 12:57:51.686884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.066135 (+1379251us) service_pool.cc:224] Handling call
0709 12:57:53.066669 (+   534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.102358 (+1035689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:54.622676 (+520318us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54000b90d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:54.622927 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2263703936}
I20260709 12:57:54.602098 17400 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.600755 17377 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:54.635829 17377 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.636261 17377 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:54.636379 17510 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:54.636997 17400 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.636963 17510 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.628592 13612 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:54.639106 17510 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:54.608404 17475 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.640158 17347 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:54.640949 17475 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:54.641081 17347 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.641706 13611 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:54.612586 17000 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:54.611756 17275 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:54.644635 17275 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.644996 17275 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:54.645788 17525 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.639118 17426 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.650005 16817 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:54.653231 17217 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:54.591838 17067 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:57:54.662850 17536 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:57:54.612231 17356 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:57:54.604856 17414 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:54.667904 13509 leader_election.cc:336] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.657s (SENT)
I20260709 12:57:54.590632 17416 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.681273 13693 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:54.678846 13685 raft_consensus.cc:2393] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:54.680127 17172 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:57:54.683547 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:54.675132 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 176001us
I20260709 12:57:54.684779 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:54.666958 17217 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:54.690220 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:54.700577 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:54.703040 17419 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:54.668728 17067 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:54.707137 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:54.715018 17541 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:54.715694 17541 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.709810 17402 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:54.716977 17402 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:54.718267 17402 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.719282 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.720057 13618 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:54.720700 17516 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.720463 17067 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.721311 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.721987 13618 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:57:54.723110 17516 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:54.723228 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:54.716085 17541 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:54.699139 17474 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:54.688616 17356 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:54.681874 17510 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 12:57:54.732509 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.708988 17355 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:54.738714 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:54.727993 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 123555us
I20260709 12:57:54.733441 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:54.746846 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:54.687633 17172 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.723994 17465 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:54.751765 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:54.752526 14241 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:57:54.752544 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:54.753533 13618 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:57:54.706185 17217 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.761207 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:54.761373 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:54.762108 13535 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:54.762457 13535 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:54.764163 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.764992 13539 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:54.749827 17355 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.766057 13534 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:57:54.757311 17516 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:54.744539 17474 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:54.768517 14239 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:54.755424 17465 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.770893 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.772701 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.773623 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:54.774943 13535 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.757972 16817 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:57:54.788854 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.778614 13618 raft_consensus.cc:2368] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:57:54.789574 13534 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:57:54.742523 17356 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:54.778954 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.793903 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.773489 17516 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.776054 17465 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:54.794552 13534 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:54.791250 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:54.796648 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 51131.4us
I20260709 12:57:54.799873 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:54.778702 16817 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:54.807756 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 1054) took 1303 ms. Trace:
I20260709 12:57:54.788113 17172 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:54.808240 14240 rpcz_store.cc:276] 0709 12:57:53.504082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.504364 (+   282us) service_pool.cc:224] Handling call
0709 12:57:54.807592 (+1303228us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b54005b18d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:54.807704 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138330880}
I20260709 12:57:54.875873 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 541) took 1192 ms. Trace:
I20260709 12:57:54.838632 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.073649 13689 rpcz_store.cc:276] 0709 12:57:53.683304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.811527 (+128223us) service_pool.cc:224] Handling call
0709 12:57:53.811766 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:53.921172 (+109406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:54.875552 (+954380us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b54002ec7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:54.875806 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162284928}
I20260709 12:57:55.074651 13618 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:55.075484 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.076218 13689 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:54.814778 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:54.817862 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:54.888072 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:54.808869 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:54.991976 13619 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:57:55.079641 13619 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:54.977274 13688 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:57:54.968012 13534 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:57:54.992532 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 643) took 1740 ms. Trace:
I20260709 12:57:55.082015 13610 rpcz_store.cc:276] 0709 12:57:53.252207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.253366 (+  1159us) service_pool.cc:224] Handling call
0709 12:57:53.253663 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:53.698055 (+444392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:54.992277 (+1294222us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54001293d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:54.992471 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835146112}
I20260709 12:57:54.904628 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 2992) took 1015 ms. Trace:
I20260709 12:57:55.083909 13685 rpcz_store.cc:276] 0709 12:57:53.889042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.891319 (+  2277us) service_pool.cc:224] Handling call
0709 12:57:54.904375 (+1013056us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b540031e054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:54.904552 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1740368128}
W20260709 12:57:54.866731 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:55.085635 13509 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.815s (SENT) [suppressed 31 similar messages]
I20260709 12:57:54.809180 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:55.089639 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:55.091197 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:55.069321 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:55.095821 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:54.797922 17355 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:54.798382 17516 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:54.981079 17511 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.044s	user 0.002s	sys 0.004s
I20260709 12:57:55.101239 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1696) took 1670 ms. Trace:
I20260709 12:57:55.117269 13611 rpcz_store.cc:276] 0709 12:57:53.431208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.684924 (+253716us) service_pool.cc:224] Handling call
0709 12:57:55.101050 (+1416126us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b540023c8d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:55.101182 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1810153216}
I20260709 12:57:54.806919 16817 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.099807 13619 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:54.982463 17550 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 12:57:54.907948 17545 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 12:57:55.126187 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.113579 13536 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:55.077209 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:55.100320 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 291573us
I20260709 12:57:55.128090 13689 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.129726 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.130920 13584 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:54.895617 17245 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:55.131603 17245 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:55.132498 17245 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.133631 17245 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.135247 13536 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:55.117000 14211 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:55.135850 13689 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:55.136409 17329 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:55.136776 17329 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.137583 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:54.850672 16960 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.276s
I20260709 12:57:54.801424 17000 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.895359 17275 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.094502 13543 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:54.842197 17543 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:57:55.068681 17238 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:54.803170 17531 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.323s
I20260709 12:57:54.961115 17414 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:55.003787 17437 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:55.162626 17472 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:55.181207 17472 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.121078 13506 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.184173 17275 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:55.086599 17063 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:55.184957 17063 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.142997 17415 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:57:55.187371 17063 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:55.067061 17356 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:55.122424 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.188980 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.189605 13610 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:55.144655 17543 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.191154 17543 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:57:55.122887 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.107748 17473 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.006161 17390 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:55.192260 13685 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:57:55.192615 17390 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:55.193347 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.193714 17390 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.194653 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.194991 17390 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.137174 17329 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:55.195892 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.209606 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:55.279798 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.136874 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.118896 13618 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:55.139281 13619 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:57:55.228516 17402 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:54.929873 17477 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.983692 17377 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.304035 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:54.896646 17274 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.163079 13686 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.139643 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:55.143803 13509 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.926s (SENT)
W20260709 12:57:55.164733 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.314021 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.164783 17238 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.319428 17238 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:55.319656 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 175879us
I20260709 12:57:55.321382 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.165720 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1057) took 1590 ms. Trace:
W20260709 12:57:55.168649 17555 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:57:55.169167 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.172330 17393 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:55.323771 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2998) took 1424 ms. Trace:
I20260709 12:57:55.324229 13686 rpcz_store.cc:276] 0709 12:57:53.898970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.323206 (+424236us) service_pool.cc:224] Handling call
0709 12:57:54.359423 (+ 36217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:55.313620 (+954197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.323527 (+  9907us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54002986d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.323711 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847627136}
I20260709 12:57:55.324285 17393 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.323819 13611 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:55.326581 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.327260 13611 raft_consensus.cc:2393] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:57:55.142377 17542 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:55.328316 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.142547 17465 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.328925 13686 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.331286 17465 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:55.178742 17541 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:54.842183 17474 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:55.333472 17544 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.565s	user 0.003s	sys 0.008s
I20260709 12:57:55.333045 16817 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.180198 17437 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.334666 17541 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:55.335578 16817 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:54.992820 17217 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:55.336529 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 740) took 1593 ms. Trace:
I20260709 12:57:55.285200 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.285795 13617 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.337292 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.337630 13538 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:57:55.337805 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.338382 13618 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:57:55.338449 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.286062 13543 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.141273 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:55.339347 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1490) took 1074 ms. Trace:
W20260709 12:57:55.287284 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.289835 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.336971 13541 rpcz_store.cc:276] 0709 12:57:53.742669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.742813 (+   144us) service_pool.cc:224] Handling call
0709 12:57:53.743107 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.101947 (+358840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.336275 (+1234328us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b540028d554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.336465 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699371392}
I20260709 12:57:55.340636 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.341396 13690 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:55.341225 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:55.342273 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.343410 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.293030 17472 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:55.293746 17275 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:55.293876 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 172867us
I20260709 12:57:55.291078 17402 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.339424 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.349751 17402 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:55.296056 13619 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:55.309844 17477 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:55.122797 17556 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:55.314455 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 174942us
I20260709 12:57:55.315361 13610 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:55.315941 17473 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:55.317342 17274 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:55.328096 13685 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:55.322232 14245 rpcz_store.cc:276] 0709 12:57:53.575261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.576912 (+  1651us) service_pool.cc:224] Handling call
0709 12:57:53.615586 (+ 38674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:53.624029 (+  8443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:53.624058 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:53.624495 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 12:57:53.735961 (+111466us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:53.736007 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:53.736076 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:54.980524 (+1244448us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:55.165381 (+184857us) raft_consensus.cc:1685] finished
0709 12:57:55.165406 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.165654 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8058,"raft.queue_time_us":9672,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7911,"spinlock_wait_cycles":9950464,"thread_start_us":51798,"threads_started":1,"wal-append.queue_time_us":203642}
I20260709 12:57:55.339205 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.340924 13689 raft_consensus.cc:2410] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:57:55.363281 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.364475 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.365155 13689 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.351136 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1693) took 1920 ms. Trace:
I20260709 12:57:55.365896 13619 rpcz_store.cc:276] 0709 12:57:53.430452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.861293 (+430841us) service_pool.cc:224] Handling call
0709 12:57:53.873132 (+ 11839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.991848 (+1118716us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54003dc5a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 12:57:55.350846 (+358998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.351084 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422755072}
I20260709 12:57:55.367766 13619 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.177894 17000 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:55.339805 13538 rpcz_store.cc:276] 0709 12:57:54.265209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.266151 (+   942us) service_pool.cc:224] Handling call
0709 12:57:55.337388 (+1071237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:55.338071 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.339173 (+  1102us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540028d550. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:55.339291 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2462750080}
I20260709 12:57:55.373056 13538 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:57:55.353037 13690 raft_consensus.cc:2393] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:55.374356 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.374990 13690 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:55.375036 13616 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:57:55.375744 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.351573 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.315541 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:55.377306 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 61869.8us
I20260709 12:57:55.377467 13616 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:55.378067 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.378571 13616 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.379145 13658 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:55.356134 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.380333 13616 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:55.336541 17437 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:55.363389 13618 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 5. Current term is 6. Ops: []
I20260709 12:57:55.305833 17377 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:55.327821 17393 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:55.354409 17347 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:55.412372 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 535) took 1731 ms. Trace:
I20260709 12:57:55.412499 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.413168 13618 raft_consensus.cc:2376] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8. Re-sending same reply.
W20260709 12:57:55.414004 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.414170 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.412905 13687 rpcz_store.cc:276] 0709 12:57:53.681221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.681385 (+   164us) service_pool.cc:224] Handling call
0709 12:57:53.708655 (+ 27270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:53.723969 (+ 15314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:53.724001 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:54.919053 (+1195052us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b2c0006a7c8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:57:54.928711 (+  9658us) log.cc:844] Serialized 26 byte log entry
0709 12:57:55.125418 (+196707us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:55.125490 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.130317 (+  4827us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:55.412031 (+281714us) raft_consensus.cc:1685] finished
0709 12:57:55.412050 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.412306 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":346,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":9581,"spinlock_wait_cycles":2827648,"thread_start_us":141016,"threads_started":1,"wal-append.queue_time_us":426048}
I20260709 12:57:55.415210 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:55.361608 17276 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:57:55.382663 13690 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:55.381831 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.388763 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.415885 13618 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:57:55.404022 17465 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:55.398435 17477 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:55.382007 17217 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.364055 17557 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:57:55.359769 13617 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:55.417255 13686 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:55.438094 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.438163 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.438763 13686 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:55.439076 13617 raft_consensus.cc:2368] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:57:55.422971 13619 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:57:55.424131 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 537) took 1742 ms. Trace:
I20260709 12:57:55.439894 13684 rpcz_store.cc:276] 0709 12:57:53.681856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.681989 (+   133us) service_pool.cc:224] Handling call
0709 12:57:53.707457 (+ 25468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.121660 (+414203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.423853 (+1302193us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400159d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.424066 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474265344}
I20260709 12:57:55.440763 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.441313 13617 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:57:55.441789 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 652) took 1721 ms. Trace:
I20260709 12:57:55.434067 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.442313 13614 rpcz_store.cc:276] 0709 12:57:53.720439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.925197 (+204758us) service_pool.cc:224] Handling call
0709 12:57:53.941130 (+ 15933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.118172 (+177042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.441512 (+1323340us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540013c9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.441715 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267683072}
I20260709 12:57:55.444471 13690 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:57:55.446683 13690 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:57:55.441309 13684 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:57:55.449869 13684 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
W20260709 12:57:55.428452 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46731.8us
I20260709 12:57:55.430596 14238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1060) took 1124 ms. Trace:
I20260709 12:57:55.451119 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:55.429311 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.453109 13618 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:55.454370 13618 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:57:55.454551 13617 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.450832 14238 rpcz_store.cc:276] 0709 12:57:54.305761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.305915 (+   154us) service_pool.cc:224] Handling call
0709 12:57:54.316307 (+ 10392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:54.576905 (+260598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:54.576940 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:54.591434 (+ 14494us) log.cc:844] Serialized 26 byte log entry
0709 12:57:54.742391 (+150957us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:54.743433 (+  1042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:54.981736 (+238303us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:55.430208 (+448472us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400497ad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.430275 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:55.430333 (+    58us) raft_consensus.cc:1685] finished
0709 12:57:55.430349 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.430519 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":379,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":9007,"spinlock_wait_cycles":649226880,"thread_start_us":113783,"threads_started":1,"wal-append.queue_time_us":169940}
I20260709 12:57:55.429281 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.455698 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.433980 13689 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:55.388177 17274 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.433984 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.485097 13506 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:55.458811 13658 leader_election.cc:400] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:57:55.488603 13538 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:57:55.459865 13689 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:55.460371 13684 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.462337 17217 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.423945 17393 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:55.430339 17465 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.430771 17473 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:55.476565 17377 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:57:55.457041 17560 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:57:55.460289 17347 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:57:55.454205 17437 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.478582 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.490716 17548 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:55.488700 13658 leader_election.cc:403] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:57:55.511494 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52799.3us
I20260709 12:57:55.512679 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:55.512902 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:55.513314 14242 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:55.514431 13582 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.514765 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:55.461678 13618 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:57:55.363955 17556 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:55.491309 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 189678us
I20260709 12:57:55.491619 13686 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8. Ops: []
I20260709 12:57:55.484719 17402 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.494370 14238 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.496588 13687 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.499643 13689 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:57:55.471364 17329 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.501475 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:55.460711 17542 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:55.509912 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:55.510548 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:55.515718 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:55.517390 13582 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:57:55.518522 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.519129 13619 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:57:55.534979 14241 raft_consensus.cc:2410] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:57:55.498299 17465 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.484974 17414 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:57:55.520428 17477 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:55.498253 17393 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.529122 13686 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:57:55.530172 14237 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:57:55.532359 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.532191 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:55.519958 17274 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:55.533207 14245 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:55.540781 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:55.557932 14242 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:55.541934 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.487974 17067 consensus_queue.cc:1059] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:57:55.558949 14241 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:55.539768 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:55.565451 13659 leader_election.cc:400] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:57:55.565204 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.566507 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:55.556227 14238 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:55.570454 13687 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:57:55.572194 14240 raft_consensus.cc:2368] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11.
I20260709 12:57:55.525779 17473 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:57:55.575340 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:55.576241 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.544152 17548 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:55.577622 17063 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:55.578166 17063 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.578609 17063 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:55.578063 17548 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.583864 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:55.543776 17477 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:55.592810 13686 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:57:55.593323 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:55.592031 17543 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.598389 17543 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:55.604276 17347 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:55.570258 17542 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:55.581444 17378 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:55.582757 17473 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.526266 17516 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.587419 17000 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:57:55.588541 17217 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:55.623090 13686 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:55.623965 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.607136 17477 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:55.580404 17419 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:57:55.608978 17512 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.605553 17497 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:55.608206 17360 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:57:55.655429 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 658) took 1310 ms. Trace:
I20260709 12:57:55.623780 17473 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:55.661079 13686 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:57:55.661561 13686 pending_rounds.cc:85] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:57:55.661775 13686 pending_rounds.cc:107] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:57:55.661996 13686 raft_consensus.cc:2889] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:55.661899 17542 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.664933 17556 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:55.665577 17296 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.666643 17296 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:55.630061 17543 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:55.659288 13613 rpcz_store.cc:276] 0709 12:57:54.344972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.507490 (+162518us) service_pool.cc:224] Handling call
0709 12:57:54.507780 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.575295 (+ 67515us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:57:54.677610 (+102315us) log.cc:844] Serialized 14 byte log entry
0709 12:57:54.677873 (+   263us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:54.677939 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:54.677991 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.655155 (+977164us) spinlock_profiling.cc:243] Waited 67.3 ms on lock 0x7b540013a6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.655217 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.655350 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8074,"prepare.queue_time_us":180808,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5449,"raft.queue_time_us":806,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4278,"spinlock_wait_cycles":154761472,"thread_start_us":80248,"threads_started":1}
I20260709 12:57:55.659761 17497 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:55.635046 16978 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:57:55.672233 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 105822us
I20260709 12:57:55.637146 17378 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:55.644855 17515 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.676858 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:55.689237 14239 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:55.689926 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.692411 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:55.692454 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 110669us
I20260709 12:57:55.694139 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:55.695830 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 130679us
I20260709 12:57:55.676911 16978 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:55.697582 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.692806 17512 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:55.698886 17067 consensus_queue.cc:1243] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:57:55.704226 17560 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:55.701304 16978 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:57:55.712075 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.725163 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:55.699440 17556 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:55.736398 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 99704.4us
I20260709 12:57:55.708967 17512 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:55.712199 17419 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:55.748003 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.755385 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.756214 13685 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
W20260709 12:57:55.758889 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.733755 17329 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:55.743973 17276 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:57:55.750177 17512 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:55.744488 17393 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:55.768246 13685 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:57:55.784948 13691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45312 (request call id 3003) took 1883 ms (client timeout 2312 ms). Trace:
W20260709 12:57:55.785616 13691 rpcz_store.cc:269] 0709 12:57:53.901194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.187853 (+286659us) service_pool.cc:224] Handling call
0709 12:57:55.784695 (+1596842us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540042d1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:55.784865 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494287616}
I20260709 12:57:55.753093 17582 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:57:55.788578 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 94521.1us
I20260709 12:57:55.740890 17556 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.733929 17378 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:55.790390 17573 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.269s	user 0.002s	sys 0.000s
I20260709 12:57:55.788177 17419 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:57:55.804991 17579 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:57:55.797425 17516 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:55.812137 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 87038.2us
I20260709 12:57:55.829629 17556 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:55.824604 17574 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:57:55.832119 13535 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.833094 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:55.836601 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:55.837208 14242 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:55.708824 17560 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.853652 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 745) took 1092 ms. Trace:
I20260709 12:57:55.854097 13542 rpcz_store.cc:276] 0709 12:57:54.761381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.761836 (+   455us) service_pool.cc:224] Handling call
0709 12:57:54.765780 (+  3944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.803154 (+ 37374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:54.803187 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:54.803239 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:54.803278 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.853413 (+1050135us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b54000ab254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.853460 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.853594 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85506944}
W20260709 12:57:55.815088 17578 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:57:55.815518 17557 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:55.817342 17408 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:55.824563 17276 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:55.827235 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:55.863217 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:55.705724 17497 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:55.828394 17587 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:55.842605 13535 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:57:55.840130 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:55.903146 13535 pending_rounds.cc:85] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 8
I20260709 12:57:55.903596 13535 pending_rounds.cc:107] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 9.9
W20260709 12:57:55.903471 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 63406us
I20260709 12:57:55.904024 13535 raft_consensus.cc:2889] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:55.904765 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 741) took 1152 ms. Trace:
I20260709 12:57:55.905113 13535 rpcz_store.cc:276] 0709 12:57:54.751972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.752107 (+   135us) service_pool.cc:224] Handling call
0709 12:57:54.762004 (+  9897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:55.831734 (+1069730us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5800625ad0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 12:57:55.904530 (+ 72796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.904715 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2424992640}
I20260709 12:57:55.787446 17581 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:57:55.858289 13612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1704) took 2425 ms (client timeout 3059 ms). Trace:
I20260709 12:57:55.858381 14242 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:55.860615 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:55.878664 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.836483 17588 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:55.906325 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:55.914076 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 53572us
I20260709 12:57:55.911445 17589 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:57:55.906327 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:55.923102 13535 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:55.904572 17354 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:57:55.910722 13612 rpcz_store.cc:269] 0709 12:57:53.433164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.755249 (+322085us) service_pool.cc:224] Handling call
0709 12:57:55.858088 (+2102839us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b5400332054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:55.858232 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818481536}
I20260709 12:57:55.914829 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:55.928058 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:55.929517 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:55.931450 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.931802 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.932485 13612 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:55.916167 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:55.861356 17515 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:57:55.921722 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:55.859643 17274 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:57:55.859790 17557 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:55.913460 17381 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:57:55.907003 17587 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.942430 13542 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:55.944614 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:55.943629 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.960465 13535 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:57:55.951702 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.960717 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:55.950237 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 661) took 1447 ms. Trace:
I20260709 12:57:55.961259 13542 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:55.961341 13611 rpcz_store.cc:276] 0709 12:57:54.502891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.357024 (+854133us) service_pool.cc:224] Handling call
0709 12:57:55.431547 (+ 74523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:55.432088 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:55.432114 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:55.433772 (+  1658us) log.cc:844] Serialized 26 byte log entry
0709 12:57:55.608354 (+174582us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:55.608392 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.608462 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:55.949939 (+341477us) raft_consensus.cc:1685] finished
0709 12:57:55.949968 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.950156 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4138,"thread_start_us":146893,"threads_started":1,"wal-append.queue_time_us":242868}
I20260709 12:57:55.926906 17276 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:55.952886 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.955312 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 3016) took 1011 ms. Trace:
I20260709 12:57:55.863060 17411 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:57:55.935803 17354 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.908017 17580 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:57:55.977358 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:55.977537 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 158163us
W20260709 12:57:55.977990 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.978202 13618 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:57:55.943382 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:55.926353 17419 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:55.964978 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.939548 17415 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:57:55.935460 17581 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:55.967195 13690 rpcz_store.cc:276] 0709 12:57:54.944229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.447624 (+503395us) service_pool.cc:224] Handling call
0709 12:57:55.448406 (+   782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:55.448906 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:55.448948 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:55.461749 (+ 12801us) log.cc:844] Serialized 26 byte log entry
0709 12:57:55.637832 (+176083us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:55.637886 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.637971 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:55.955102 (+317131us) raft_consensus.cc:1685] finished
0709 12:57:55.955121 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.955254 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":21788,"thread_start_us":175644,"threads_started":1,"wal-append.queue_time_us":234574}
I20260709 12:57:55.957361 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 629) took 2253 ms. Trace:
I20260709 12:57:55.963856 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:55.951372 17583 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:55.940112 17000 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:55.939908 17408 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:57:55.935349 17067 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:57:55.956377 17557 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.913290 17543 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:57:55.925474 17497 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:55.981128 13542 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:57:55.935986 17548 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:57:56.028644 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:55.925264 17556 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:57:55.983165 13539 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
W20260709 12:57:55.984095 17275 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.640s	user 0.000s	sys 0.020s
I20260709 12:57:55.984403 17276 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:55.988281 16978 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:55.952767 13509 leader_election.cc:400] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:57:56.047673 13509 leader_election.cc:403] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:55.992578 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:55.994055 14246 rpcz_store.cc:276] 0709 12:57:53.691886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.785309 (+ 93423us) service_pool.cc:224] Handling call
0709 12:57:53.785612 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:54.046292 (+260680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:54.046325 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:54.046898 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:57:54.126954 (+ 80056us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:57:54.127013 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:55.944637 (+1817624us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400681fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:55.944686 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:55.944727 (+    41us) raft_consensus.cc:1685] finished
0709 12:57:55.944739 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:55.944946 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309152512,"thread_start_us":79647,"threads_started":1,"wal-append.queue_time_us":264163,"wal-append.run_cpu_time_us":1675,"wal-append.run_wall_time_us":1020443}
I20260709 12:57:55.994568 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:55.989264 13535 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:56.049762 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.050390 13535 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:56.050851 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 102105us
I20260709 12:57:55.926523 17582 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:57:56.002220 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:55.976845 17354 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:56.014420 17000 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:56.013447 17411 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:56.000065 17415 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:55.999791 17581 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:55.943990 17584 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 12:57:56.031096 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:56.031459 17408 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:55.963608 17587 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:56.066934 14246 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEARNER]: Advancing to term 10
I20260709 12:57:56.054883 17557 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:56.055230 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:56.071395 14242 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:57:56.055400 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:56.073135 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:56.000283 17360 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:57:56.030484 17542 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:56.014940 17583 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:56.058406 17576 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:57:56.051291 17275 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.059544 17415 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:57:56.103492 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 50309.6us
I20260709 12:57:55.958490 17353 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:57:56.055060 17296 raft_consensus.cc:3055] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:57:56.074767 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:56.078063 14209 leader_election.cc:336] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 2.789s (SENT)
W20260709 12:57:56.105698 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:56.084744 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:56.081787 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:56.055826 17521 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:57:56.061779 17581 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:56.109512 14209 connection.cc:256] RPC call timeout handler was delayed by -1.3138554680008383s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
I20260709 12:57:56.063290 17512 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.093246 17582 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.097841 17594 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:56.098156 17000 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.061393 17063 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:56.070080 17465 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 12:57:56.058008 17067 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:57:56.110450 13690 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:56.095419 17583 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:56.061430 17411 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.107308 17296 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:56.067473 17419 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.126612 13686 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 12. Ops: []
I20260709 12:57:56.083379 17591 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:57:56.115716 14246 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:57:56.095428 17542 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.128522 17560 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.119314 17465 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.118077 17594 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.117537 17600 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:57:56.117547 17513 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:56.109390 17274 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:57:56.117513 17604 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:56.136401 17296 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.156445 17513 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.144264 17473 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:56.112166 17276 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.158706 17513 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:56.175510 17519 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.403s	user 0.025s	sys 0.000s
I20260709 12:57:56.159946 17604 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.182845 17604 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.184894 17587 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.188053 17587 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.194387 17581 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.178025 17588 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:56.197081 17581 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.199594 17584 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:57:56.200217 17584 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:56.201555 17583 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.204005 17580 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:57:56.204205 17577 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.459s	user 0.003s	sys 0.000s
I20260709 12:57:56.204499 17580 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:57:56.206260 17602 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:57:56.207237 17594 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:56.106590 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:56.244937 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 138143us
W20260709 12:57:56.247725 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 165876us
I20260709 12:57:56.073412 13535 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:57:56.254232 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.255569 13535 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:56.257066 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.257862 13535 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:56.259187 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.260591 13535 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:56.261999 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.262823 13535 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:56.265784 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.267084 13541 raft_consensus.cc:2368] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11.
I20260709 12:57:56.268323 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.269909 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.271138 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.281206 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.282051 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.282821 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.283015 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1614) took 1467 ms. Trace:
I20260709 12:57:56.283504 13540 rpcz_store.cc:276] 0709 12:57:54.815308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.843249 (+ 27941us) service_pool.cc:224] Handling call
0709 12:57:56.281519 (+1438270us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:57:56.282945 (+  1426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4448640}
I20260709 12:57:56.291021 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:56.285277 13539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1612) took 1476 ms (client timeout 1781 ms). Trace:
W20260709 12:57:56.292119 13539 rpcz_store.cc:269] 0709 12:57:54.808542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.814411 (+  5869us) service_pool.cc:224] Handling call
0709 12:57:56.279855 (+1465444us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54002ba7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:56.285113 (+  5258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083032832}
W20260709 12:57:56.293079 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:56.294643 13540 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:56.286037 13536 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:57:56.301388 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:56.284500 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.304193 13543 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: []
I20260709 12:57:56.305531 13539 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:57:56.291311 13537 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:56.119009 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:56.317867 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:56.336510 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:56.336959 13615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56784 (request call id 1708) took 2902 ms (client timeout 2001 ms). Trace:
I20260709 12:57:56.337757 13685 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:57:56.337524 13615 rpcz_store.cc:269] 0709 12:57:53.434108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.515090 (+ 80982us) service_pool.cc:224] Handling call
0709 12:57:56.336413 (+2821323us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b5400209c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:56.336645 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1571483520}
I20260709 12:57:56.339295 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:56.319211 13691 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:56.322952 16978 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:56.327095 17353 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:57:56.329447 17296 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:56.341817 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1557) took 1365 ms. Trace:
I20260709 12:57:56.342273 13684 rpcz_store.cc:276] 0709 12:57:54.976144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.457429 (+481285us) service_pool.cc:224] Handling call
0709 12:57:55.460189 (+  2760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:55.490320 (+ 30131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:56.341263 (+850943us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b54002972d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:56.341552 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5610880}
I20260709 12:57:56.171202 17329 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:56.344660 17329 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.345577 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:56.146708 17408 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.348135 17067 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.348284 17353 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:56.349094 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2996) took 2450 ms. Trace:
I20260709 12:57:56.349143 17353 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.332738 13536 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:57:56.323496 17275 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.335601 13690 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:57:56.349705 13688 rpcz_store.cc:276] 0709 12:57:53.898305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.260045 (+361740us) service_pool.cc:224] Handling call
0709 12:57:54.303315 (+ 43270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:55.080281 (+776966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:56.348464 (+1268183us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400159854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:56.348845 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549391616}
I20260709 12:57:56.362187 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:56.367784 17513 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:56.368609 17513 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.370961 17513 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.373196 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.379041 13541 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:57:56.377628 13539 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:56.323017 17000 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:56.390023 17512 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:56.381408 13539 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:56.393183 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:56.395792 13543 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:57:56.396071 13688 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:57:56.397807 13688 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:57:56.399196 13539 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:56.400153 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:56.400664 13539 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:56.402962 17560 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:56.404076 17600 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:56.404887 17600 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.405480 13541 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:57:56.405283 17000 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.408201 17603 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:56.409394 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:56.410301 14241 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:56.411356 17353 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:56.181227 17519 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.417s	user 0.025s	sys 0.000s
W20260709 12:57:56.413071 14243 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 1055) took 2904 ms (client timeout 2671 ms). Trace:
W20260709 12:57:56.413584 14243 rpcz_store.cc:269] 0709 12:57:53.508277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:53.508466 (+   189us) service_pool.cc:224] Handling call
0709 12:57:56.412545 (+2904079us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540053acd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:56.412815 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454078208}
I20260709 12:57:56.419579 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.421447 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.422242 13541 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:56.422065 17513 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:56.423735 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:56.397534 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:56.428332 13619 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:56.455265 13539 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:57:56.380941 17329 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.463330 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.159504 17477 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.465467 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:56.464004 13543 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
W20260709 12:57:56.470501 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51177.2us
I20260709 12:57:56.467443 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:56.473322 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:56.486724 17586 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:56.486871 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:56.488204 14240 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:56.487864 17586 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.490440 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:56.490851 17557 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:57:56.495286 13615 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:57:56.496507 17000 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:56.524430 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 223315us
I20260709 12:57:56.526131 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:57:56.520376 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 147449us
I20260709 12:57:56.538949 13539 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:56.528399 13690 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
W20260709 12:57:56.525660 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 128559us
W20260709 12:57:56.516580 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55027.8us
I20260709 12:57:56.534718 13541 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:57:56.530791 13535 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:56.513260 13613 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:57:56.505647 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2792) took 1844 ms. Trace:
I20260709 12:57:56.512751 13617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2816) took 1386 ms. Trace:
I20260709 12:57:56.512996 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:56.546078 13610 rpcz_store.cc:276] 0709 12:57:54.661429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.356146 (+694717us) service_pool.cc:224] Handling call
0709 12:57:55.356433 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:55.416107 (+ 59674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:55.416136 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:57:55.416173 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:55.416206 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:56.505010 (+1088804us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b540012b1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:56.505105 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:56.505391 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136811776}
I20260709 12:57:56.508833 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1619) took 1406 ms. Trace:
I20260709 12:57:56.522957 17465 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.524096 17063 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:56.509828 17411 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.554778 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1521) took 1034 ms. Trace:
I20260709 12:57:56.550941 13617 rpcz_store.cc:276] 0709 12:57:55.126696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.442247 (+315551us) service_pool.cc:224] Handling call
0709 12:57:55.442551 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:55.455318 (+ 12767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:56.512175 (+1056857us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b540013bfd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:56.512534 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27078016}
I20260709 12:57:56.520820 17554 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:56.505430 17607 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:57:56.525022 17329 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.509681 17582 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.572781 13543 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
I20260709 12:57:56.564816 13535 rpcz_store.cc:276] 0709 12:57:55.511607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:56.261452 (+749845us) service_pool.cc:224] Handling call
0709 12:57:56.545437 (+283985us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:57:56.545789 17491 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:56.509099 17472 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:57:56.587528 17497 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.001s
I20260709 12:57:56.588099 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.588775 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:56.596251 17548 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:57:56.599453 13615 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
W20260709 12:57:56.600827 17551 log.cc:927] Time spent T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.755s	user 0.006s	sys 0.008s
I20260709 12:57:56.602628 13539 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:57:56.574892 17600 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.511718 17354 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.603881 17621 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:56.604686 17329 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:57:56.605366 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:56.605647 17067 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:57:56.573371 17554 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:56.606315 17477 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.577580 17607 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:56.603185 17553 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.758s	user 0.002s	sys 0.009s
I20260709 12:57:56.503880 17415 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.611773 14241 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:56.600282 17296 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:57:56.555543 13534 rpcz_store.cc:276] 0709 12:57:55.102123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.107830 (+  5707us) service_pool.cc:224] Handling call
0709 12:57:55.111888 (+  4058us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:55.142856 (+ 30968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:55.142887 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:55.144693 (+  1806us) log.cc:844] Serialized 26 byte log entry
0709 12:57:56.293366 (+1148673us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b18004aeae0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:57:56.508138 (+214772us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:57:56.508206 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:56.508310 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:56.508350 (+    40us) raft_consensus.cc:1685] finished
0709 12:57:56.508364 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:56.508615 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138170,"raft.queue_time_us":31788,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":15510,"spinlock_wait_cycles":44879232,"thread_start_us":93169,"threads_started":1,"wal-append.queue_time_us":1358322}
I20260709 12:57:56.632637 17437 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:57:56.614396 17586 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.633456 17067 consensus_queue.cc:1243] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:57:56.633266 17437 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:56.629494 17607 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:56.633726 13538 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 6. Current term is 7. Ops: []
I20260709 12:57:56.635068 17437 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.641571 17415 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.627130 17554 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:56.647022 17621 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:56.649302 17354 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:56.650135 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 77337.6us
W20260709 12:57:56.650957 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 71994.1us
I20260709 12:57:56.652109 17217 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:56.652228 17600 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:57:56.660564 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 176078us
I20260709 12:57:56.658881 17621 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:56.661631 17217 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:56.661635 17600 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:56.685976 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:56.690804 17600 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:56.704526 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:56.709801 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 137073us
I20260709 12:57:56.711972 13542 raft_consensus.cc:2368] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8.
I20260709 12:57:56.714766 17583 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.716262 17615 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:57:56.716912 17615 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:56.717358 17615 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:56.720319 17621 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.722203 17621 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:56.722663 17619 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:57:56.723948 14209 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:56.726788 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56551.5us
I20260709 12:57:56.727902 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:56.728384 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 3025) took 1381 ms. Trace:
I20260709 12:57:56.728856 13686 rpcz_store.cc:276] 0709 12:57:55.346981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.691492 (+344511us) service_pool.cc:224] Handling call
0709 12:57:55.691793 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:56.127086 (+435293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:56.727850 (+600764us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54002981d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:56.728178 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999528320}
I20260709 12:57:56.729732 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.731213 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.732959 13584 leader_election.cc:400] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11.
I20260709 12:57:56.716372 17542 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.733664 17542 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:57:56.734870 17390 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:56.730839 17580 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.733261 13584 leader_election.cc:403] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:57:56.766207 17402 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:57:56.766741 17402 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:56.767246 17402 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:57:56.772261 17415 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11.
I20260709 12:57:56.775564 17437 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:57:56.794322 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 64973.7us
W20260709 12:57:56.797752 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:56.799876 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:57:56.803836 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:56.809897 13537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1620) took 1700 ms (client timeout 1773 ms). Trace:
W20260709 12:57:56.810305 13537 rpcz_store.cc:269] 0709 12:57:55.109550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.140671 (+ 31121us) service_pool.cc:224] Handling call
0709 12:57:56.809687 (+1669016us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b540007dad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:56.809838 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765858048}
W20260709 12:57:56.823138 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 75866.7us
W20260709 12:57:56.826484 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 96847.9us
I20260709 12:57:56.832660 14209 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.801456 17580 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:57:56.837791 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 153224us
I20260709 12:57:56.835372 17390 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:57:56.851197 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 69744.1us
W20260709 12:57:56.858318 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:56.822468 17607 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.865090 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.833065 17554 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.813457 17217 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.891518 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 3011) took 2363 ms. Trace:
W20260709 12:57:56.903702 17579 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:57:56.928748 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 74225.7us
I20260709 12:57:56.933310 17607 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.934837 17641 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:56.897038 17599 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:56.938441 17524 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:56.942502 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 94048.9us
I20260709 12:57:56.942713 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:56.943431 17554 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.943897 17629 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.947539 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:56.948807 17632 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:57:56.950431 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.952232 17515 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:56.952907 17516 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:56.953019 17515 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.955417 17515 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:56.958184 17377 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:56.958665 17377 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:56.944304 17631 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.928458 17437 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:56.936614 17615 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.953452 17516 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:56.944908 17599 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:56.964359 17643 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:57:56.943573 17390 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.945473 17642 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:57:56.962203 13619 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:56.897931 13692 rpcz_store.cc:276] 0709 12:57:54.484200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.584163 (+ 99963us) service_pool.cc:224] Handling call
0709 12:57:54.584443 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:54.590356 (+  5913us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:57:54.606400 (+ 16044us) log.cc:844] Serialized 14 byte log entry
0709 12:57:54.744768 (+138368us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:54.745240 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:57:54.745362 (+   122us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:57:54.749108 (+  3746us) log.cc:844] Serialized 14 byte log entry
0709 12:57:54.899685 (+150577us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b18003ba160. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6932a6 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1
0709 12:57:54.914897 (+ 15212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:54.915013 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:55.014684 (+ 99671us) spinlock_profiling.cc:243] Waited 934 us on lock 0x7b18003ba160. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:55.185229 (+170545us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b4800212958. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 12:57:55.334958 (+149729us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b18003ba160. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:57:56.602267 (+1267309us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:56.826089 (+223822us) raft_consensus.cc:1685] finished
0709 12:57:56.826112 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:56.847247 (+ 21135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3107,"prepare.queue_time_us":16166,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":11297,"raft.run_cpu_time_us":2227,"raft.run_wall_time_us":448965,"spinlock_wait_cycles":40458880,"thread_start_us":249558,"threads_started":2,"wal-append.queue_time_us":178658}
I20260709 12:57:56.965238 17599 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.976409 17416 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:56.980120 17524 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.981089 17584 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.982527 17524 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.982733 17584 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:56.984298 17543 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 12:57:56.985180 17626 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:57:56.985284 17543 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:57:56.985838 17543 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:57:56.986624 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 154095us
I20260709 12:57:56.963240 17607 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:56.988960 17607 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.989310 17520 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:56.989660 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:56.990077 17520 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.991279 17607 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.981477 17615 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:56.923839 17586 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:57:56.992344 17586 consensus_queue.cc:1243] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:57:56.992902 17586 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:56.993237 17586 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:56.993283 17641 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:57:56.993640 13542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1517) took 1494 ms (client timeout 1899 ms). Trace:
I20260709 12:57:56.994302 17586 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:56.994899 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 44642.6us
W20260709 12:57:56.994084 13542 rpcz_store.cc:269] 0709 12:57:55.499145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:56.028282 (+529137us) service_pool.cc:224] Handling call
0709 12:57:56.993392 (+965110us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b540028cb54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:56.993562 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383996288}
I20260709 12:57:56.995859 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:56.998468 13687 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:57:57.000022 17378 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:57:57.000703 17378 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:57:57.001767 17378 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.002813 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1516) took 1505 ms. Trace:
I20260709 12:57:57.003533 17594 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:57.004199 17594 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:57.004984 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1730) took 1050 ms. Trace:
I20260709 12:57:57.005357 17594 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.005409 13613 rpcz_store.cc:276] 0709 12:57:55.954144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.979842 (+ 25698us) service_pool.cc:224] Handling call
0709 12:57:56.512534 (+532692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:56.545955 (+ 33421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:57.004726 (+458771us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540013bfd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:57.004914 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224131840}
I20260709 12:57:56.959823 17377 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.013250 17377 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:57.014411 17347 consensus_queue.cc:1175] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" responder_term: 10 status { last_received { term: 9 index: 8 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:57:57.013854 17520 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:57.016345 17496 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:57.017081 17473 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:57:56.991232 14209 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:56.903646 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:57:57.003260 13538 rpcz_store.cc:276] 0709 12:57:55.497820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.498123 (+   303us) service_pool.cc:224] Handling call
0709 12:57:56.509228 (+1011105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:56.636229 (+127001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:57.001616 (+365387us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540028b754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:57.002537 (+   921us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b540028b750. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:57.002733 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2504775424}
I20260709 12:57:57.027902 17588 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:57.017042 17496 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:56.974579 14211 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:56.986955 17543 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:56.995791 17586 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:57.053453 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:57.054600 17641 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:57.055192 17641 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:57.056633 17329 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:57:57.057657 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 2999) took 3158 ms. Trace:
I20260709 12:57:57.058053 13693 rpcz_store.cc:276] 0709 12:57:53.899290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:54.540873 (+641583us) service_pool.cc:224] Handling call
0709 12:57:54.573626 (+ 32753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:54.681784 (+108158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:57.057248 (+2375464us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540002c454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:57.057490 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247088384}
I20260709 12:57:57.059551 17641 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:57.061062 17625 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.348s	user 0.003s	sys 0.000s
I20260709 12:57:57.063347 13692 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:57:57.065428 17645 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:57.066202 17600 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.065068 17516 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.063197 17217 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:57.081916 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 86240.7us
I20260709 12:57:57.082190 13616 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:57:57.087334 17600 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:57.087455 17516 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:57.088691 17000 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.082185 17543 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:57.089473 13619 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:57:57.090202 17563 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:57:57.080085 17542 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:57.106889 17402 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:57.040406 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 101540us
W20260709 12:57:57.108417 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 117343us
W20260709 12:57:57.114339 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 171787us
W20260709 12:57:57.116382 17636 log.cc:927] Time spent T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
I20260709 12:57:57.078312 17329 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:57.127023 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:57.131784 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.141450 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:57:57.145082 17630 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.255s	user 0.000s	sys 0.003s
I20260709 12:57:57.144928 17643 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:57.146375 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1631) took 1309 ms. Trace:
I20260709 12:57:57.146731 13536 rpcz_store.cc:276] 0709 12:57:55.837171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:56.359691 (+522520us) service_pool.cc:224] Handling call
0709 12:57:56.366183 (+  6492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:56.471393 (+105210us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:56.471433 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:56.546632 (+ 75199us) log.cc:844] Serialized 26 byte log entry
0709 12:57:56.701419 (+154787us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:57:56.701457 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:56.701516 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:57.117218 (+415702us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:57.146096 (+ 28878us) raft_consensus.cc:1685] finished
0709 12:57:57.146117 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:57.146316 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95705,"raft.queue_time_us":105558,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":8089,"spinlock_wait_cycles":23844608,"thread_start_us":94486,"threads_started":1,"wal-append.queue_time_us":243627}
I20260709 12:57:57.116871 17000 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.148985 13658 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:57.120987 17588 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:57.120743 17542 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.116997 17437 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:57:57.157217 17416 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:57.137247 17646 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:57:57.178706 17416 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:57.201259 13619 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:57:57.180397 16978 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:57:57.216260 13618 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 11. Ops: []
I20260709 12:57:57.205296 17416 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:57.176990 17477 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:57:57.225912 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.233843 13693 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:57:57.282678 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 125743us
I20260709 12:57:57.293977 13611 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:57.197072 17588 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:57.266570 17621 consensus_meta.cc:287] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.544s	user 0.014s	sys 0.000s
I20260709 12:57:57.299598 13618 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
W20260709 12:57:57.300733 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 70882.4us
W20260709 12:57:57.303058 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.226205 17589 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.167s
I20260709 12:57:57.282990 17402 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:57.295161 17496 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:57.320529 17580 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:57.326193 17580 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.321439 17437 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:57.329433 17381 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.322690 13584 leader_election.cc:400] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 7. Current term is 8.
I20260709 12:57:57.329900 17588 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:57.330034 17381 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.209954 17329 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.329936 17437 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.296563 17563 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:57.338550 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.341881 17542 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:57.313076 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 110910us
W20260709 12:57:57.365175 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.389756 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:57.359035 17580 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:57.413275 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:57.432420 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:57.434131 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 63093.7us
I20260709 12:57:57.443610 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.371169 17563 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.448487 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:57.442680 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:57.453290 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:57.416042 17621 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:57:57.437752 16978 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:57.473461 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:57.481379 13615 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:57.482203 14245 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:57:57.486574 17633 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 12:57:57.487764 17649 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:57:57.496163 13615 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:57.497377 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:57.497628 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 45178.1us
W20260709 12:57:57.508080 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:57.513968 17648 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.219s	user 0.002s	sys 0.000s
I20260709 12:57:57.611749 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.608501 14241 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEARNER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:57.508433 17658 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.614482 17658 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.615350 17419 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.616945 17658 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:57.532181 17496 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.617918 17496 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:57.618453 17217 consensus_meta.cc:287] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.538s	user 0.018s	sys 0.000s
I20260709 12:57:57.479018 17477 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:57:57.508913 17393 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:57:57.506654 16978 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.616438 17419 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:57.626559 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 123460us
I20260709 12:57:57.634774 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:57.635849 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:57.644086 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.621122 17496 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:57.655813 13537 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:57.657311 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.627600 17477 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:57.667012 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:57.670989 14245 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:57:57.671492 14240 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:57:57.674418 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:57.675711 17639 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:57.676204 17639 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:57.677173 17639 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.678269 17639 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:57.681643 17638 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:57:57.682083 17638 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:57:57.683001 17638 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.683517 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.684247 14237 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:57.684648 14242 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:57:57.685811 14244 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:57.687873 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:57.691521 17543 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.002s
W20260709 12:57:57.692097 17408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.009s	sys 0.017s
W20260709 12:57:57.692426 17408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.010s	sys 0.017s
I20260709 12:57:57.696784 14239 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:57.721177 14238 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:57:57.721786 17586 consensus_meta.cc:287] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.000s
I20260709 12:57:57.659324 17381 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:57.722754 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:57:57.722810 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:57.724217 17657 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.143s	user 0.002s	sys 0.002s
I20260709 12:57:57.724438 13612 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:57:57.724762 13612 pending_rounds.cc:85] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 5
I20260709 12:57:57.724994 13612 pending_rounds.cc:107] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:57:57.725260 13612 raft_consensus.cc:2889] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:57.729300 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:57:57.665557 17477 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:57:57.729993 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.730365 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:57.729522 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:57.731247 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.731173 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.732302 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.733348 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.733943 13584 leader_election.cc:400] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11.
I20260709 12:57:57.734532 17408 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:57.737717 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.738523 14237 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:57:57.741088 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:57.742057 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.745462 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:57.746592 17513 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:57.747008 17513 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:57.747231 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.747388 17513 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:57.737762 17543 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:57:57.754137 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 119453us
I20260709 12:57:57.757115 13508 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:57.740123 17496 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:57:57.763366 13658 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:57:57.788609 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:57.789211 13615 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:57:57.739971 17408 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.758605 17437 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:57.764845 17563 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:57.800380 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 154947us
I20260709 12:57:57.804555 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.806155 14241 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:57.808388 17381 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:57.812489 17663 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:57:57.814445 17416 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.004s
I20260709 12:57:57.816603 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.818619 17000 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:57.819723 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:57.820094 17600 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.820233 17541 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:57:57.820569 17524 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.821153 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:57.822976 17408 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:57.823694 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 69826.7us
W20260709 12:57:57.824905 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.788301 17583 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:57.794243 17378 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:57:57.794049 17542 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:57:57.836329 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.834513 17586 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.835000 17393 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:57.855005 13614 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:57:57.827744 14246 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:57:57.870332 14246 pending_rounds.cc:85] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 7
I20260709 12:57:57.865154 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:57:57.872394 14246 pending_rounds.cc:107] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:57:57.824815 17543 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.850746 17584 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:57:57.874207 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 151552us
I20260709 12:57:57.840694 17437 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.879616 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.884455 14246 raft_consensus.cc:2889] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:57:57.896747 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:57.904142 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:57.905871 13619 raft_consensus.cc:2410] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:57:57.868207 17524 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.846998 17416 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:57.889550 17664 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.256s	user 0.001s	sys 0.001s
I20260709 12:57:57.875684 17393 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:57.921350 13615 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
W20260709 12:57:57.899190 17516 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.785s	user 0.010s	sys 0.003s
W20260709 12:57:57.924271 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 69390.1us
W20260709 12:57:57.922086 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 71686.7us
I20260709 12:57:57.886160 17621 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:57:57.862757 17496 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:57:57.897675 17583 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:57.932274 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:57.875629 17600 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.876276 17381 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:57.939514 17668 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:57:57.941252 17416 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:57.943197 17524 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:57.943877 17496 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:57:57.945068 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:57.931381 13429 maintenance_manager.cc:643] P 7c7c6d1a2ec946878b513e7eb3a47a90: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 4.806s	user 0.186s	sys 0.095s Metrics: {"bytes_written":29882,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":238653,"dirs.run_cpu_time_us":643,"dirs.run_wall_time_us":56368,"drs_written":1,"lbm_read_time_us":450,"lbm_reads_lt_1ms":4,"lbm_write_time_us":276102,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":11,"lbm_writes_lt_1ms":22,"mutex_wait_us":6320,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":181693,"threads_started":1,"update_count":11,"wal-append.queue_time_us":19428}
I20260709 12:57:57.946544 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:57:57.948609 17667 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:57:57.952366 13611 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 11. Ops: []
I20260709 12:57:57.924386 17643 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.875825 17660 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:57:57.916210 17408 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:57:57.970304 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:57.936715 17583 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:57:57.917745 17586 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.977252 13619 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:57.978973 13509 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:57:57.979626 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 53417.4us
I20260709 12:57:57.928007 17437 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:57.979450 17583 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:57.937009 17377 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:57:57.937450 17584 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:57:57.965302 17541 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:57.987130 17541 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.987576 17586 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:57.960903 17516 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.982479 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:57.990368 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 61172.1us
I20260709 12:57:57.991209 13508 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:57.993093 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:57.993620 13614 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:57.962426 17496 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:57.937316 17621 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.989970 17679 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:57.995898 17563 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:57.997957 13610 raft_consensus.cc:2393] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:57.999050 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:57.997709 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:57:58.006070 17589 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:58.010474 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:58.010396 17589 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.011029 17381 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.011809 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:58.012656 17679 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:58.012944 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:58.012748 17589 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.016855 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:58.019220 13610 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 8: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:57:57.995011 17580 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:57:58.013324 17563 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.020686 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:58.021169 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.021384 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.022185 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.022748 13430 maintenance_manager.cc:419] P 7c7c6d1a2ec946878b513e7eb3a47a90: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9655 bytes on disk
I20260709 12:57:57.998504 17347 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:57.970716 17643 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:57.994975 17516 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.026569 17381 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:57:58.081132 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:57:58.048825 17594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.008s	sys 0.014s
I20260709 12:57:58.087883 17329 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:58.087694 17594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.008s	sys 0.015s
W20260709 12:57:58.087639 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 88665.5us
I20260709 12:57:58.088749 13429 maintenance_manager.cc:643] P 7c7c6d1a2ec946878b513e7eb3a47a90: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.040s	user 0.005s	sys 0.008s Metrics: {"cfile_init":1,"lbm_read_time_us":1506,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260709 12:57:58.089511 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:57:58.092361 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:58.093698 13610 raft_consensus.cc:2410] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:57:58.095467 13610 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:57:58.096156 17627 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:57:58.099293 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 101632us
I20260709 12:57:58.106523 13614 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:57:58.104720 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:58.013933 17513 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.044668 17580 raft_consensus.cc:697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:58.112636 13614 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: []
I20260709 12:57:58.087761 17274 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:58.115785 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 91232.1us
I20260709 12:57:58.119683 13685 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:57:58.124354 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.084247 17381 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:57:58.126247 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 135107us
I20260709 12:57:58.127101 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:58.128719 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:58.128867 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:58.133579 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.134832 13610 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:57:58.136245 17673 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 12:57:58.137964 17677 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:57:58.129254 17381 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:58.135581 17274 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.146339 17274 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:58.147429 17600 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.148761 17563 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.149451 17496 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:58.151000 17496 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:58.153334 17513 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:57:58.156599 17656 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.816s	user 0.006s	sys 0.000s
W20260709 12:57:58.132793 17669 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:57:58.164132 17381 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.165818 17437 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:58.167037 17381 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:58.166848 17580 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:58.172472 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 54100.3us
I20260709 12:57:58.173377 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:58.174729 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:58.179169 13685 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
W20260709 12:57:58.179291 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 50646.3us
I20260709 12:57:58.183167 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 674) took 1118 ms. Trace:
I20260709 12:57:58.130227 17347 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:57:58.128508 17679 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:57:58.186460 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.175786 17563 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:57:58.038602 17393 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.187752 13508 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:58.206707 17347 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.219192 17347 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.221787 17347 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:58.188431 17274 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:57:58.184218 17437 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:57:58.225924 13693 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:57:58.183605 13613 rpcz_store.cc:276] 0709 12:57:57.064852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:57.077189 (+ 12337us) service_pool.cc:224] Handling call
0709 12:57:57.082944 (+  5755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:57.090871 (+  7927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:57.090902 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:57.091451 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:57:57.216110 (+124659us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:57.216158 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:57.216237 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:58.155828 (+939591us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:58.177151 (+ 21323us) raft_consensus.cc:1685] finished
0709 12:57:58.177177 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:58.183084 (+  5907us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5007,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":4149504,"thread_start_us":124258,"threads_started":1,"wal-append.queue_time_us":248369}
I20260709 12:57:58.228103 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:58.185956 17684 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:57:58.234126 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 58224.9us
I20260709 12:57:58.229746 17347 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.235806 17347 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.236917 17541 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.237926 17686 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:57:58.255607 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:58.258807 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:58.259740 13618 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:57:58.260907 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:58.262200 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:58.262842 13618 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:57:58.267448 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2814) took 3420 ms. Trace:
I20260709 12:57:58.268333 17419 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.267953 13616 rpcz_store.cc:276] 0709 12:57:54.846982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:55.381017 (+534035us) service_pool.cc:224] Handling call
0709 12:57:55.381245 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:57.082683 (+1701438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:58.267121 (+1184438us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b540020a654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:58.267359 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3897241344}
I20260709 12:57:58.273017 17643 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:58.278329 13613 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:57:58.280856 13685 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:57:58.282088 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 101623us
W20260709 12:57:58.282887 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:58.284994 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:58.285564 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 153715us
I20260709 12:57:58.180667 17477 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.230253 17679 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:58.239806 17347 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:58.565403 13693 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
W20260709 12:57:58.561323 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 246560us
I20260709 12:57:58.567241 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:58.568140 13618 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:58.580602 13613 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:57:58.575416 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.305820 17542 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.305267 17621 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:57:58.584795 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 293538us
I20260709 12:57:58.245993 17583 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.310212 17347 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:58.572959 17607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.257s	user 0.020s	sys 0.072s
I20260709 12:57:58.322297 17217 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:58.582909 17416 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.658490 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.307826 17684 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.314s
I20260709 12:57:58.660357 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:58.591246 17541 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.649085 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.669673 13613 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:57:58.668269 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:58.617619 17607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.301s	user 0.020s	sys 0.072s
I20260709 12:57:58.629664 17639 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:57:58.690547 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:58.706477 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:57:58.712409 13613 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:58.711365 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:58.715083 13685 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:57:58.656741 17347 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:58.702178 17329 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.003s
I20260709 12:57:58.632331 17542 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.716147 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:58.696311 17516 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:58.694904 17691 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:57:58.746577 13693 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:57:58.674149 17416 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.694329 17541 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.685972 17217 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.705637 17639 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.775274 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:58.747701 17329 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:58.783447 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:58.754120 17685 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.816382 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:58.818167 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:58.762274 17580 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.824075 17419 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.828925 17274 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.828776 17419 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.829495 17274 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.831110 17419 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.836809 13693 raft_consensus.cc:2410] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:57:58.839389 17416 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.839869 17347 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:58.759639 17691 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.843461 17691 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:58.833621 17693 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.844242 17699 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.844964 17583 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.872308 13687 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:57:58.844288 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.847182 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2838) took 1331 ms. Trace:
I20260709 12:57:58.848389 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:58.849472 13535 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:57:58.850109 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:58.831738 17274 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:58.851127 13534 raft_consensus.cc:1240] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:57:58.703220 17679 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.864113 13541 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:57:58.876334 13539 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:57:58.850967 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.877630 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:58.877821 17541 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:58.874094 13612 rpcz_store.cc:276] 0709 12:57:57.516178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:57.611349 (+ 95171us) service_pool.cc:224] Handling call
0709 12:57:57.611673 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:57.725731 (+114058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:58.846942 (+1121211us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54003b0454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:58.847113 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258292992}
W20260709 12:57:58.879041 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:58.874954 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:57:58.880421 17698 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.881294 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.880702 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.826222 17685 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.908890 13543 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:57:58.922767 13659 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.022s	sys 0.040s
W20260709 12:57:58.923144 13659 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.022s	sys 0.040s
W20260709 12:57:58.923403 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 674706us
I20260709 12:57:58.908890 13693 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:57:58.905412 13685 raft_consensus.cc:2410] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:57:58.888077 13692 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:58.910562 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:57:58.913561 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:58.854481 17329 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:57:58.916080 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.886008 17347 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:58.885514 17477 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:58.948832 13689 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:57:58.949337 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:57:58.921224 17496 consensus_meta.cc:287] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.000s
I20260709 12:57:58.852501 17580 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:58.889221 17680 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:57:58.873257 17693 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:58.995589 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.011209 13542 raft_consensus.cc:2410] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:57:58.943475 17541 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:57:59.024650 14242 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.846s	user 0.008s	sys 0.003s
I20260709 12:57:59.025311 14242 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:57:58.998684 17419 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:59.027024 17419 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:59.030704 17419 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:59.039144 13686 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: []
I20260709 12:57:58.966223 17274 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:59.042900 17704 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:59.043692 17704 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:59.045094 17684 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:59.045745 17684 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:57:59.047878 17703 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:57:58.996084 17680 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:59.052593 17680 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.053984 17680 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:57:59.055227 17700 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:57:59.056648 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:57:58.995097 17347 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:59.064208 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1752) took 1043 ms. Trace:
I20260709 12:57:59.065961 17347 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.064805 13614 rpcz_store.cc:276] 0709 12:57:58.021262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.130703 (+109441us) service_pool.cc:224] Handling call
0709 12:57:58.185449 (+ 54746us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:58.218761 (+ 33312us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:58.219097 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 12:57:58.579698 (+360601us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:58.591365 (+ 11667us) log.cc:844] Serialized 26 byte log entry
0709 12:57:58.612935 (+ 21570us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:57:58.612981 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:58.613061 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:59.063886 (+450825us) raft_consensus.cc:1685] finished
0709 12:57:59.063931 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:59.064124 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5561,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":32233,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":19967,"thread_start_us":359960,"threads_started":1,"wal-append.queue_time_us":774169}
I20260709 12:57:59.067193 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2833) took 1771 ms. Trace:
I20260709 12:57:59.067479 13615 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:57:59.067433 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:59.072530 13614 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:57:59.050922 17580 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.067556 13611 rpcz_store.cc:276] 0709 12:57:57.295724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:57.300028 (+  4304us) service_pool.cc:224] Handling call
0709 12:57:57.339248 (+ 39220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:57.967296 (+628048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:59.066937 (+1099641us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b54001469d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:57:59.067136 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374867712}
W20260709 12:57:59.079493 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 133002us
I20260709 12:57:59.084208 13536 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:57:59.089576 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:57:59.092233 13614 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:57:59.084769 13536 pending_rounds.cc:85] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 8
I20260709 12:57:58.873807 17699 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:59.101133 13618 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.002s
I20260709 12:57:59.101572 13618 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:57:59.101495 17477 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:59.102406 17477 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.104638 17477 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.109841 17707 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:59.110128 17697 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:57:59.094471 13536 pending_rounds.cc:107] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 9.9
I20260709 12:57:59.111513 13536 raft_consensus.cc:2917] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Aborting config change with OpId 9.9: Aborted: Op aborted by new leader
I20260709 12:57:59.111862 13614 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:57:59.113847 13536 raft_consensus.cc:2889] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:57:59.122721 17685 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.126468 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:58.988705 17698 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.130806 13613 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:57:59.110822 17542 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.102537 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:59.094772 17580 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:57:59.081522 17541 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.091502 17274 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:59.153311 17665 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.425s	user 0.016s	sys 0.000s
W20260709 12:57:59.161034 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 240893us
I20260709 12:57:59.120497 17639 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.158169 13614 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:57:59.186614 13614 pending_rounds.cc:85] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 8
I20260709 12:57:59.186955 13614 pending_rounds.cc:107] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:57:59.187254 13614 raft_consensus.cc:2889] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:57:59.157341 17686 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.005s	sys 0.014s
I20260709 12:57:59.147644 17685 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:59.160295 17477 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:57:59.132154 17217 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.183800 13617 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:57:59.110488 17707 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:59.158314 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 55895us
I20260709 12:57:59.161726 13538 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:57:59.197291 17686 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.005s	sys 0.015s
I20260709 12:57:59.311908 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:57:59.314617 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:59.314977 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:59.315852 14240 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:57:59.195355 17639 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:59.318804 14239 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:59.320842 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:59.321328 17541 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.324620 17699 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.323645 17639 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.327325 17704 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.328351 17639 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.330211 17541 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:57:59.332855 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.333988 13537 raft_consensus.cc:2410] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:57:59.339733 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:59.340421 14244 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
W20260709 12:57:59.344592 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.000s	sys 0.032s
W20260709 12:57:59.344985 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.032s
W20260709 12:57:59.345223 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 561999us
W20260709 12:57:59.354884 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:59.369329 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.373317 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:59.374302 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:57:59.374694 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:59.339897 17685 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.648483 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.649314 13537 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:59.564477 17708 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:59.602535 17580 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:59.659953 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:57:59.650863 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 68112.5us
I20260709 12:57:59.662922 14244 raft_consensus.cc:2410] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:57:59.641769 17477 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:59.678323 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.690858 17543 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.647797 17347 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:57:59.701217 13543 consensus_meta.cc:287] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.002s
I20260709 12:57:59.701849 13543 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 6.
W20260709 12:57:59.703667 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:59.638469 17704 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:57:59.706769 17693 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.712054 17347 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:57:59.713181 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:57:59.716478 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.762679 13537 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
W20260709 12:57:59.720567 17723 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.349s	user 0.000s	sys 0.000s
I20260709 12:57:59.724393 14239 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
W20260709 12:57:59.728546 17716 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.396s	user 0.000s	sys 0.001s
I20260709 12:57:59.737924 17698 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.738667 17274 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.740007 17707 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.674742 17684 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.703303 17477 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.743805 17679 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:57:59.673846 17686 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:57:59.789675 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:59.789901 17686 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:57:59.764338 17732 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:57:59.791678 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:59.792424 17732 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.793035 14245 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:57:59.801062 17679 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:57:59.801682 17679 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:57:59.802762 17679 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:59.749689 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:59.804728 14244 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:57:59.808092 17477 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.811542 17685 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:57:59.775843 13508 connection.cc:256] RPC call timeout handler was delayed by -1.1078671419991224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
I20260709 12:57:59.823694 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:59.817797 17715 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.415s	user 0.000s	sys 0.001s
I20260709 12:57:59.795650 17732 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.831424 17679 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:57:59.841015 17686 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:57:59.869561 17685 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:57:59.769351 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1738) took 1891 ms. Trace:
W20260709 12:57:59.880905 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 180024us
I20260709 12:57:59.755352 13543 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:57:59.758033 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 475630us
I20260709 12:57:59.882715 13615 rpcz_store.cc:276] 0709 12:57:57.878040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:57.930874 (+ 52834us) service_pool.cc:224] Handling call
0709 12:57:59.067248 (+1136374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:59.068255 (+  1007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:59.769004 (+700749us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001469d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:57:59.769245 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2550117248}
W20260709 12:57:59.760367 17676 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.780s	user 0.004s	sys 0.021s
I20260709 12:57:59.668248 17580 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:59.667943 17708 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.774055 17347 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.775132 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:57:59.886225 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:59.897567 13537 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:57:59.900071 13612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.033s	sys 0.041s
W20260709 12:57:59.901794 13612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.033s	sys 0.041s
I20260709 12:57:59.884832 17685 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.906404 17685 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.910245 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:57:59.911300 17707 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:57:59.912233 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.912761 17580 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:57:59.912861 13543 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:57:59.913201 13539 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:57:59.891518 17728 consensus_queue.cc:1059] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:57:59.914497 17728 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:59.914822 17728 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:59.914860 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.915756 17708 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:57:59.915654 17728 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:57:59.917045 17728 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:57:59.917584 13539 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:57:59.892287 17704 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:57:59.919499 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.920341 13539 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:57:59.922298 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:59.888090 17477 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:57:59.926374 17477 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:57:59.926771 17477 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:57:59.928445 17477 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:57:59.929942 17477 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:59.930229 17712 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.589s	user 0.000s	sys 0.004s
I20260709 12:57:59.921849 17709 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:57:59.896037 17274 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:57:59.935894 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1080) took 1234 ms. Trace:
I20260709 12:57:59.936288 14243 rpcz_store.cc:276] 0709 12:57:58.701597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.701735 (+   138us) service_pool.cc:224] Handling call
0709 12:57:58.710816 (+  9081us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:58.739900 (+ 29084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:57:58.739970 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:58.763273 (+ 23303us) log.cc:844] Serialized 26 byte log entry
0709 12:57:59.070731 (+307458us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:57:59.259035 (+188304us) log.cc:844] Serialized 14 byte log entry
0709 12:57:59.259255 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:59.259327 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:57:59.929603 (+670276us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:57:59.935553 (+  5950us) raft_consensus.cc:1685] finished
0709 12:57:59.935670 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:57:59.935808 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12820,"prepare.queue_time_us":279709,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1860,"raft.queue_time_us":152335,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":15263,"spinlock_wait_cycles":5799168,"thread_start_us":329763,"threads_started":2,"wal-append.queue_time_us":367826}
I20260709 12:57:59.939591 14243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d73171142594388972ed22eae44f95f"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:57:59.940469 14243 raft_consensus.cc:606] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:57:59.950464 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:59.972447 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:57:59.973456 13543 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:59.978060 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:59.962416 17680 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.908s	user 0.016s	sys 0.000s
I20260709 12:57:59.979578 13539 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:57:59.980690 13543 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:57:59.982076 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:59.982625 13543 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:57:59.995012 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 92459.3us
I20260709 12:57:59.995407 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:57:59.998951 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:00.003715 17653 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 2.630s	user 0.000s	sys 0.017s
W20260709 12:57:59.997540 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47671.6us
I20260709 12:58:00.001523 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:00.012600 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:57:59.994608 17643 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.255s	user 0.007s	sys 0.020s
I20260709 12:58:00.011829 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:00.013856 17477 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:00.015811 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.016211 17477 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.016749 17477 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:00.017704 13539 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
W20260709 12:58:00.023178 14209 connection.cc:256] RPC call timeout handler was delayed by -1.1634907309999107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
I20260709 12:58:00.027206 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.027938 13685 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:58:00.028270 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.030452 13688 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:00.031388 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.031653 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.032029 13685 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 7.
I20260709 12:58:00.033035 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.033789 13689 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:00.034512 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:00.034924 14242 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59162 (request call id 721) took 2306 ms (client timeout 1859 ms). Trace:
I20260709 12:58:00.035465 13685 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
W20260709 12:58:00.014848 17643 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.275s	user 0.007s	sys 0.020s
I20260709 12:58:00.037022 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.038364 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1658) took 1004 ms. Trace:
I20260709 12:58:00.038779 13537 rpcz_store.cc:276] 0709 12:57:59.033734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:59.777546 (+743812us) service_pool.cc:224] Handling call
0709 12:57:59.777992 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:59.898058 (+120066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.038123 (+140065us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540007e4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:00.038305 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274191104}
I20260709 12:58:00.035197 13692 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
W20260709 12:58:00.035391 14242 rpcz_store.cc:269] 0709 12:57:57.728900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:57.729052 (+   152us) service_pool.cc:224] Handling call
0709 12:58:00.034750 (+2305698us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b5400650254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:00.034858 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026107136}
I20260709 12:58:00.047609 13543 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:00.049737 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.050801 14243 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:00.052842 13684 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:58:00.053610 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:00.054350 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.041754 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.066125 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:00.066396 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 78230.2us
I20260709 12:58:00.071326 17709 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.072116 17709 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.074971 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.075809 14245 raft_consensus.cc:2393] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:00.030930 17743 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:58:00.039978 17726 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:00.034648 17658 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:00.079432 14242 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:00.040115 17730 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.078791 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.082592 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:00.103570 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.104276 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.105914 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.107683 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.110244 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.109606 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.124039 14240 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:00.111089 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.077117 17274 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.130528 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:00.121012 17695 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.364s	user 0.003s	sys 0.006s
I20260709 12:58:00.136214 14245 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:58:00.140295 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.101900 17726 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.108383 17743 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:00.142877 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.129846 17732 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:00.145885 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.149194 17674 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:00.160310 13540 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:58:00.150854 17709 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:00.139938 17724 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.474s	user 0.002s	sys 0.001s
I20260709 12:58:00.144575 17274 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.129896 17730 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.166738 13688 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:58:00.175725 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.129438 17658 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.144105 17726 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:00.159160 17329 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:00.179388 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1651) took 1262 ms. Trace:
I20260709 12:58:00.179837 13541 rpcz_store.cc:276] 0709 12:57:58.917425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.917941 (+   516us) service_pool.cc:224] Handling call
0709 12:57:58.918214 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:58.918397 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:57:58.918664 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 12:58:00.158592 (+1239928us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b18004ae120. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a
0709 12:58:00.175441 (+ 16849us) log.cc:844] Serialized 14 byte log entry
0709 12:58:00.179033 (+  3592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:00.179083 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:00.179133 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.179201 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:00.179311 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15716,"prepare.queue_time_us":81691,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":23850,"raft.queue_time_us":20017,"spinlock_wait_cycles":5702912,"thread_start_us":163967,"threads_started":1,"wal-append.queue_time_us":268832}
I20260709 12:58:00.182250 17730 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:00.183002 14240 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:00.192776 14240 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:58:00.189433 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:00.183827 17696 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.490s	user 0.005s	sys 0.000s
I20260709 12:58:00.198383 17729 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:00.198900 17729 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.199286 17729 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:58:00.196628 14209 leader_election.cc:336] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.595s (SENT)
I20260709 12:58:00.206705 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.207569 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.208410 13541 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:00.213697 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.214648 13543 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:00.206040 17408 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:00.224784 17408 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.227270 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1649) took 1337 ms. Trace:
I20260709 12:58:00.227741 13535 rpcz_store.cc:276] 0709 12:57:58.889438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.889744 (+   306us) service_pool.cc:224] Handling call
0709 12:57:58.911320 (+ 21576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:57:59.012175 (+100855us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:59.083343 (+ 71168us) log.cc:844] Serialized 14 byte log entry
0709 12:57:59.315650 (+232307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:57:59.316159 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:57:59.316284 (+   125us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:57:59.316391 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:57:59.316455 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:00.028200 (+711745us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:00.226958 (+198758us) raft_consensus.cc:1685] finished
0709 12:58:00.227066 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:00.227209 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80565,"prepare.queue_time_us":86733,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":130871,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":8786,"spinlock_wait_cycles":22614272,"thread_start_us":192027,"threads_started":1,"wal-append.queue_time_us":257456}
I20260709 12:58:00.228940 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:00.234798 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.232326 17408 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:00.244313 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.248086 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1754) took 1999 ms. Trace:
I20260709 12:58:00.248606 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.251717 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.255235 13541 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:58:00.258879 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2849) took 1520 ms. Trace:
I20260709 12:58:00.259398 13613 rpcz_store.cc:276] 0709 12:57:58.738189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:59.134207 (+396018us) service_pool.cc:224] Handling call
0709 12:57:59.134561 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:59.134807 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:59.159038 (+ 24231us) log.cc:844] Serialized 14 byte log entry
0709 12:58:00.258432 (+1099394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:00.258508 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:58:00.258571 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.258661 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:00.258801 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":917223,"prepare.queue_time_us":25403,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"thread_start_us":181692,"threads_started":1,"wal-append.queue_time_us":571026}
W20260709 12:58:00.251662 13659 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 2.003s (SENT) [suppressed 18 similar messages]
I20260709 12:58:00.265091 13612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2848) took 1527 ms. Trace:
W20260709 12:58:00.271282 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 74684.5us
I20260709 12:58:00.245612 17679 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:00.265666 13612 rpcz_store.cc:276] 0709 12:57:58.737513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:59.128241 (+390728us) service_pool.cc:224] Handling call
0709 12:57:59.155332 (+ 27091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:59.155613 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:57:59.376062 (+220449us) log.cc:844] Serialized 14 byte log entry
0709 12:58:00.263858 (+887796us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:00.264659 (+   801us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:00.264751 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.264849 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:00.265013 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366411,"prepare.queue_time_us":109905,"prepare.run_cpu_time_us":7823,"prepare.run_wall_time_us":27645,"thread_start_us":711868,"threads_started":2,"wal-append.queue_time_us":572146}
I20260709 12:58:00.263111 17473 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:58:00.277390 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:00.247225 17721 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:00.249607 13610 rpcz_store.cc:276] 0709 12:57:58.248361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.248477 (+   116us) service_pool.cc:224] Handling call
0709 12:57:58.289277 (+ 40800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:58.295205 (+  5928us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:57:58.566409 (+271204us) log.cc:844] Serialized 14 byte log entry
0709 12:58:00.198441 (+1632032us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18003b0f80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 12:58:00.247533 (+ 49092us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:00.247607 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:58:00.247678 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.247768 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:00.247965 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17521,"prepare.queue_time_us":275967,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":11637,"raft.queue_time_us":1584655,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4448,"spinlock_wait_cycles":28004096,"thread_start_us":102484,"threads_started":2,"wal-append.queue_time_us":172587,"wal-append.run_cpu_time_us":2326,"wal-append.run_wall_time_us":1315109}
W20260709 12:58:00.299264 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 1.44492e+06us
I20260709 12:58:00.300922 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.234421 17554 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 12:58:00.305187 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53558.5us
I20260709 12:58:00.268965 17543 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.268810 17686 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:00.308650 13508 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.268805 17586 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.276274 17679 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.275113 17607 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.284693 17408 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.282260 17217 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.322278 17583 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:00.211577 17743 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:00.329231 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.329869 17718 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:00.330449 17718 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:00.332039 17416 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.332754 17416 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.335642 17658 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.338600 17756 raft_consensus.cc:2764] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:58:00.338696 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.339829 17729 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.340480 13584 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.342063 17600 raft_consensus.cc:2764] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:58:00.342461 17729 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.342113 17473 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:00.342978 17600 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:00.317968 17686 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.343413 17600 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.345393 17686 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:00.345669 17600 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:00.317804 17679 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:00.314497 17755 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:00.322772 17680 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:00.360950 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:00.365182 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.315146 17543 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.354405 17542 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:00.316862 17586 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.322588 17408 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.337376 17217 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.321323 17607 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.370882 17477 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.363201 17680 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:00.363600 17755 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:00.372402 17274 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:00.376302 13542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1641) took 1530 ms (client timeout 1547 ms). Trace:
W20260709 12:58:00.395267 13542 rpcz_store.cc:269] 0709 12:57:58.845859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.848040 (+  2181us) service_pool.cc:224] Handling call
0709 12:58:00.374289 (+1526249us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54000daa54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:00.376228 (+  1939us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311679232}
I20260709 12:58:00.331648 17718 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:00.378597 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 98268.6us
I20260709 12:58:00.390059 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.400414 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.377854 17542 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:00.470579 17554 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:00.405478 17759 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 12:58:00.485342 17554 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.485805 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.483843 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.487272 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1597) took 1636 ms. Trace:
I20260709 12:58:00.487569 13616 raft_consensus.cc:2393] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 8: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:00.484308 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:00.489864 17638 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:00.492705 17554 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:00.429399 17755 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.493343 17408 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.455861 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.494232 13690 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:58:00.494424 17542 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.437829 17217 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.441043 17416 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:00.495653 17542 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:00.425272 17607 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.495942 17586 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.443141 17477 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:00.456297 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:00.497874 13692 raft_consensus.cc:2410] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:58:00.457931 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:00.498744 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 157132us
I20260709 12:58:00.487612 13686 rpcz_store.cc:276] 0709 12:57:58.850943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.851603 (+   660us) service_pool.cc:224] Handling call
0709 12:57:58.874390 (+ 22787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:59.042568 (+168178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.487068 (+1444500us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400238cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:00.487215 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378344704}
I20260709 12:58:00.499714 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:00.467844 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.503901 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:00.505019 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 102937us
I20260709 12:58:00.447994 17718 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:00.504521 13539 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:58:00.516566 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.512605 13693 raft_consensus.cc:2410] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:58:00.470100 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 781) took 1714 ms. Trace:
I20260709 12:58:00.517345 13688 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:00.517525 13538 rpcz_store.cc:276] 0709 12:57:58.755827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.755971 (+   144us) service_pool.cc:224] Handling call
0709 12:57:58.776760 (+ 20789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:59.282795 (+506035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.469749 (+1186954us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b540006efd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:00.470020 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880984320}
I20260709 12:58:00.450150 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.523852 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:00.523893 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 73849.2us
I20260709 12:58:00.529227 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1640) took 1701 ms. Trace:
I20260709 12:58:00.529603 13536 rpcz_store.cc:276] 0709 12:57:58.827463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:58.827789 (+   326us) service_pool.cc:224] Handling call
0709 12:57:58.838825 (+ 11036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:57:59.187770 (+348945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.529010 (+1341240us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54002846d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:00.529145 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529657088}
W20260709 12:58:00.533833 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.450601 17686 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.450824 17584 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:00.538959 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.534607 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.535845 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:00.416844 17543 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.543483 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.542691 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.546628 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.548019 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.547672 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.549896 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:00.554366 13686 raft_consensus.cc:2410] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:58:00.553475 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:00.562042 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:00.563620 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.542913 17381 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:00.521087 17516 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:58:00.569576 17763 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:58:00.570065 17556 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:00.441444 17680 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.545079 17743 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.550253 17541 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:00.572571 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.575893 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.593030 14240 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:00.537055 17726 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.533406 17627 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:58:00.607731 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.609356 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.609565 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:00.555521 17477 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:58:00.531302 17600 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.573139 17599 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:00.588738 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.558290 17274 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:00.594594 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.595219 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.600929 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.632298 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.632462 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.633634 13687 raft_consensus.cc:2410] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:58:00.633742 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.602595 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.603948 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.634668 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:00.635978 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.605563 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.636772 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.638383 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.639190 13613 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:58:00.639083 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.639740 13618 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:58:00.640669 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.641217 13618 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 6.
I20260709 12:58:00.598456 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.642079 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.556250 17732 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:00.573993 17381 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.537243 17554 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:58:00.573532 17419 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:00.616997 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.549657 17686 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.667819 13692 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:00.664537 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.598461 17556 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:00.588900 17543 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:58:00.631071 17599 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.556106 17615 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:58:00.626406 13535 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:00.617380 17477 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:00.630693 17755 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:00.628405 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:00.629254 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:00.545293 17584 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.637439 13616 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:58:00.573652 17621 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:58:00.644483 13691 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:58:00.644670 13540 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:00.637400 17563 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.645171 17643 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:00.646121 17642 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:00.646821 17702 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:00.649200 17600 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:00.650297 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.616885 17728 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:00.652886 17726 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:00.578488 17586 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:00.668121 17541 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:58:00.656047 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.735870 13613 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:00.659288 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.659571 13612 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:00.662516 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.568027 17759 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:58:00.665287 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.749882 13615 raft_consensus.cc:2410] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:58:00.664489 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.750797 13686 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:00.751807 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.666852 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.707331 17751 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.752431 13686 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:58:00.669054 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.754009 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.570978 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.618224 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 2850) took 1800 ms. Trace:
I20260709 12:58:00.670720 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.676712 17763 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.678265 17419 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:58:00.678442 13584 connection.cc:256] RPC call timeout handler was delayed by -1.4996063039998262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
I20260709 12:58:00.625985 17416 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:00.626905 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:00.764667 13584 connection.cc:256] RPC call timeout handler was delayed by -1.4978063039998233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18610000000000002s for a response
I20260709 12:58:00.600970 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.655241 17680 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:00.697194 13539 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:00.665458 17381 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:00.654440 17732 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:00.700858 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.610491 17743 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.706810 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:00.637552 17217 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:58:00.718323 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 87320.4us
I20260709 12:58:00.756361 13611 rpcz_store.cc:276] 0709 12:57:58.817697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:59.183026 (+365329us) service_pool.cc:224] Handling call
0709 12:57:59.259828 (+ 76802us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:00.273034 (+1013206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:00.273065 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:00.304715 (+ 31650us) log.cc:844] Serialized 26 byte log entry
0709 12:58:00.379535 (+ 74820us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:00.617706 (+238171us) log.cc:844] Serialized 14 byte log entry
0709 12:58:00.617943 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:00.618012 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:00.618052 (+    40us) raft_consensus.cc:1685] finished
0709 12:58:00.618063 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:00.618173 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1012914,"prepare.queue_time_us":268474,"raft.queue_time_us":1009434,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":14032,"spinlock_wait_cycles":15758592,"thread_start_us":307584,"threads_started":3,"wal-append.queue_time_us":66387}
I20260709 12:58:00.780344 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:00.723018 17771 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:58:00.722769 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.722640 17556 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:00.723424 17768 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:58:00.695492 17477 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:00.682616 17599 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:00.729604 13614 raft_consensus.cc:2410] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:58:00.590252 16978 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:00.683780 17408 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:00.737290 13617 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:00.694753 17729 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 12:58:00.752372 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 133272us
I20260709 12:58:00.756227 13684 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:00.756865 13690 raft_consensus.cc:2410] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:58:00.753098 13693 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:00.759236 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:00.765899 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 164998us
W20260709 12:58:00.768342 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.694425 17679 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.728938 17702 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.769594 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.774333 13616 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:00.754077 17686 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.768199 17416 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.735172 17563 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.771086 17644 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:00.746320 17658 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:58:00.775148 17732 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:58:00.802661 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 254948us
I20260709 12:58:00.799130 17728 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:00.813802 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.767587 17763 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.800537 17600 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.832504 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.801203 16978 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:00.773121 17751 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.811565 17726 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.812803 17702 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:00.840205 13689 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:00.838722 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:00.816581 17416 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:00.850167 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.844969 13539 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:00.850795 13611 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:00.810878 17408 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:00.842633 17728 raft_consensus.cc:1068] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Unable to promote non-voter 0f34cda059124f839711dbef61bb2c28: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:58:00.858196 13685 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:00.853822 13509 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:00.863005 13584 connection.cc:256] RPC call timeout handler was delayed by -1.3961063039996588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 12:58:00.843112 16978 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:58:00.863569 13584 connection.cc:256] RPC call timeout handler was delayed by -1.3124063039995235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 12:58:00.865621 13584 connection.cc:256] RPC call timeout handler was delayed by -1.1864063040002293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
W20260709 12:58:00.866573 13584 connection.cc:256] RPC call timeout handler was delayed by -1.1774063040002147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260709 12:58:00.865510 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.847116 17779 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:00.878623 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:00.880880 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.879604 13611 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
W20260709 12:58:00.883760 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45131.5us
I20260709 12:58:00.842439 17778 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:58:00.862928 17217 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:00.898957 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.904203 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.904657 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:00.877631 17779 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.874115 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:00.848218 17600 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.850054 17541 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:58:00.848886 17726 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:00.893689 17784 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:58:00.897590 17554 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:00.900262 17686 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:00.905478 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:00.874953 17584 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.901182 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:00.913183 13506 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:00.933073 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:00.911013 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:00.924193 13584 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:00.925364 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 145112us
I20260709 12:58:00.934477 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:00.926390 17786 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:00.869056 17783 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:58:00.940924 13506 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:00.909031 17763 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.897398 17755 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:58:00.947840 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.921947 17784 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:00.934109 13611 raft_consensus.cc:2376] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11. Re-sending same reply.
I20260709 12:58:00.905208 17787 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:00.908455 17583 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:00.855655 17419 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:00.908785 17779 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:00.921702 17554 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:00.971299 17756 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:58:00.953809 17780 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:58:00.974220 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:00.991492 13690 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:00.938426 17775 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:00.973448 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.938881 17777 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:00.943583 17541 consensus_queue.cc:1243] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:58:01.008852 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:00.910328 17217 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:00.995162 17782 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:58:00.995697 17756 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:58:00.969277 17784 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:58:00.998726 13616 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:00.993819 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:00.980091 17772 log.cc:927] Time spent T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:58:00.960831 17781 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:58:01.020208 17789 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:58:00.985742 17774 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:58:01.024111 17473 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:58:01.005731 17773 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:58:01.027335 17783 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:01.027812 17600 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:00.988888 17743 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.027968 17702 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.013144 17679 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.032234 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:01.100966 14237 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:01.036002 17638 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.003s
I20260709 12:58:00.988286 17763 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:00.983767 17554 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:58:01.042063 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 131113us
W20260709 12:58:01.048355 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 113993us
I20260709 12:58:01.106004 13509 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:01.106439 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:01.065568 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56824.4us
I20260709 12:58:01.040242 17563 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:00.939390 17686 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:58:01.125115 17542 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.003s
I20260709 12:58:01.125766 17686 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.127178 17416 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.014890 17787 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:01.132139 14244 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:01.041772 17583 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:01.044271 17627 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:01.094633 17586 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:01.078889 17543 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:01.081734 17709 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:01.081964 17390 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:01.117471 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:01.102792 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 288092us
I20260709 12:58:01.120899 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.090066 17774 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:00.942199 17786 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:01.145047 17599 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.148304 17790 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:58:01.147135 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 206168us
I20260709 12:58:01.224906 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.136837 17781 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:01.137504 17583 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:01.162184 17765 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:58:01.171272 14237 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:01.140976 17408 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.104928 17513 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:58:01.217633 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 100256us
W20260709 12:58:01.220201 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 99404.2us
I20260709 12:58:01.136243 17563 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:58:01.221289 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 227563us
I20260709 12:58:01.238375 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.238471 13584 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.239465 13688 raft_consensus.cc:2393] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:01.239545 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.226049 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.158124 17751 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.140823 17584 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:58:01.125784 17718 consensus_meta.cc:287] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
I20260709 12:58:01.210673 17729 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:01.230187 17781 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:01.222229 17786 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:01.240588 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:01.240880 13610 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.015s	sys 0.011s
I20260709 12:58:01.103190 17783 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.237042 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:01.216097 17709 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:01.242763 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.244971 13534 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 12: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:01.219466 17686 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:01.171191 17778 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:58:01.223541 17774 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:01.242383 17416 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.233841 17583 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:01.229549 17755 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:01.222956 17790 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:01.304775 17790 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.306051 17790 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:01.307518 14238 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:01.233364 17393 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 12:58:01.257906 13610 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.015s	sys 0.011s
I20260709 12:58:01.253194 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.240455 13535 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:58:01.231513 17765 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:01.259428 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 151830us
I20260709 12:58:01.328385 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.265806 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.269001 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.210700 17627 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.272481 17796 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:58:01.341890 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.343326 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:01.343326 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.224274 17390 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.255450 17781 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:01.273648 16978 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.266822 17709 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:01.274746 17783 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:01.275607 17792 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 12:58:01.280637 17794 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260709 12:58:01.281519 17788 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:58:01.254914 17513 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:01.285722 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.292374 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.340715 17743 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:01.302466 17477 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.267992 17757 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:01.255369 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.308344 13543 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:01.308950 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.375093 13540 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
W20260709 12:58:01.319509 17680 consensus_meta.cc:287] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.546s	user 0.001s	sys 0.011s
I20260709 12:58:01.257820 13537 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
W20260709 12:58:01.322187 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:01.322402 13508 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:01.378237 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 55918.3us
I20260709 12:58:01.325531 13542 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:01.332144 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.325492 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.332293 13538 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
W20260709 12:58:01.336443 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70752.8us
I20260709 12:58:01.380946 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.338188 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.340281 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.382261 13616 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 6: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:01.272271 17586 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.344502 13613 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:58:01.341437 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:01.353708 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.383463 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.357023 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:01.260221 17729 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.335626 17719 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:01.293934 17217 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.363471 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.393788 13534 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
W20260709 12:58:01.394845 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56732.7us
I20260709 12:58:01.368884 13617 raft_consensus.cc:2376] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12. Re-sending same reply.
I20260709 12:58:01.319012 17732 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.386296 14244 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:58:01.398869 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 151860us
I20260709 12:58:01.399091 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:01.353114 17416 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:01.390913 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.402396 13536 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:01.391842 13611 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:01.403024 13687 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:01.404007 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.401211 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.386420 17556 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:01.362120 17408 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:58:01.420401 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.347388 17779 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.439410 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.439608 17797 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.179s
I20260709 12:58:01.440272 17779 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:01.440546 17797 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:01.440960 17797 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:01.441419 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.441795 17797 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.443040 17797 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:01.444024 13610 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:58:01.444392 13610 pending_rounds.cc:85] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 5
I20260709 12:58:01.444603 13610 pending_rounds.cc:107] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:58:01.444825 13610 raft_consensus.cc:2889] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:01.445536 16978 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.446153 17799 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:01.446636 17799 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:01.446861 13610 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:01.446967 17799 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:01.448276 17541 consensus_queue.cc:1175] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ad2fdb1517c542439654bd3f7c72dad0" responder_term: 10 status { last_received { term: 10 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 6 } }
I20260709 12:58:01.448966 13614 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:01.440580 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.426638 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.430853 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.452828 13611 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:58:01.397930 17719 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.397699 17274 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:01.395013 17765 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.436520 17806 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:01.414948 17777 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:01.457353 13613 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:01.489902 17786 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.414333 17680 consensus_queue.cc:1059] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:58:01.492722 17513 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.496709 13614 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:01.497895 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:01.498939 13538 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: [8.4-10.5]
I20260709 12:58:01.401844 17686 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:01.394266 17798 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:01.460000 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.407128 17775 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:01.416658 17607 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:01.437247 17729 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.407305 17778 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 12:58:01.483340 17381 consensus_meta.cc:287] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.002s
I20260709 12:58:01.510322 17798 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:01.515508 17477 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.515722 17798 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.518728 17732 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.520066 17381 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.521103 17390 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.522153 17381 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.524526 13534 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:01.513408 17775 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:01.526530 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:01.526702 17775 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.528653 17786 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.409413 17743 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:58:01.511615 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:01.485466 17679 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:01.485705 17777 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:01.506702 17607 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:01.426203 17416 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.509534 17783 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:01.527732 17583 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:58:01.549083 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 145165us
I20260709 12:58:01.538642 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.457027 13687 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:58:01.560623 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.534727 17599 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.566073 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.566622 13509 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:01.576094 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 327910us
I20260709 12:58:01.576895 13612 raft_consensus.cc:2410] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:58:01.577526 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:01.545363 17777 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:01.574860 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:01.557761 17783 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:01.547957 17477 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:01.583572 17778 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:01.617029 17775 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:01.617234 17627 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:01.583804 17679 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:01.508028 17784 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:01.620640 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 222397us
W20260709 12:58:01.576887 13659 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.552s (SENT)
W20260709 12:58:01.648779 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 210345us
I20260709 12:58:01.655640 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:01.628036 13688 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:01.651348 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:01.665460 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 88601.7us
W20260709 12:58:01.666275 13659 leader_election.cc:336] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.592s (SENT)
I20260709 12:58:01.672895 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.676751 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:01.577719 17781 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.637334 17775 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:01.713387 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.716495 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:01.717077 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.718873 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:01.720108 13539 raft_consensus.cc:2393] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 7: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:01.721762 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.721894 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.722481 13613 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:01.724331 13541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:01.724992 13541 raft_consensus.cc:606] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.725371 13541 raft_consensus.cc:614] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:01.726236 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.730260 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:01.725308 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:01.731065 14245 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:58:01.732671 17554 consensus_meta.cc:287] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.003s
I20260709 12:58:01.733443 17784 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.681347 17743 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:58:01.734545 17818 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:58:01.735018 17743 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:58:01.734572 17819 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:58:01.733155 17554 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.736074 17743 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.738030 17554 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:01.738322 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.738812 14244 raft_consensus.cc:2410] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:58:01.739300 13613 raft_consensus.cc:2410] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:58:01.740347 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:01.741425 13613 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:01.744836 13540 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:01.745350 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.745524 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:01.730412 13659 leader_election.cc:336] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.508s (SENT)
I20260709 12:58:01.747073 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.747804 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:01.746306 13611 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
W20260709 12:58:01.750975 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:01.751866 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:01.588292 17774 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.722436 17586 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:01.755316 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:01.649287 17477 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.722334 17775 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:01.761348 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:01.728572 17642 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:01.727371 17732 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:01.746073 17419 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:01.650522 17778 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.749055 17804 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:01.770885 14244 raft_consensus.cc:2368] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:58:01.748968 17641 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:01.779263 13688 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:01.768672 17642 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:01.784055 17816 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:01.784842 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:01.843750 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 59001.6us
I20260709 12:58:01.785635 17416 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:01.764694 17563 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:01.788172 17809 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 12:58:01.759435 17784 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:01.789086 17777 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.794078 17811 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:01.803551 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 2864) took 1138 ms. Trace:
I20260709 12:58:01.804715 17583 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:01.806082 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:01.860656 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.861347 13691 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:01.862030 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.863013 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.807173 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.808269 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.809553 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:01.810777 17810 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:58:01.752736 17390 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:01.813992 17477 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:01.804739 17817 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:58:01.815771 17799 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.751721 17783 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:01.819176 17790 consensus_meta.cc:287] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.003s
I20260709 12:58:01.841197 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:01.852682 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.828090 17419 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.749282 17607 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.852532 17680 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:01.851953 17642 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:01.863894 13691 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:01.867691 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.870167 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.852136 17816 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:01.854509 17381 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:01.871614 13658 leader_election.cc:400] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:58:01.899768 13658 leader_election.cc:403] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:58:01.883189 13610 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:58:01.858705 13618 rpcz_store.cc:276] 0709 12:58:00.665512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:00.668801 (+  3289us) service_pool.cc:224] Handling call
0709 12:58:01.803311 (+1134510us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b540013abd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:01.803455 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2009967488}
I20260709 12:58:01.901407 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.902084 13618 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:58:01.903391 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:01.868973 17390 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:01.897360 14238 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 1075) took 4168 ms (client timeout 2119 ms). Trace:
I20260709 12:58:01.857084 17563 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.890277 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.911263 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:01.906054 17583 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:58:01.906329 17823 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:58:01.907259 17607 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:01.899696 17381 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:01.908187 17726 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:01.868314 17811 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.909946 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.912082 13685 raft_consensus.cc:2410] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:58:01.915900 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:01.922375 14244 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:58:01.910445 14238 rpcz_store.cc:269] 0709 12:57:57.729244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:57.730134 (+   890us) service_pool.cc:224] Handling call
0709 12:58:01.897136 (+4167002us) spinlock_profiling.cc:243] Waited 3.38 s on lock 0x7b54007238d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:01.897278 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7782810368}
I20260709 12:58:01.923242 14245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:01.926911 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.928514 13686 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:01.933928 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.937058 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.938397 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:01.939213 13688 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:01.940461 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.941803 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:01.943513 13688 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:58:01.928812 17381 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:01.946131 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.876278 17784 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.948386 13693 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:01.949199 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:01.921957 17390 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:01.898682 17816 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:01.925042 17541 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:58:01.966022 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:01.944031 13610 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:58:01.947860 17822 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:58:01.963145 13618 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 7.
I20260709 12:58:01.909332 17680 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.969117 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:01.961061 13688 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:01.971024 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:01.951099 17416 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:01.954573 17783 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:02.012732 13692 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:02.015182 17584 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.015748 17584 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.018090 17584 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:02.026872 17791 log.cc:927] Time spent T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.939s	user 0.006s	sys 0.000s
I20260709 12:58:02.026587 17642 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:01.965952 17393 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:58:02.055616 14238 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:01.973991 17543 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:01.967859 17781 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.062306 17642 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:02.064821 17826 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:58:02.000101 17774 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:02.000286 17827 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:58:02.000537 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:02.001874 17784 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.005252 17765 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:02.092801 13659 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.094581 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:02.006724 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:02.036214 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 578) took 1406 ms. Trace:
I20260709 12:58:02.037005 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:02.037335 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:02.040092 17709 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.005s
W20260709 12:58:02.046733 17825 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 12:58:02.047859 17816 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:01.967864 17586 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:02.072736 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.119629 13613 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:02.078459 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.080960 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.084550 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.085410 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:02.086364 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 115424us
W20260709 12:58:02.102077 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 64821.7us
I20260709 12:58:02.096159 13687 rpcz_store.cc:276] 0709 12:58:00.629488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:00.634366 (+  4878us) service_pool.cc:224] Handling call
0709 12:58:02.036010 (+1401644us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b540031d154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:02.036121 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765247232}
I20260709 12:58:02.123790 17784 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:02.171566 13618 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:02.177597 17811 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:02.163049 17541 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:58:02.180233 17777 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.180931 13686 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:02.176543 17784 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:02.087760 17607 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:02.183192 13619 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:02.184973 13509 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:02.194329 17777 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:58:02.194904 17777 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:02.196022 17777 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.139780 13610 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:58:02.187124 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 108796us
W20260709 12:58:02.187500 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 57886.7us
I20260709 12:58:02.191282 13618 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:02.180352 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:02.193171 13613 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:58:02.211735 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.212740 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.218585 13613 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:58:02.219552 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.225246 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.118902 17726 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:02.127529 17274 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:02.215232 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.186167 17784 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:02.225033 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:02.162788 17729 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:02.162873 17813 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:02.237088 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:02.164824 17816 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.170305 17680 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.176543 17799 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.209700 17800 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:02.223717 17584 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:02.124150 17709 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.188746 17543 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:58:02.211585 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.263762 13610 raft_consensus.cc:2368] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7.
I20260709 12:58:02.264952 13614 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:02.271761 17709 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.273578 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.275079 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.278501 13610 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:02.250357 17641 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:02.280550 14245 raft_consensus.cc:606] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.280962 14245 raft_consensus.cc:614] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:02.281733 14244 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:02.282927 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.257164 17781 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:02.235426 17274 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:02.222983 17393 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:58:02.249512 17778 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:02.230732 17779 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:58:02.250586 17812 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:58:02.264274 17729 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:02.316108 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 104634us
I20260709 12:58:02.342783 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.344434 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:02.345516 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:02.346452 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:02.326360 13619 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:58:02.328452 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:02.332999 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.348915 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.352427 13539 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:02.352761 13534 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 11: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:02.354267 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:02.339534 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.347646 17586 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:02.330202 17416 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.265331 17584 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:02.283588 17816 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:58:02.342451 17797 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.899s	user 0.007s	sys 0.009s
I20260709 12:58:02.325433 17274 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:02.358947 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.359714 13610 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:02.370222 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.372493 13610 raft_consensus.cc:2368] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:58:02.360147 17543 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:58:02.373327 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.370234 17416 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.342645 17799 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:02.382104 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:02.383064 13538 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:02.376811 17763 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:02.376308 17416 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.384389 17763 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:02.384099 17702 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.384862 17763 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:02.385459 17390 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.389007 17784 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.391278 17784 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.393225 17799 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.395605 17799 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.363662 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.370256 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.400197 13613 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:02.323786 17607 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:02.366660 17584 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:02.376212 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.376971 13691 raft_consensus.cc:2410] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:58:02.264027 17813 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.360746 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.343302 17815 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:02.378348 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 141333us
I20260709 12:58:02.380096 13534 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:58:02.407970 13534 pending_rounds.cc:85] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 9
I20260709 12:58:02.408281 13534 pending_rounds.cc:107] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:58:02.408511 13534 raft_consensus.cc:2889] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:02.397486 13537 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:02.409175 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:02.410178 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:02.381843 17701 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.400825 13539 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:02.401101 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.402292 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:02.416636 13613 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:02.373442 17543 consensus_queue.cc:1243] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:58:02.404788 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 162459us
I20260709 12:58:02.363682 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.361419 17643 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:58:02.411553 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:02.418603 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.398947 17719 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:02.397373 17816 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:02.422013 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.433728 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:02.434885 13619 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:58:02.441360 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:02.428668 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65061.1us
I20260709 12:58:02.450299 13537 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:02.413117 17815 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.453215 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:02.454720 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:02.456351 13543 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:02.458144 13686 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:02.419603 17757 raft_consensus.cc:3055] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:02.423221 17686 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:58:02.459698 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:02.461355 13687 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:02.462036 13611 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:58:02.462399 13611 pending_rounds.cc:85] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 8
I20260709 12:58:02.462626 13611 pending_rounds.cc:107] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:58:02.462527 13613 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:02.463009 13611 raft_consensus.cc:2889] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:02.448554 17719 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:02.465911 13506 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.429505 17795 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:02.471277 13539 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:58:02.471385 17795 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:02.418968 17701 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.438002 17783 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:02.468603 13619 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:58:02.472007 17795 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:02.476867 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 710) took 1233 ms. Trace:
I20260709 12:58:02.472870 17784 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:02.460935 17765 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:02.478938 17784 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:02.477370 13616 rpcz_store.cc:276] 0709 12:58:01.243393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:01.442318 (+198925us) service_pool.cc:224] Handling call
0709 12:58:01.442609 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:01.498482 (+ 55873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:01.498514 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:02.067934 (+569420us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b2c000676f8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:58:02.068325 (+   391us) log.cc:844] Serialized 26 byte log entry
0709 12:58:02.452460 (+384135us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:02.452524 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:02.452615 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:02.473345 (+ 20730us) raft_consensus.cc:1685] finished
0709 12:58:02.473364 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:02.476774 (+  3410us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55506,"raft.queue_time_us":55925,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":7171,"spinlock_wait_cycles":8278400,"thread_start_us":364624,"threads_started":1,"wal-append.queue_time_us":383851}
I20260709 12:58:02.479410 17784 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:02.480675 13612 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:02.460227 17586 raft_consensus.cc:2764] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:58:02.485251 13611 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:58:02.488575 13614 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:58:02.483052 17795 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:02.477404 17416 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:02.499418 13619 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:02.501842 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.486387 17390 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.507683 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.504278 17416 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:02.511622 17783 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:02.513545 17416 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:02.509186 17841 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:02.475072 17719 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:02.515897 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 104432us
I20260709 12:58:02.519232 17783 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:02.530463 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 70839.9us
I20260709 12:58:02.515614 17843 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:02.515570 17686 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:02.533162 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.537050 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.469501 17757 raft_consensus.cc:740] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:02.534796 13534 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:58:02.525939 17775 consensus_meta.cc:287] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.001s
I20260709 12:58:02.540292 13614 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 12: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:02.541189 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.519579 17729 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:02.497241 17816 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:02.546139 13584 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:02.549435 13584 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:02.550268 17586 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:02.550695 17543 raft_consensus.cc:2764] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:58:02.551211 17701 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.601073 17543 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:58:02.601956 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52752.1us
I20260709 12:58:02.602677 17701 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.603672 17701 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.604571 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:02.605870 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:02.606120 17701 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.606679 13534 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:02.558378 13437 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 3 to 10, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 10 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
W20260709 12:58:02.560060 17839 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:58:02.607043 17543 consensus_queue.cc:1059] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:02.608880 13534 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7. Ops: []
I20260709 12:58:02.562882 17813 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.569331 17719 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.578737 17419 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.581756 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:02.589885 17381 consensus_meta.cc:287] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.002s
I20260709 12:58:02.590503 13685 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:58:02.547434 17775 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.592669 17815 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.529116 17393 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.660203 17393 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.600049 17670 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.663066 17670 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.665246 17765 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:02.665967 17765 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:02.603039 17563 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.608531 13437 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 7 to 9, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 9 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:02.548226 17845 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:02.555145 17686 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.610909 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.624123 14241 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:58:02.624747 14246 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:02.626494 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.675613 13615 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:02.677260 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.678476 13615 raft_consensus.cc:2410] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:58:02.634879 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:02.618758 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:02.637090 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.628870 17642 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:58:02.625267 17721 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:58:02.646652 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.646845 14243 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:58:02.684157 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 73337.3us
I20260709 12:58:02.644613 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.651800 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.680382 14240 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:02.685122 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.656481 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:02.657430 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:02.686173 13688 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:02.686452 14244 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:02.658988 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:02.686618 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.687343 13686 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 8.
I20260709 12:58:02.662380 17393 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:02.665622 17670 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.671144 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.616899 17812 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:02.695541 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 192594us
I20260709 12:58:02.655400 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.689172 13691 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:02.672664 17757 consensus_queue.cc:260] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.704879 13584 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.690739 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.692478 13612 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:58:02.692339 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:02.693032 13540 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:58:02.620744 17784 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.695789 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:02.709524 14238 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:02.709789 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.613998 17763 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.711328 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.616344 17274 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.711874 17763 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.713198 17763 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:58:02.714049 17763 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:02.712149 13611 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:58:02.715065 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:02.712556 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:02.718812 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.718958 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.719731 13535 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:02.720063 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.720054 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:02.720952 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:02.720953 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.722160 13692 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:02.723117 13535 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:02.689210 13659 leader_election.cc:400] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 6 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 6. Current term is 7.
I20260709 12:58:02.723819 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.724704 13659 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.725701 17393 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:02.724752 13535 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:58:02.727051 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.728211 13684 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:58:02.728255 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.728579 13684 pending_rounds.cc:85] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:58:02.729833 13684 pending_rounds.cc:107] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:58:02.730841 13684 raft_consensus.cc:2889] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:02.735159 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:02.741987 13684 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:02.730307 13584 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:02.747004 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 111766us
I20260709 12:58:02.731462 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.726084 13659 leader_election.cc:400] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 8. Current term is 9.
I20260709 12:58:02.740576 17815 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.736508 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:02.753918 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:02.755322 13506 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:02.756821 13693 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:02.694546 17563 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:02.761462 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:02.693621 17419 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:02.773885 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:02.749310 13437 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 8 to 9, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:02.768095 13691 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:02.761452 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.771173 14239 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:02.777274 14238 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:58:02.770709 17785 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:58:02.760277 17777 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:02.778111 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:02.767278 17556 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:02.778599 17785 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:58:02.778900 14237 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:02.779421 17785 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:02.780243 17757 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:02.780332 17785 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.780934 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.781668 17785 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:02.783237 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:02.784081 17813 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:02.784294 17556 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:02.784960 17556 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:02.785403 17784 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:02.786194 17721 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:02.786763 17721 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.787243 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.696614 17803 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:02.787679 13616 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:02.811167 13616 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:02.670547 17381 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.793958 14246 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:02.699353 17812 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.767439 17607 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:02.770100 17393 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:58:02.684019 17759 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:58:02.787698 17784 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:02.930612 13437 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:02.800206 17728 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:02.771319 13612 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:58:02.809587 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:02.946182 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 136664us
I20260709 12:58:02.948145 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.783712 17743 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:58:02.787169 17807 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:58:02.783146 17416 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.816355 13692 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
W20260709 12:58:02.926715 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 139525us
I20260709 12:58:02.928985 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:02.925711 13616 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:02.959668 13616 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:02.768247 17755 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:02.943714 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.965243 13542 raft_consensus.cc:1240] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:02.772751 17419 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:02.813594 17781 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:02.973348 13612 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:58:02.768402 17815 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:02.967051 13610 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:02.969201 13437 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 3 to 6, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 6 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:02.968019 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:02.965193 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.950603 17784 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:58:02.944882 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 282352us
W20260709 12:58:03.000388 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 271862us
I20260709 12:58:02.960538 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:02.993477 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1616) took 1173 ms. Trace:
I20260709 12:58:03.001112 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:02.964273 17855 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:58:02.953756 17807 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:02.966100 17709 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:03.009727 17709 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:02.993376 17813 consensus_queue.cc:1059] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:58:03.010466 17863 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:58:03.011003 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:38540 (request call id 6) took 1089 ms. Trace:
I20260709 12:58:03.011461 14245 rpcz_store.cc:276] 0709 12:58:01.922043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:01.923122 (+  1079us) service_pool.cc:224] Handling call
0709 12:58:03.010792 (+1087670us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540005fd54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ac853 00007fa03377a60a 00007fa02cd007d4 00007fa02cd00762 00007fa02cd00691 00007fa02cd0060c 00007fa02ccff8c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:03.010912 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":795669888}
I20260709 12:58:03.012403 17274 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:02.994153 17743 raft_consensus.cc:3055] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:58:03.012900 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.013300 17743 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:03.013937 13612 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.017361 13617 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:58:03.006212 17621 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:58:03.018607 13441 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49896 (request call id 304) took 25607 ms (client timeout 15000 ms). Trace:
I20260709 12:58:02.967325 17803 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:03.022594 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.023337 13541 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:02.974606 17419 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:02.984256 13614 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:02.976718 17643 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:02.971307 17800 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:03.007431 13539 raft_consensus.cc:2410] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:58:03.007543 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 47104.2us
I20260709 12:58:03.020139 17381 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:03.009729 13437 catalog_manager.cc:5697] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:03.062634 13437 catalog_manager.cc:5697] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:02.945269 17777 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:03.025604 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.033252 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.001417 13685 rpcz_store.cc:276] 0709 12:58:01.819944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:01.948479 (+128535us) service_pool.cc:224] Handling call
0709 12:58:01.948777 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:02.667991 (+719214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:02.993184 (+325193us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b540015ead4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:02.993410 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475182976}
I20260709 12:58:02.977056 17759 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:03.038956 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.043283 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.071561 13613 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:03.072844 13437 catalog_manager.cc:5697] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 3 to 11, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 11 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
W20260709 12:58:03.060724 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 112644us
I20260709 12:58:03.010174 17709 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.008018 17784 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.039650 13535 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:03.026063 17556 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:03.084224 17556 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:03.085121 17851 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:58:03.086833 17556 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:02.974355 17845 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:58:03.081502 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.088569 13536 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:03.088788 17812 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:02.999289 17816 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:58:03.083032 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.073393 17721 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.094179 17709 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:03.094892 17853 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:58:03.095429 13692 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:58:03.096546 13437 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 7 to 8, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 8 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:03.100557 13541 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:03.038944 17643 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:03.037608 17781 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:03.170938 13437 catalog_manager.cc:5697] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 2 to 9. New cstate: current_term: 9 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
W20260709 12:58:03.172832 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 89342.8us
I20260709 12:58:03.074260 17729 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:03.019775 17765 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:03.185089 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.191067 13536 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:03.193317 13437 catalog_manager.cc:5697] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 5 to 7, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 7 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
W20260709 12:58:03.192426 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.199627 13541 pending_rounds.cc:85] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 6
I20260709 12:58:03.200086 13541 pending_rounds.cc:107] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:58:03.200415 13541 raft_consensus.cc:2889] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:03.175470 17643 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.025597 17743 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.205744 13611 raft_consensus.cc:1240] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: []
W20260709 12:58:03.206540 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 228075us
I20260709 12:58:03.203159 13437 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 4 to 11, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:03.207239 13508 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:03.208457 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:03.209631 13508 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:03.074576 17393 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:58:03.182576 17729 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.188863 17775 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:03.106171 17803 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 12:58:03.222255 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:03.226428 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.189395 17812 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:58:03.233058 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.177739 17781 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.220022 17813 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:58:03.018999 13441 rpcz_store.cc:269] 0709 12:57:37.209103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:37.328078 (+118975us) service_pool.cc:224] Handling call
0709 12:58:02.775419 (+25447341us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:57:54.831512 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:57:55.013664 (+182152us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:57:55.013699 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:57:55.013716 (+    17us) tablet.cc:662] Decoding operations
0709 12:57:55.014720 (+  1004us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:57:55.014794 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 12:57:55.014821 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:57:55.185064 (+170243us) tablet.cc:701] Row locks acquired
0709 12:57:55.185092 (+    28us) write_op.cc:260] PREPARE: finished
0709 12:57:55.185288 (+   196us) write_op.cc:270] Start()
0709 12:57:55.185436 (+   148us) write_op.cc:276] Timestamp: P: 1783601875185231 usec, L: 0
0709 12:57:55.185470 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:57:55.318066 (+132596us) log.cc:844] Serialized 4375 byte log entry
0709 12:57:56.100103 (+782037us) op_driver.cc:464] REPLICATION: finished
0709 12:57:56.180032 (+ 79929us) write_op.cc:301] APPLY: starting
0709 12:57:56.180118 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:57:56.181042 (+   924us) tablet.cc:1370] finished BulkCheckPresence
0709 12:57:56.184167 (+  3125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:57:56.185339 (+  1172us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:57:56.185625 (+   286us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 12:57:56.185959 (+   334us) write_op.cc:312] APPLY: finished
0709 12:57:56.186286 (+   327us) log.cc:844] Serialized 83 byte log entry
0709 12:57:56.186465 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:56.186610 (+   145us) write_op.cc:454] Released schema lock
0709 12:57:56.186663 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:57:56.194706 (+  8043us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540041e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ade28 00007fa0321e8581 00007fa0321e7d58 00007fa0321ec9f1 00007fa0321ec9a9 00007fa0321ec939 00007fa0321ec901 00007fa0321ebb2d 000000000038f354 000000000038f199
0709 12:57:56.195039 (+   333us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:57:56.195082 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10917,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":78668,"apply.run_cpu_time_us":3909,"apply.run_wall_time_us":7064,"mutex_wait_us":8365,"num_ops":11,"prepare.queue_time_us":218985,"prepare.run_cpu_time_us":16910,"prepare.run_wall_time_us":1395346,"raft.queue_time_us":186447,"raft.run_cpu_time_us":13925,"raft.run_wall_time_us":75548,"replication_time_us":914392,"spinlock_wait_cycles":241778944,"thread_start_us":449730,"threads_started":4,"wal-append.queue_time_us":172280,"wal-append.run_cpu_time_us":21169,"wal-append.run_wall_time_us":1734987}]]}
I20260709 12:58:03.240134 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.242988 13610 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
W20260709 12:58:03.240451 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 259461us
I20260709 12:58:03.240952 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:03.220525 17847 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:03.244378 14209 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:03.203192 17274 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 12:58:03.253258 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.245303 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:03.254215 13686 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:03.227895 17729 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.225914 17867 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:03.249583 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.236454 17607 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:03.243155 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:03.224114 17563 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
W20260709 12:58:03.256035 17865 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:58:03.255880 17759 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:03.279871 17765 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:03.284626 17860 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:58:03.289075 17864 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:58:03.250352 17862 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:58:03.256742 17329 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:03.291600 17329 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:03.257436 17781 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.261395 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:03.261631 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:03.249394 17812 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:03.245311 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.248356 13437 catalog_manager.cc:5697] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 6 to 8, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 8 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:03.295326 13612 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
W20260709 12:58:03.265451 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 230268us
I20260709 12:58:03.246505 17621 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:03.229863 17799 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:03.236893 17743 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:58:03.250871 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:03.267956 13617 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
W20260709 12:58:03.304771 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 53984.4us
I20260709 12:58:03.297362 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:03.238667 17846 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:03.267168 17416 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.278204 13537 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:58:03.245977 17813 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:58:03.262849 17795 consensus_meta.cc:287] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.004s
I20260709 12:58:03.251056 17775 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:03.297367 13619 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:58:03.291831 17563 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:03.267478 17803 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:03.300482 13437 catalog_manager.cc:5697] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:03.269624 17607 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.278035 17815 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:03.307421 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:03.294268 17329 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.324505 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:03.315624 17862 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:03.308897 17799 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.336086 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:03.349009 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:03.338742 13437 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 8 to 9, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 9 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:03.324608 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.319177 17813 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:03.274849 17867 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:03.335985 17584 consensus_meta.cc:287] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.915s	user 0.013s	sys 0.000s
I20260709 12:58:03.335590 17607 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.299993 17866 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:58:03.358122 17862 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:03.358282 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.349838 17869 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:03.324968 17803 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:03.305068 17812 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.357545 13616 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 12:58:03.380141 13616 pending_rounds.cc:85] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 4
I20260709 12:58:03.380412 13616 pending_rounds.cc:107] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:58:03.380645 13616 raft_consensus.cc:2889] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:58:03.354271 17785 consensus_meta.cc:287] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.009s
I20260709 12:58:03.372028 13437 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 5 to 9, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:03.325592 17563 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:03.352312 17274 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:03.389215 17274 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.391479 17274 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:03.392650 17868 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.149s	user 0.002s	sys 0.002s
W20260709 12:58:03.393167 17763 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:58:03.318586 17846 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.398036 17563 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.399595 17563 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:03.401326 17867 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.404597 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:03.404881 17846 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:03.405391 17846 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:03.405877 17846 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:03.406693 17846 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.407248 13509 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:03.363086 17584 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.408797 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.409384 13535 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:03.410516 17584 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.334843 17621 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.414687 17755 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:03.415077 17755 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:03.415939 17755 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.417457 17774 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:03.417889 17774 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:03.387770 13437 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 8 to 11, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:03.418632 17774 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.419198 13437 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 4 to 5, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 5 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:03.420696 17778 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:03.421087 17778 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:03.421911 17778 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.423023 17778 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.425634 17799 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.426904 17803 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.428079 17274 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:03.428153 17803 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:03.428583 17274 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.429076 17274 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:03.430590 13685 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:03.433225 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.326367 17795 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.436935 13613 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:03.438273 17795 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.419824 17774 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.355894 17709 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:03.440383 17719 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:03.440776 17719 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:03.441339 17862 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.441615 17719 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.442670 17719 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:03.445816 17419 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.448024 13539 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:03.361086 17872 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:03.448683 17872 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.449044 17872 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:03.448115 13690 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:03.378254 17875 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:03.455021 17875 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:03.455422 17875 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:03.455672 17381 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:03.456220 17381 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:03.456854 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.457610 13534 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:03.457260 17381 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.463982 17755 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.464951 17728 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:03.465472 17728 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:03.410567 13535 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:58:03.472980 17785 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:03.472710 17728 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.473563 17785 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.474033 17728 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:03.488623 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 447254us
I20260709 12:58:03.494086 17785 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:03.476217 17757 consensus_meta.cc:287] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.695s	user 0.009s	sys 0.003s
I20260709 12:58:03.497653 13618 raft_consensus.cc:2410] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:58:03.431667 17812 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:03.383563 17869 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.500214 17869 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:03.502281 17807 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.507143 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.514321 17800 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:03.515097 17800 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.503897 17807 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:03.533824 13543 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
W20260709 12:58:03.535116 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 127945us
I20260709 12:58:03.536048 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:03.537586 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.546478 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:03.545128 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 490015us
I20260709 12:58:03.547503 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:03.567159 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.580159 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.543947 17843 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:03.584686 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.599292 13542 raft_consensus.cc:2393] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 12: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:03.587384 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:03.592294 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.601044 13540 raft_consensus.cc:2410] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:58:03.595674 17419 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:03.596989 17815 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:03.570122 17670 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:03.585850 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:03.614784 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47718.7us
I20260709 12:58:03.618851 13537 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 12: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:03.621232 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.624213 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.625308 13537 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:58:03.637138 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 86509us
I20260709 12:58:03.638016 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.617017 13509 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:03.645319 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.646095 13618 raft_consensus.cc:2393] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 12: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:03.647248 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.647473 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.648092 17799 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:03.648510 17658 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:03.648665 17799 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:03.648960 17658 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.649376 17658 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:03.649690 17799 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.652235 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:03.652526 13685 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:03.651907 13613 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:03.653501 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:03.573839 17584 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:58:03.657491 13689 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:58:03.657876 13689 pending_rounds.cc:85] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 4
I20260709 12:58:03.658087 13689 pending_rounds.cc:107] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:58:03.658311 13689 raft_consensus.cc:2889] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:03.659106 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.643697 17701 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:58:03.661823 13618 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:58:03.647418 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:03.644285 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.618587 17670 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:03.670259 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 320404us
I20260709 12:58:03.670655 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.679116 13537 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
W20260709 12:58:03.671870 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 347203us
I20260709 12:58:03.673604 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:03.674260 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.685083 13619 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:03.674401 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.689248 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.689950 13615 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:03.675143 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:03.692159 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.692633 13616 raft_consensus.cc:2410] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 12 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:58:03.693814 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.694532 13616 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:03.695420 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.705914 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:03.677840 17643 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:03.713757 17784 consensus_meta.cc:287] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 12:58:03.713464 17607 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.713865 17875 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.714200 17784 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.714803 17643 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.715909 14244 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
W20260709 12:58:03.716838 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:03.717419 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 78079.4us
W20260709 12:58:03.717660 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 44139.4us
I20260709 12:58:03.718070 13610 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:58:03.718600 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.720633 13619 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:03.722947 13543 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
W20260709 12:58:03.724432 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 176981us
I20260709 12:58:03.717464 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.718222 14211 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:03.729624 13618 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:03.718802 13615 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:03.728498 17876 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:03.735114 17876 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.736291 17871 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:03.737389 17876 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.735114 17775 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:58:03.742247 13610 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:58:03.728205 17843 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:03.715824 17816 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:58:03.744545 13509 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:03.744930 17816 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:03.745183 13611 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:58:03.745340 17816 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:03.748005 13692 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:58:03.748337 13692 pending_rounds.cc:85] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:58:03.748535 13692 pending_rounds.cc:107] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:58:03.748734 13692 raft_consensus.cc:2889] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:03.749769 13536 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 7. Current term is 8. Ops: []
I20260709 12:58:03.744310 17843 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:03.759123 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:03.759982 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:03.769182 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.773411 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:03.774058 13540 raft_consensus.cc:606] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:03.778075 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.715700 17670 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:58:03.792359 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 63194us
I20260709 12:58:03.728781 17850 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:03.746989 17390 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:58:03.802138 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 117727us
W20260709 12:58:03.795748 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 77588.2us
I20260709 12:58:03.797967 13612 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:03.799373 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1697) took 1045 ms. Trace:
I20260709 12:58:03.803982 13536 rpcz_store.cc:276] 0709 12:58:02.754087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:03.196603 (+442516us) service_pool.cc:224] Handling call
0709 12:58:03.224675 (+ 28072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:03.750195 (+525520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:03.799103 (+ 48908us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b540028b754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:03.799304 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207042432}
W20260709 12:58:03.819672 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:03.820137 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:03.820441 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.822643 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:03.823125 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:03.823053 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:03.775534 17843 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.824878 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:03.773267 17584 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:03.825052 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:03.825873 13692 raft_consensus.cc:1240] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:58:03.826133 13687 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:03.827507 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.834748 13691 raft_consensus.cc:2410] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:58:03.836077 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:03.830660 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.843569 17274 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.846692 17274 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:03.850383 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 72401us
I20260709 12:58:03.853140 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:03.854261 13612 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 12: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:03.728729 17841 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:58:03.860502 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:03.803246 17850 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.866106 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:03.819284 17390 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.818217 17642 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:03.861936 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:03.858572 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:03.775240 17757 consensus_queue.cc:1059] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:58:03.866725 14246 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.752193 17729 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.880214 17584 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:58:03.881088 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 77102us
I20260709 12:58:03.881558 17729 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:03.881213 17584 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.881812 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:58:03.867614 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70156.2us
I20260709 12:58:03.883169 14211 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:03.883518 17728 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:58:03.884168 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 92015.4us
I20260709 12:58:03.884776 17887 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:58:03.885150 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.885679 17887 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:03.801952 17607 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:03.886053 17888 raft_consensus.cc:3055] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:03.849004 17583 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:03.886518 17888 raft_consensus.cc:740] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:03.786576 17777 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:58:03.869395 17850 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.888327 17777 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:03.888708 17841 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:58:03.889196 14246 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:03.890872 14242 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:58:03.891294 17775 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:58:03.884166 17876 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:03.884809 17874 consensus_queue.cc:1059] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:58:03.892681 17876 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:03.892591 17869 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.893689 17876 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.894833 17869 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:03.838690 17743 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:58:03.891686 17775 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:03.897063 17775 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.898195 17775 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:03.900218 13686 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:03.900878 17874 raft_consensus.cc:3055] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:03.901577 17874 raft_consensus.cc:740] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:03.901899 17784 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.902594 17874 consensus_queue.cc:260] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.795609 17556 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:58:03.873134 17875 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:03.904476 17777 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:58:03.905952 17583 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:58:03.900856 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:03.907783 17784 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:58:03.908295 17784 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:03.909111 17784 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.910215 17784 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:03.910427 17872 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.911548 13690 raft_consensus.cc:2368] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13.
W20260709 12:58:03.914338 17813 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.540s	user 0.001s	sys 0.014s
I20260709 12:58:03.914875 17813 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.916085 17642 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:03.917058 17642 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:03.920171 13691 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:58:03.923266 13692 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:58:03.875465 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:03.864431 17274 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:03.876274 17757 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:03.829311 17883 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
W20260709 12:58:03.877153 17329 consensus_meta.cc:287] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.005s
I20260709 12:58:03.873989 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:03.933743 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:03.931136 17702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.014s	sys 0.009s
W20260709 12:58:03.876233 17886 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:58:03.872792 17803 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:58:03.875121 17390 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:03.914433 17800 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:58:03.946319 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70928.7us
I20260709 12:58:03.930186 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:03.831049 17816 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:03.936553 17870 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:03.954398 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 80496us
I20260709 12:58:04.043694 17846 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:58:03.977448 17702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.017s	sys 0.011s
I20260709 12:58:03.939503 17880 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:03.971377 17743 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:03.988034 17803 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:03.989099 17872 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:03.991097 17563 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.000s
I20260709 12:58:03.993366 17583 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:03.954195 17329 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.000401 17607 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:03.956310 17621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.016s
I20260709 12:58:03.946411 17757 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:04.030431 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:04.117735 17743 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:03.887346 17888 consensus_queue.cc:260] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.004786 17889 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:04.078399 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 148321us
I20260709 12:58:04.078652 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:04.048632 17381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.020s	sys 0.022s
I20260709 12:58:04.096027 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:04.153980 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 75389.7us
W20260709 12:58:04.154541 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 58647.9us
I20260709 12:58:04.155642 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:03.954968 13691 raft_consensus.cc:2368] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
W20260709 12:58:04.108438 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 225388us
I20260709 12:58:04.112251 13616 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:04.160385 17869 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:58:04.153985 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 294331us
I20260709 12:58:04.107486 17870 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:04.137377 17621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.008s	sys 0.017s
I20260709 12:58:04.165619 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.166388 13613 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:04.156927 17381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.021s	sys 0.023s
W20260709 12:58:04.115818 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 263970us
I20260709 12:58:04.153076 17743 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:04.173342 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:04.148420 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:04.036906 17881 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:58:04.166364 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:04.107208 17815 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:58:04.080214 17658 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.106431 17274 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:58:04.121042 17803 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:04.159475 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:04.185626 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:04.118592 17846 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.093564 17755 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:04.094157 17543 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:04.135990 13688 raft_consensus.cc:2410] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:58:04.167629 17680 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:04.122371 17813 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.162552 17563 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:04.162285 17872 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:04.174791 17889 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:04.165269 17867 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 12:58:04.180315 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 297180us
I20260709 12:58:04.168637 17774 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:04.148512 17607 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.192551 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.182775 17890 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:04.178351 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:04.220831 14242 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 13: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:04.165903 17729 consensus_queue.cc:1059] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:58:04.208443 13613 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:04.179399 17869 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:04.164417 17778 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:58:04.194608 17274 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:04.223367 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.228895 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.206197 17755 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:04.241740 13537 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:58:04.245235 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:04.221052 17872 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:04.179384 17757 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.215142 17889 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.204725 17680 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.247785 17778 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:04.251214 17889 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.251493 17680 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.251794 17870 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.253252 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:04.231485 17274 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:04.265195 17274 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:04.266033 13536 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:04.267488 17621 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:04.268277 17621 consensus_queue.cc:1243] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:58:04.268287 17755 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.272286 17757 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:04.273386 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:04.274215 13537 raft_consensus.cc:2393] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 11: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:04.274447 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:04.275166 13543 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:04.242818 17906 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:04.276373 17906 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:04.276903 17906 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:04.280861 17329 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:04.230796 17890 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.199959 17543 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.284512 17890 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.286175 17543 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.287003 17543 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:04.252352 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:04.262199 17658 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:04.232033 17869 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.288558 17607 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.289248 17607 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:04.289487 17874 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:04.289880 17869 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.289723 17607 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.219506 17774 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.230268 17803 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:04.264690 17900 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:58:04.291970 17774 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.234969 17862 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:04.292954 17872 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:04.302613 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 118371us
I20260709 12:58:04.287559 17543 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:04.305696 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.306777 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.252635 17800 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:04.325557 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:04.329764 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.342036 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:04.342665 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:04.343535 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:04.351159 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.353868 17543 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.355479 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:04.359987 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:04.360140 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:04.363862 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:04.364392 17914 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:04.364622 17774 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:04.365097 17774 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:04.364929 17914 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:04.365679 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:04.367137 17914 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.304538 17390 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.370608 17390 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:04.300840 17670 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.372941 14241 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:04.373394 17274 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.373950 17670 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:04.370534 17607 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:04.379582 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 194035us
I20260709 12:58:04.382890 14242 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
W20260709 12:58:04.384560 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 161259us
I20260709 12:58:04.388041 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:04.388244 17889 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:04.376415 13613 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:58:04.376899 14244 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:04.378953 13615 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:58:04.394408 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 50096.2us
I20260709 12:58:04.395324 13611 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:04.423547 14246 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:58:04.462534 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:04.476336 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:04.480222 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.531088 17816 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:04.531692 17880 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:04.503823 17719 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:04.532730 13617 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:04.439888 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:04.398223 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:04.440515 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 52578.4us
I20260709 12:58:04.535153 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:04.509462 13613 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:58:04.529698 14242 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:04.470989 17846 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:04.537247 17709 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:04.396813 17918 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:04.537793 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:04.538751 17884 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.801s	user 0.001s	sys 0.006s
I20260709 12:58:04.540535 17918 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.536983 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.542659 17918 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:04.511381 17850 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.001s
I20260709 12:58:04.542786 17906 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.532312 17719 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:04.543486 17850 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.544771 13536 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 11: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:04.545645 17850 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.543793 13613 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:04.460764 17774 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.550259 17719 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.551554 17719 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:04.505267 17921 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:04.552441 17921 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.555846 17921 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:04.409345 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 466712us
I20260709 12:58:04.422878 13509 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.559330 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:04.560451 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.565136 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:04.445151 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.561256 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:04.568037 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 122528us
I20260709 12:58:04.409935 17274 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:04.568964 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:04.505038 17843 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.002s
I20260709 12:58:04.568605 13619 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 12:58:04.469486 17784 consensus_meta.cc:287] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.007s
I20260709 12:58:04.571177 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:04.574573 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 151808us
I20260709 12:58:04.577983 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:04.575527 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:04.460731 17778 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:58:04.415818 17775 consensus_meta.cc:287] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.517s	user 0.014s	sys 0.000s
I20260709 12:58:04.576001 17867 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:04.444730 17757 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:04.395737 17914 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:04.570180 17643 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:04.532271 17880 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.582325 17843 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.584087 17906 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:04.585033 14245 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:04.587431 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:04.588186 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.595896 17757 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.596225 17921 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:58:04.573088 17918 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:04.605507 13658 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.573436 17890 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:04.608090 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.598874 13506 leader_election.cc:400] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:04.619427 13506 leader_election.cc:403] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:58:04.589938 17846 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:04.625408 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:04.595968 17910 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:04.626252 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:04.626276 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:04.628413 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:04.643772 13688 raft_consensus.cc:2376] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9. Re-sending same reply.
I20260709 12:58:04.607774 17274 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:04.629267 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:04.596601 17799 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:58:04.625586 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:04.651218 13689 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:04.639295 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:04.652843 13619 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:04.643883 13685 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:04.629146 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:04.609400 17920 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:04.647063 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.569097 17925 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:58:04.604022 17778 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:58:04.620826 17775 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:04.653808 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:04.649214 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 61853.3us
I20260709 12:58:04.596026 17919 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:58:04.660785 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.604113 17914 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.599081 17867 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.644860 17643 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:04.670060 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:04.635556 17870 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:58:04.675889 13690 raft_consensus.cc:2368] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12.
I20260709 12:58:04.671166 17890 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:04.659462 17563 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:04.658763 17813 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:04.621681 17918 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.689751 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:04.664641 17846 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:04.668385 17778 consensus_queue.cc:1243] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:58:04.694265 17803 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:04.676736 17670 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:04.692695 17890 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:04.652109 17743 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.741524 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:04.741783 13690 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:58:04.742127 13690 pending_rounds.cc:85] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 5
I20260709 12:58:04.742364 13690 pending_rounds.cc:107] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:58:04.742612 13690 raft_consensus.cc:2889] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:04.758056 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:04.660737 17920 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.763526 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:04.771420 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 155036us
I20260709 12:58:04.770066 13684 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:04.822541 13542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:04.823173 13542 raft_consensus.cc:606] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:04.823531 13542 raft_consensus.cc:614] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:04.836229 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:04.763250 17781 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:04.740521 17846 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:04.842046 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:04.848265 13614 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:04.834770 17816 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:04.859503 17709 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:04.860097 17709 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:04.865494 17893 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.867s	user 0.003s	sys 0.000s
W20260709 12:58:04.866909 17895 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.762s	user 0.001s	sys 0.001s
I20260709 12:58:04.871073 13617 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:04.871141 17888 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:04.852823 17816 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:04.872452 17934 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:58:04.874264 17816 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:04.875123 13685 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:58:04.875326 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.880400 13542 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:58:04.880750 14241 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:58:04.881464 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:04.882166 13617 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:04.883016 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:04.882117 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:04.885591 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:04.885684 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:04.886132 13542 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:58:04.886420 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:04.886324 13535 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:04.903537 17945 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:58:04.887338 17781 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:04.903643 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:04.910329 13540 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:04.909262 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:58:04.914341 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:04.909137 17889 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:04.917476 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 185337us
I20260709 12:58:04.889423 17843 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:04.923666 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 45262.6us
W20260709 12:58:04.924595 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:04.916117 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:04.924626 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.926194 13617 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:04.928634 13614 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:58:04.911593 17880 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:04.933341 13582 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:04.943987 17914 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:04.945127 17781 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:04.950392 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:04.952100 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:04.928995 17940 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:58:04.954912 14238 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:04.928840 17939 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:04.958549 13582 leader_election.cc:400] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13.
I20260709 12:58:04.959599 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:04.968916 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:04.967562 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:04.970300 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:04.970458 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:04.970621 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:04.945082 17743 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:04.969527 13690 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:04.973670 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:04.974489 14239 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:04.971758 13582 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:04.981427 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.007036 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.007756 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:04.955477 17921 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:04.960954 17914 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:05.015842 14209 leader_election.cc:400] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12.
I20260709 12:58:05.014669 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:05.022022 14209 leader_election.cc:403] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:58:04.945071 17686 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:58:05.018509 17670 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:05.018651 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.013433 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:05.024957 14239 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:58:05.029982 14209 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.033224 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.012048 17923 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:05.035454 13540 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:05.011695 17658 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:58:05.006356 17729 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:05.006539 17918 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.044422 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:05.011814 17939 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:05.038015 17719 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.034942 17906 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.046312 17670 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:05.063259 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:05.064791 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 82071us
I20260709 12:58:05.070343 17709 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.079205 17709 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:05.072149 17939 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.072827 17920 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.065318 17846 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.076830 17843 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:05.071048 17813 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:05.065798 13582 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.058357 17584 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.062740 17680 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.058318 17658 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:05.053063 17879 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:58:05.088212 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.086741 17543 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:58:05.096748 17955 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:58:05.105996 17899 log.cc:927] Time spent T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.953s	user 0.001s	sys 0.001s
W20260709 12:58:05.096758 17948 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:58:05.114039 17658 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:05.115487 17894 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.933s	user 0.001s	sys 0.001s
I20260709 12:58:05.090269 17584 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:05.118675 17944 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:58:05.119220 17584 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.090250 17680 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.122666 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:05.123451 17701 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:05.127636 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:05.128050 17381 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:05.128589 17381 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.130879 17867 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.131551 17381 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.134449 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.135291 13536 raft_consensus.cc:2393] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:05.136219 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:05.136623 17942 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:05.136874 14241 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:05.136263 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1812) took 1455 ms. Trace:
I20260709 12:58:05.137894 13616 rpcz_store.cc:276] 0709 12:58:03.680689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:03.707260 (+ 26571us) service_pool.cc:224] Handling call
0709 12:58:03.707647 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:04.160245 (+452598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:05.136060 (+975815us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540013bfd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:05.136210 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929997696}
I20260709 12:58:05.138808 17942 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.122810 17680 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.140190 17843 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.140525 17680 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:05.140725 17843 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.141085 17680 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.123126 17543 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:58:05.142553 17892 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.007s	user 0.000s	sys 0.003s
I20260709 12:58:05.142732 17543 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.143052 17843 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.146859 17889 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.147969 17882 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:05.148413 17882 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:05.149202 17882 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.150327 17882 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:05.151995 14212 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:58:05.153983 17898 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.997s	user 0.000s	sys 0.004s
I20260709 12:58:05.154605 17702 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:05.155169 17702 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:05.158481 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:05.161427 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:05.162071 13611 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.164052 17743 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:58:05.165670 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.168669 13535 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:05.168743 13540 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:05.169450 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:05.170043 13543 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.170168 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:05.170363 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:05.170751 13540 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.171581 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:05.124996 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:05.173094 13542 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:05.175086 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.176481 13616 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:58:05.178773 13611 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:05.180120 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.178779 13543 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:05.180923 17709 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:05.182031 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:05.182276 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.174621 13508 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:05.183106 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.184535 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.185104 13691 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:05.189101 13584 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:05.183671 13539 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:05.193912 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.195107 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.195528 13614 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:05.197345 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.198639 13543 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:05.199271 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.203078 13543 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:05.199144 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:05.205713 13543 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:58:05.173929 17919 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.219516 13540 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:05.220649 13536 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:58:05.222221 13536 pending_rounds.cc:85] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 6
I20260709 12:58:05.222393 13536 pending_rounds.cc:107] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:58:05.222608 13536 raft_consensus.cc:2889] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:05.192978 17709 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:05.223677 17709 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:05.224064 17778 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:05.224599 17951 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.224572 17709 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.225267 17951 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.197950 17918 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:05.197510 17914 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:05.202009 17880 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.228611 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:05.197054 17584 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:05.198521 17906 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12.
I20260709 12:58:05.216454 17775 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.197650 17867 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:58:05.249176 17950 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.219070 17890 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:58:05.268122 14244 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 1125) took 1509 ms (client timeout 1790 ms). Trace:
I20260709 12:58:05.269749 17908 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:58:05.248924 17914 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:58:05.276532 14244 rpcz_store.cc:269] 0709 12:58:03.758597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:03.758761 (+   164us) service_pool.cc:224] Handling call
0709 12:58:05.267795 (+1509034us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54005cf3d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:05.268011 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1396698496}
I20260709 12:58:05.253103 17584 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.218128 17945 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:58:05.256131 17778 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:58:05.303088 17876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.273s	user 0.038s	sys 0.029s
I20260709 12:58:05.202073 17702 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.231716 17919 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.221256 17875 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:58:05.302745 17890 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:05.303614 17921 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:05.286294 17951 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.324707 17914 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:05.258152 17880 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.308423 17908 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:58:05.331985 17876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.317s	user 0.040s	sys 0.029s
I20260709 12:58:05.262113 17775 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.355374 14239 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:05.342012 17890 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:05.329665 17584 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:05.336738 17862 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:05.303504 17950 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.348347 17680 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.358392 17728 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:05.386739 17728 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:05.387588 17728 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.388746 17728 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:58:05.393411 17962 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:58:05.342406 17778 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:05.380553 17658 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.344933 17921 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:05.382092 17643 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.400588 17643 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.401667 17880 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.402195 17543 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.402698 17643 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:05.402879 17658 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.407622 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.408227 13612 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:05.409281 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.409780 14212 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.409966 13540 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:05.410234 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.411058 13611 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:05.412726 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.412832 13542 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:05.414181 13535 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:05.414757 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.413372 13536 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:05.383982 17862 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:05.413287 14212 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.399425 17583 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.414312 17658 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:05.432945 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:05.435976 17967 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:58:05.439116 17897 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 1.292s	user 0.003s	sys 0.000s
I20260709 12:58:05.431026 17803 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:05.444761 17803 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.418217 17883 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:05.449676 17950 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.422536 17719 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:58:05.452720 17937 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.575s	user 0.000s	sys 0.002s
I20260709 12:58:05.433490 17583 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.472880 17919 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.447592 17803 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:05.506633 17543 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.419193 17939 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:05.457636 17862 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.462990 17778 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.527204 13611 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:05.532398 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1578) took 1947 ms. Trace:
I20260709 12:58:05.539249 13534 rpcz_store.cc:276] 0709 12:58:03.585155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:03.585261 (+   106us) service_pool.cc:224] Handling call
0709 12:58:03.640081 (+ 54820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:03.640497 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:58:03.640539 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:03.644674 (+  4135us) log.cc:844] Serialized 26 byte log entry
0709 12:58:03.951413 (+306739us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:58:03.951452 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:03.951515 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:05.438458 (+1486943us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:05.529317 (+ 90859us) raft_consensus.cc:1685] finished
0709 12:58:05.529349 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:05.532325 (+  2976us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":6552,"spinlock_wait_cycles":82503040,"thread_start_us":306439,"threads_started":1,"wal-append.queue_time_us":464482}
I20260709 12:58:05.526610 17883 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.541744 17883 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:05.541868 17583 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.542778 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1821) took 1005 ms. Trace:
I20260709 12:58:05.543155 13618 rpcz_store.cc:276] 0709 12:58:04.559652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:04.560070 (+   418us) service_pool.cc:224] Handling call
0709 12:58:04.561664 (+  1594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:04.561970 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:04.561993 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:04.569949 (+  7956us) log.cc:844] Serialized 26 byte log entry
0709 12:58:04.666639 (+ 96690us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:58:04.666678 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:04.666741 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:05.450018 (+783277us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:05.542496 (+ 92478us) raft_consensus.cc:1685] finished
0709 12:58:05.542517 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:05.542720 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":484,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3333,"spinlock_wait_cycles":10407040,"thread_start_us":67490,"threads_started":1,"wal-append.queue_time_us":236869}
I20260709 12:58:05.417878 17908 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.528857 17719 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.542661 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:05.537636 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 346068us
I20260709 12:58:05.548314 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:05.551432 17921 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:58:05.551699 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.552409 13619 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
W20260709 12:58:05.554561 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:05.554940 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:05.556650 13688 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:05.559319 13506 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.549265 17719 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.562901 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.565064 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:05.552618 17909 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.365s	user 0.001s	sys 0.002s
I20260709 12:58:05.567984 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.564608 17680 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:05.569486 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.569774 17680 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:05.570271 17680 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:05.571203 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:05.561225 17945 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.573127 17945 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.575223 17945 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:05.564205 17919 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.582159 13688 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:58:05.545086 17939 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:05.585513 17939 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.587383 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.589622 17719 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.580271 17775 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.590237 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.550130 17862 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.591758 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.592412 13691 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:58:05.593417 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.594776 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.595810 17923 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:58:05.596309 17970 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:58:05.600867 13691 raft_consensus.cc:2410] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:58:05.602195 13688 pending_rounds.cc:85] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 5
I20260709 12:58:05.602041 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.602502 13688 pending_rounds.cc:107] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:58:05.604065 13688 raft_consensus.cc:2889] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:05.557826 17942 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:05.612653 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.566641 17921 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:05.615579 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.616386 13686 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:05.639739 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.640571 13684 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
W20260709 12:58:05.640609 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 45913.3us
I20260709 12:58:05.641914 17921 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:05.608999 17680 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.645756 14239 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:58:05.646616 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.665382 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:05.665884 13684 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:05.666164 13612 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.666632 14209 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.668787 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:05.669632 13616 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:05.670125 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:05.652276 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:05.673537 13618 raft_consensus.cc:2393] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:05.653929 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.676127 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.676959 13542 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:05.657122 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.663558 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 822) took 1425 ms. Trace:
I20260709 12:58:05.680240 13541 rpcz_store.cc:276] 0709 12:58:04.238499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:04.238688 (+   189us) service_pool.cc:224] Handling call
0709 12:58:04.239057 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:04.253134 (+ 14077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:04.253164 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:04.253209 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:04.253240 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:05.663300 (+1410060us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54000b81d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:05.663349 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:05.663497 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31900032}
I20260709 12:58:05.682369 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.683902 13582 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.684264 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.688062 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.645437 17719 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:05.691340 17960 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.692114 17643 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:05.692595 17643 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.692983 17643 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:05.694808 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.695526 13693 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:05.696272 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.696444 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:05.657860 17935 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.779s	user 0.002s	sys 0.000s
I20260709 12:58:05.697091 13693 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.689178 17921 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.662935 17701 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:05.702026 17701 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.644492 17973 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:05.704404 17973 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.707510 17973 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.708624 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.708786 17729 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:05.709860 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.710435 17914 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.711370 17775 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.714430 13611 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:05.715988 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.717082 17381 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:05.717515 17381 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.717919 17381 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:05.718005 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.718624 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:05.718884 17583 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:05.719358 17583 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.719703 17583 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:05.720317 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:05.720960 13685 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:05.721753 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:05.723265 17890 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.696925 13684 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:58:05.653982 17891 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.526s	user 0.004s	sys 0.000s
I20260709 12:58:05.739287 13506 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.745632 13612 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:05.747700 14212 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.753158 13685 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:05.648810 17680 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.640233 17923 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:05.647706 17964 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.231s	user 0.000s	sys 0.002s
I20260709 12:58:05.738564 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:05.733559 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49777.5us
I20260709 12:58:05.762403 13689 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:05.649801 17757 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.628527 17942 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.686842 17543 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:05.656347 17784 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 12:58:05.643800 17867 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 12:58:05.656219 17843 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:58:05.771168 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:05.764271 13612 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:05.780004 14239 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:05.659264 17883 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.783602 14242 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:05.760177 17719 raft_consensus.cc:2804] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:05.760527 17950 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:05.730885 17584 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.796308 17870 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:58:05.741655 17728 consensus_queue.cc:1059] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:58:05.804436 17883 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:05.806623 17584 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.782020 17680 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.808246 17680 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.813665 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:05.814457 13612 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:05.795639 14239 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:05.798106 17543 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:05.808238 17914 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.833837 13693 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:05.813315 17800 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:05.840564 13612 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:05.844710 14212 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:05.849380 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 109216us
I20260709 12:58:05.850757 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:05.854281 13534 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:58:05.855278 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.860658 14239 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:05.860836 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.865322 13582 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.874351 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.875747 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:05.865994 17728 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:58:05.884950 17680 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:05.890079 17978 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:58:05.904605 17942 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:05.850030 17729 raft_consensus.cc:2804] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:05.852677 17923 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:05.839905 17543 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:05.914157 17543 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:05.914829 17543 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.916805 14242 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:58:05.865844 17784 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.921025 17950 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:05.957840 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 71527.5us
I20260709 12:58:05.847390 17960 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:05.915335 17543 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:05.972465 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:05.878511 17584 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.933754 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:05.951453 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:05.965252 17914 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:05.961730 17862 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:58:05.987027 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 53413.1us
I20260709 12:58:05.963855 17729 raft_consensus.cc:697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:06.002255 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:06.010517 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.978952 17719 raft_consensus.cc:697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:05.878489 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:05.984905 17874 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:05.855854 17800 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.988912 17914 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:05.989075 17728 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:05.966998 17701 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.981256 17987 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:05.980898 17960 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:05.988445 17921 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.015887 17778 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:58:06.053007 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:05.982056 17973 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.018091 17942 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:06.060062 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 181666us
W20260709 12:58:06.062678 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 186170us
I20260709 12:58:06.065826 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:06.068420 17728 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:06.077775 17800 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.078150 17583 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.045087 17803 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:58:06.060153 17670 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:06.080164 17381 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.058054 17914 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:06.093338 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:06.078088 17701 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.119537 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:06.109829 17992 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:58:06.122912 17583 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.123237 17992 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:06.123704 17992 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:06.123818 17680 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:06.124121 17992 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:06.081952 17973 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.099350 17841 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:06.048182 17941 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:58:06.084944 17982 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:58:06.108804 17996 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:06.069074 17778 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:06.108433 17990 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:06.117389 17643 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.055676 17874 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.080336 17960 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.092475 17986 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:06.134779 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:06.117213 17800 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:06.102421 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:06.120647 17882 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:58:06.149850 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 47561.8us
I20260709 12:58:06.152817 13684 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:58:06.153337 13684 pending_rounds.cc:85] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 8
I20260709 12:58:06.153609 13684 pending_rounds.cc:107] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:58:06.153803 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.153884 13684 raft_consensus.cc:2889] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:06.109350 17728 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.141899 17997 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:06.157697 17941 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:06.137766 17841 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.141700 17990 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:06.173640 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54186.7us
I20260709 12:58:06.175168 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:06.069249 17729 consensus_queue.cc:237] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.160642 17543 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.161262 17381 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:06.161540 17984 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:58:06.139674 17778 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:06.118376 17670 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.172839 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.173524 13584 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:06.188864 13684 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 9: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:06.144927 17996 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:06.190891 17996 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:06.191347 17996 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:06.178983 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.108734 17719 consensus_queue.cc:237] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.144491 17882 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:06.168831 17643 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.168345 17997 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.184886 13506 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:06.189791 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.201247 13684 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:06.196090 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:06.196585 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:06.181386 17784 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:06.180074 17702 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:06.180238 17973 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:58:06.209553 17784 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:06.175390 17960 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:58:06.210119 17784 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:06.209779 17702 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.145860 17800 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.212179 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.212806 13685 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:06.192242 17996 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.228842 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:06.229732 13610 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 8.
I20260709 12:58:06.229972 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:06.230587 13617 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:58:06.234606 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.235204 13541 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:06.235867 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.236546 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:06.237095 13616 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:06.237243 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.237950 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.241534 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:06.266070 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.266639 13619 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:58:06.271262 13685 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:06.282666 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.283383 13538 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:58:06.259716 18001 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:58:06.260056 17875 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:06.278573 13617 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:06.233592 17381 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:06.302423 13541 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:06.304262 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.300521 13616 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:06.305886 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:06.309922 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 175213us
I20260709 12:58:06.287585 17850 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:06.293500 17921 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.309609 13685 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:06.295027 17702 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:06.319255 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:06.301055 17994 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:06.325696 17874 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.335317 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.347080 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:06.312282 17850 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:06.356402 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:06.318460 17381 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:06.301424 17875 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.296336 17882 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.362605 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.378880 13692 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:06.358843 17850 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:06.359396 17942 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:06.392774 17784 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.393878 18000 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:06.396561 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.397161 13693 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:06.394721 18000 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.392828 17800 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.401036 17919 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:06.382470 17728 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:06.404724 17841 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.363514 17670 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.360105 18006 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:06.406435 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.401849 17882 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:06.347688 17994 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:06.415673 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 174181us
I20260709 12:58:06.388710 18007 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:06.422367 17997 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.408522 18006 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:06.427621 13693 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:06.429266 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:06.428815 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 186565us
I20260709 12:58:06.431869 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.413851 17990 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.446523 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:06.413305 17841 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:06.448496 13684 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:06.448397 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.449817 13693 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:06.450680 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.418797 13685 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:06.452013 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.445688 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:06.455214 13684 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:06.425625 17914 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.462468 17914 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.465169 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.425159 18006 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:06.472234 17556 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:06.474069 18000 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:06.419782 17992 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.464555 17841 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.481617 17686 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:06.425401 17381 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.489391 17556 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:06.495026 17686 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:06.496843 17556 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.502316 17686 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.504608 17875 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:06.511301 17784 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.520512 17994 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:06.541896 17841 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.546111 17992 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.548331 17850 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.551020 17850 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.553333 17757 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:06.553800 17757 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:06.553982 18006 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.554602 17757 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.557402 17778 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.559018 13693 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:58:06.559438 17976 log.cc:927] Time spent T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.754s	user 0.001s	sys 0.003s
I20260709 12:58:06.631968 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:06.627454 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:06.636957 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:06.662350 14240 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:06.596735 17919 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:06.673169 17919 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:06.653463 17879 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:06.677706 17879 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:06.561038 17974 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.952s	user 0.004s	sys 0.000s
W20260709 12:58:06.680406 17999 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.534s	user 0.000s	sys 0.005s
W20260709 12:58:06.564065 17977 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.702s	user 0.000s	sys 0.002s
I20260709 12:58:06.567907 17936 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:06.684309 17757 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:06.575001 17908 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:06.650105 17960 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:06.701031 17960 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:06.699256 17936 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:06.626737 17890 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.560678 18006 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.703112 17960 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.712037 17960 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:06.712627 17936 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.714401 17936 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:06.716486 17923 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.025s	sys 0.012s
W20260709 12:58:06.845242 17923 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.026s	sys 0.012s
W20260709 12:58:06.755302 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:06.767409 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.773433 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:06.850524 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 418611us
I20260709 12:58:06.851125 13542 raft_consensus.cc:3060] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:06.851325 13534 raft_consensus.cc:3060] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:06.852351 17890 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:06.853025 17890 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.777182 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.850545 13615 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:06.857610 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.858649 13535 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
W20260709 12:58:06.778388 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:06.860607 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:06.861500 18006 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:06.862563 18006 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.780745 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.864818 13540 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 11: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:06.865990 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.866426 14244 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:58:06.869318 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:06.783432 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.871304 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.800608 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 3143) took 1211 ms. Trace:
I20260709 12:58:06.871723 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:06.872109 13690 rpcz_store.cc:276] 0709 12:58:05.588650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:05.588753 (+   103us) service_pool.cc:224] Handling call
0709 12:58:05.589079 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:05.589463 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:05.589492 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:05.589909 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 12:58:05.734364 (+144455us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:58:05.734422 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:05.734511 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:06.558808 (+824297us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:06.795611 (+236803us) raft_consensus.cc:1685] finished
0709 12:58:06.795655 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:06.800292 (+  4637us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":384,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":13355392,"thread_start_us":144130,"threads_started":1,"wal-append.queue_time_us":198994}
I20260709 12:58:06.873895 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.577713 13693 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:06.867525 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.806192 14209 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:06.593479 13692 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:06.821116 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:06.827060 14239 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:06.561245 17778 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.831432 17879 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.833581 17803 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:06.836723 17381 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:06.836946 17584 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:06.840296 17686 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:06.875540 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.878460 17890 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.893397 17686 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.898012 17803 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:06.898967 17381 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.892150 17584 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.900924 14242 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:06.904466 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.908882 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:06.910912 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 47833.9us
I20260709 12:58:06.912717 13535 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:06.913640 17686 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:06.913746 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:06.925690 17584 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:06.931442 17643 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:06.933888 13658 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:06.935040 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 45455.1us
I20260709 12:58:06.937036 14244 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:06.943130 17381 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:06.939843 18016 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.529s	user 0.001s	sys 0.000s
I20260709 12:58:06.944505 17778 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:06.945137 17778 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:06.945616 17778 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:06.946444 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1724) took 1274 ms. Trace:
I20260709 12:58:06.946872 13537 rpcz_store.cc:276] 0709 12:58:05.673212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:05.673373 (+   161us) service_pool.cc:224] Handling call
0709 12:58:05.673761 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:05.724683 (+ 50922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:05.724714 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:05.725180 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:58:06.307867 (+582687us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b18003e7dc0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:58:06.386713 (+ 78846us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:06.386796 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:06.386895 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:06.938959 (+552064us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:06.945977 (+  7018us) raft_consensus.cc:1685] finished
0709 12:58:06.946006 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:06.946211 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50457,"raft.queue_time_us":50588,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":10628,"spinlock_wait_cycles":23481984,"thread_start_us":40686,"threads_started":1,"wal-append.queue_time_us":643264}
I20260709 12:58:06.931608 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:06.950456 17803 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:06.951530 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 145795us
I20260709 12:58:06.953675 13540 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:06.952164 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:06.955349 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:06.956357 13691 raft_consensus.cc:2393] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:06.952492 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:06.944316 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:06.958231 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:06.958438 17658 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:06.959403 17658 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:06.960697 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:06.959829 17658 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:06.962308 17658 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:06.962426 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:06.966578 13612 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:06.968091 17643 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:06.969349 17643 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.971788 14245 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:06.972720 13540 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:58:06.974176 13540 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:58:06.987726 17781 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:06.989207 17781 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:06.973729 13539 raft_consensus.cc:2368] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10.
I20260709 12:58:06.975677 13540 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:07.033113 13534 raft_consensus.cc:2468] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
W20260709 12:58:07.036854 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 584534us
I20260709 12:58:05.983486 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:07.038985 13584 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.040082 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:07.040465 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.040885 13690 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:58:07.041594 13616 raft_consensus.cc:2410] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:58:07.042227 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.042484 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:07.043251 13616 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:58:07.032907 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:07.048851 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:07.045696 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.015141 17781 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:07.059158 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45839.9us
I20260709 12:58:07.062587 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.067276 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:07.067955 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.068954 13612 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 12: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:07.070652 17701 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:07.070621 13612 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
W20260709 12:58:07.081789 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:07.079512 13691 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:58:07.090657 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 75198.6us
I20260709 12:58:07.091358 17729 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.092077 17729 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.092602 17890 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:07.093168 17890 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:07.093607 17890 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:07.094808 17729 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.097942 13617 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:07.098652 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:07.099344 13542 raft_consensus.cc:2468] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:07.116712 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.071877 17919 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.127328 13535 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:58:07.100116 17803 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:07.135851 17869 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:07.146333 13584 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.099623 17951 consensus_queue.cc:1059] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:58:07.158874 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:07.160404 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46267.9us
I20260709 12:58:07.160418 17869 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:07.163291 17869 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:07.164412 17996 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:07.165452 18042 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:07.166214 18042 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:07.168617 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:07.169888 17923 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:07.165576 17996 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:58:07.172155 17658 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.174628 17658 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.176615 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.178009 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.179437 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.180318 13690 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:07.184906 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.185715 13542 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 9.
I20260709 12:58:07.167495 18042 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:07.188709 18038 log.cc:927] Time spent T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:58:07.190466 18039 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 12:58:07.190685 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 53460.3us
I20260709 12:58:07.194991 17960 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.195626 17960 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.198773 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:07.199698 13534 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:07.198747 17960 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.202380 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.203548 17996 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.203619 18008 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:07.204174 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.204627 18008 consensus_queue.cc:1243] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:58:07.204903 13610 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:07.204567 17996 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.206190 17757 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:07.206905 17757 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.209139 17757 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.209534 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1831) took 1541 ms. Trace:
I20260709 12:58:07.209826 13619 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:07.209985 13611 rpcz_store.cc:276] 0709 12:58:05.667848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:05.668251 (+   403us) service_pool.cc:224] Handling call
0709 12:58:05.668549 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:05.714960 (+ 46411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.209003 (+1494043us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b540012a2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:07.209324 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105057920}
I20260709 12:58:07.212955 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.218855 13539 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:07.218959 13693 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:58:07.211964 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 113404us
I20260709 12:58:07.221366 13542 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:07.207469 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.226919 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1833) took 1032 ms. Trace:
I20260709 12:58:07.227407 13613 rpcz_store.cc:276] 0709 12:58:06.194815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:06.194974 (+   159us) service_pool.cc:224] Handling call
0709 12:58:06.211899 (+ 16925us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:06.215935 (+  4036us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:58:06.216225 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 12:58:06.875362 (+659137us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b18003d6360. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 12:58:07.060036 (+184674us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:07.150525 (+ 90489us) log.cc:844] Serialized 26 byte log entry
0709 12:58:07.191214 (+ 40689us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:07.191254 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.191335 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:07.211574 (+ 20239us) raft_consensus.cc:1685] finished
0709 12:58:07.224226 (+ 12652us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:07.226724 (+  2498us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":599578,"prepare.queue_time_us":794,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":690093,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":29469,"spinlock_wait_cycles":10300288,"thread_start_us":254046,"threads_started":2,"wal-append.queue_time_us":142036}
I20260709 12:58:07.232134 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:07.232538 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 86744.9us
I20260709 12:58:07.233722 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:07.234540 14212 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.235520 17729 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:07.236967 17729 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:07.236888 17875 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:07.237504 17381 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:07.237783 13584 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:07.239244 18043 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:07.260254 14211 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:07.248725 13616 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:07.238127 17381 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:07.257830 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:07.280093 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 58751.5us
I20260709 12:58:07.283710 13691 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:07.285247 17329 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:07.281543 17757 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:58:07.316009 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 126311us
I20260709 12:58:07.285969 17869 raft_consensus.cc:2764] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
W20260709 12:58:07.321399 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:07.322866 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.322901 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.324507 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.325356 18043 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:07.329214 17997 raft_consensus.cc:2764] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:58:07.332172 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 95128.9us
I20260709 12:58:07.306993 17329 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:58:07.354548 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65903.2us
I20260709 12:58:07.335665 18043 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:07.340248 17329 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:07.362129 17381 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.393920 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:07.364925 18043 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:07.404310 17875 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:07.405089 17875 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.400266 17869 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:07.408864 17869 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.409965 18041 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:07.410473 18041 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:07.410949 18041 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:07.407681 17875 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:07.411767 17875 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:07.412842 17778 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.410944 17869 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.416139 17778 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:07.417330 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52517.3us
I20260709 12:58:07.419313 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.420050 13613 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:07.421813 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.423300 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.423672 13619 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:07.424337 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.425369 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.425388 13610 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:58:07.426151 13610 pending_rounds.cc:85] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 6
I20260709 12:58:07.426216 14245 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 13: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:07.426445 13610 pending_rounds.cc:107] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:58:07.426883 17918 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.426851 13610 raft_consensus.cc:2889] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:07.427560 17918 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.428232 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:07.429023 13611 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:07.429826 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1839) took 1190 ms. Trace:
I20260709 12:58:07.429986 17918 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.430264 13615 rpcz_store.cc:276] 0709 12:58:06.239660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:06.239788 (+   128us) service_pool.cc:224] Handling call
0709 12:58:06.240097 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:06.856519 (+616422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.429561 (+573042us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540013bfd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:07.429769 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1402522496}
I20260709 12:58:07.428267 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 621) took 1863 ms. Trace:
I20260709 12:58:07.432193 13613 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:07.432082 13689 rpcz_store.cc:276] 0709 12:58:05.544948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:05.545094 (+   146us) service_pool.cc:224] Handling call
0709 12:58:05.551694 (+  6600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:05.768065 (+216371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.407586 (+1639521us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54002eccd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:07.407798 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484166912}
W20260709 12:58:07.433885 18046 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.110s	user 0.000s	sys 0.003s
I20260709 12:58:07.434918 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.435895 14245 raft_consensus.cc:2410] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:58:07.436252 17908 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.436833 17908 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.439199 17908 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.439697 14243 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:58:07.441064 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:07.439208 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:07.442571 14244 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:07.451172 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.440052 17996 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.453541 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:07.461956 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 211729us
I20260709 12:58:07.470506 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.471472 14242 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:07.472971 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.474700 17329 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.476387 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.482282 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.485186 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.488442 13613 pending_rounds.cc:85] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 6
I20260709 12:58:07.488783 13613 pending_rounds.cc:107] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:58:07.489044 13613 raft_consensus.cc:2889] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:07.491031 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.489859 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:07.491865 13615 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:07.496480 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.496733 14239 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:58:07.497228 14239 pending_rounds.cc:85] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 9
I20260709 12:58:07.497507 14239 pending_rounds.cc:107] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:58:07.497574 14245 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 12: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:07.497817 14239 raft_consensus.cc:2889] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:07.499189 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.499761 17329 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.506911 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.507225 17584 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:07.511731 14244 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:07.511693 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:07.509016 17584 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:07.514984 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.515223 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.516382 13534 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:07.516606 14239 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEARNER]: Advancing to term 11
I20260709 12:58:07.507875 17951 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
W20260709 12:58:07.518738 17936 consensus_meta.cc:287] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.012s
I20260709 12:58:07.520538 14245 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:58:07.522539 13538 raft_consensus.cc:2410] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:58:07.508379 17658 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:07.525807 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:07.519361 17584 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:07.527148 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.527834 17951 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:07.528314 17951 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:07.529147 17951 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.530383 17951 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:07.532119 18015 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:07.532676 18015 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:07.533553 18015 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.536190 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:07.536609 18049 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:58:07.538079 17918 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:07.541697 13508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.68:39527 timed out after 1.790s (SENT) [suppressed 18 similar messages]
I20260709 12:58:07.544620 17890 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.546944 17890 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.550557 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:07.551689 14244 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 13: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:07.555079 14239 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:07.547670 17908 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:07.570960 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:07.571156 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:07.580679 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 71974.9us
I20260709 12:58:07.584386 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.584710 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:07.585359 14237 raft_consensus.cc:2393] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 12: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:07.585958 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:07.586369 13542 raft_consensus.cc:1275] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:07.586563 13539 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:07.591200 14246 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 10. Ops: []
I20260709 12:58:07.593232 13537 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:58:07.572863 17908 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:07.595036 17701 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.569828 17329 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:58:07.591737 17996 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:58:07.601526 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.600174 17908 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:07.630903 13615 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:07.637576 17862 raft_consensus.cc:2764] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:58:07.638988 18050 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.315s	user 0.004s	sys 0.001s
I20260709 12:58:07.640911 17329 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:07.648660 13584 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:07.648613 17996 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:07.650403 17960 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:07.650722 17996 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:07.651250 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1834) took 1455 ms. Trace:
I20260709 12:58:07.651644 13614 rpcz_store.cc:276] 0709 12:58:06.195270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:06.195385 (+   115us) service_pool.cc:224] Handling call
0709 12:58:06.195670 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:06.278386 (+ 82716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:06.278433 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:06.278490 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:06.278542 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.650828 (+1372286us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b540013a1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:07.650892 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:07.651096 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189641088}
I20260709 12:58:07.650746 17584 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.653738 17719 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:58:07.657104 17584 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.663075 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 3135) took 2106 ms. Trace:
I20260709 12:58:07.663726 13539 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:07.629432 17923 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:07.665874 17923 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.668406 14211 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:07.663487 13687 rpcz_store.cc:276] 0709 12:58:05.556724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:05.560272 (+  3548us) service_pool.cc:224] Handling call
0709 12:58:05.560644 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:05.787095 (+226451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:05.787133 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:05.787175 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:58:05.787202 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.662519 (+1875317us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400122fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:07.662595 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:07.662832 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520291200}
I20260709 12:58:07.670514 17329 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:07.670935 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.671425 18058 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.605355 17701 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.671988 18058 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.713768 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.714587 13687 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 10: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:07.716171 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:07.716899 13690 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:07.717782 17996 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:07.718331 17996 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.722321 17701 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:07.722831 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:07.723807 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 60254.2us
I20260709 12:58:07.725739 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.726532 13614 raft_consensus.cc:1275] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:07.726769 17936 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:58:07.727205 17939 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:07.728355 17936 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:07.729985 13689 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:07.732131 17936 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:58:07.732963 17936 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:07.733287 17936 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:07.734297 17936 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.735706 17936 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:07.711674 18043 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:07.741894 18044 log.cc:927] Time spent T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.424s	user 0.003s	sys 0.000s
W20260709 12:58:07.744268 18051 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.411s	user 0.002s	sys 0.000s
I20260709 12:58:07.747463 17876 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:07.748075 17876 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:07.749233 17876 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.750872 17876 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:07.752751 18006 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:07.754006 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:07.680429 18008 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:07.778218 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.787205 18058 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.788112 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:07.798655 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.801318 18008 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.683844 17875 consensus_queue.cc:1048] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:58:07.813508 13684 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:07.813570 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:07.815412 18045 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.492s	user 0.001s	sys 0.001s
I20260709 12:58:07.816371 17996 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.817854 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1731) took 1030 ms. Trace:
I20260709 12:58:07.815429 13539 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:07.818362 13543 rpcz_store.cc:276] 0709 12:58:06.787698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:06.939310 (+151612us) service_pool.cc:224] Handling call
0709 12:58:06.939696 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:58:06.940526 (+   830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:06.940572 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:58:06.941309 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 12:58:07.055603 (+114294us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:07.055658 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.055828 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:07.741065 (+685237us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:07.812295 (+ 71230us) raft_consensus.cc:1685] finished
0709 12:58:07.812320 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:07.817737 (+  5417us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1537,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":104,"spinlock_wait_cycles":34521984,"thread_start_us":113807,"threads_started":1,"wal-append.queue_time_us":281582}
I20260709 12:58:07.828088 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:07.832329 13693 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:58:07.835575 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 294012us
I20260709 12:58:07.825314 17936 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:07.827153 17951 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:07.860287 13692 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:58:07.859715 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.860688 13692 pending_rounds.cc:85] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 9
I20260709 12:58:07.871062 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:07.834018 17918 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:58:07.874579 13508 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:07.876216 17919 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.876730 17992 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:07.865024 13692 pending_rounds.cc:107] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.10
I20260709 12:58:07.880851 13692 raft_consensus.cc:2917] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Aborting config change with OpId 10.10: Aborted: Op aborted by new leader
I20260709 12:58:07.881366 13692 raft_consensus.cc:2889] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:07.881650 18008 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.883438 17923 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.885455 13689 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:07.881749 13612 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:58:07.887126 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:07.887787 17918 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.888484 17918 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.888906 13693 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:07.868062 17936 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:07.890800 17918 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.890532 17936 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:07.892045 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.893210 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.893322 18041 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.894790 13618 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:58:07.895526 18041 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:07.898111 17729 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:07.898707 17729 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.900897 17729 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:07.905086 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.905290 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.905498 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.906286 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.906507 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.910166 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.925319 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:07.902518 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.935724 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:07.944772 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:07.943173 13658 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.951980 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.954108 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:07.956171 13691 raft_consensus.cc:1275] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:58:07.932927 17381 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:07.954870 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.965898 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:07.966351 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:07.973802 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:07.976132 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 73697.9us
I20260709 12:58:07.976552 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:07.978300 14241 raft_consensus.cc:2393] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:07.977584 17803 raft_consensus.cc:3055] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:07.979457 17803 raft_consensus.cc:740] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:07.972986 17918 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:07.980414 17803 consensus_queue.cc:260] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.981572 17803 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:07.984076 18059 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:07.984220 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.984822 13614 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:07.985632 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:07.986166 13614 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:07.987210 17908 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:07.989745 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.967252 17757 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:07.971716 17869 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:07.993327 13508 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:07.993233 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:07.995208 13684 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:07.997306 18059 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:08.000492 18059 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:08.000341 18005 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:58:08.000988 18059 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:08.001497 18005 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:08.001881 18005 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:08.002573 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:08.002832 18005 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:08.003756 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:08.003985 17910 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:58:07.999120 17729 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:08.006603 17729 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:08.006986 17729 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:07.978076 18008 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:58:08.009829 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:07.999382 17992 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:08.024361 13508 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:07.992357 17875 consensus_queue.cc:1048] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:58:08.032442 17951 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:08.003893 18005 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:08.014989 17919 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:08.011199 17329 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:08.023442 17774 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:58:08.053190 17799 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:07.997979 17381 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:58:08.016630 17869 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.063966 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:08.066047 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 57026.4us
I20260709 12:58:08.102367 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:08.133311 14241 raft_consensus.cc:2393] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 12: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:08.135504 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:08.136539 14244 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:08.090761 17381 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:08.140007 17869 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:08.175427 14246 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.031s	sys 0.027s
W20260709 12:58:08.176024 14246 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.031s	sys 0.027s
W20260709 12:58:07.988104 18006 consensus_queue.cc:1175] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ad2fdb1517c542439654bd3f7c72dad0" responder_term: 11 status { last_received { term: 9 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:58:08.192968 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 198582us
I20260709 12:58:08.228433 13612 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:08.247717 17778 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:08.250123 13658 leader_election.cc:400] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10.
I20260709 12:58:08.251466 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:08.252774 14240 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:08.253206 13658 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:08.253651 17935 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:58:08.253569 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:08.254562 17936 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:08.254735 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:08.255929 17936 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:08.256631 17936 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:08.257128 18008 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:08.252645 13691 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:08.252650 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:08.235747 17869 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.269248 13692 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:08.269378 17869 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:08.224069 18052 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.276352 18052 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.278756 18052 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:08.280100 18070 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:58:08.280436 18071 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:58:08.281468 17643 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:08.282819 17643 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:08.283281 17643 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:08.288335 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:08.290154 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 244826us
I20260709 12:58:08.292865 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.293809 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.226078 17908 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:08.310191 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:08.313756 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:08.314455 13534 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:08.296555 17869 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:08.304775 17643 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.333427 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:08.344063 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:08.225888 17329 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.346109 13541 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
W20260709 12:58:08.350257 18075 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:58:08.354631 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:08.368141 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:08.369593 13612 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:08.376569 17996 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.377132 17996 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:08.388607 18078 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:58:08.373239 17643 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:58:08.393851 18072 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:58:08.395154 18068 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:58:08.371587 18015 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:08.405316 13612 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:58:08.409078 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:08.405678 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:08.411123 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:08.408468 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:08.412526 13536 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:08.413146 18015 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:08.413297 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:08.413869 18036 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:58:08.414199 18015 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.418176 17879 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:08.419039 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:08.422060 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:08.425908 13615 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:08.406762 13506 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:08.428447 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:08.383287 17996 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:08.429422 18076 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:08.469616 17910 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:58:08.472743 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:08.473585 13686 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:08.433323 17729 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.440337 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:08.440235 17862 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:08.420264 17908 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:08.441675 18067 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.304s	user 0.004s	sys 0.000s
I20260709 12:58:08.441498 17997 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.442224 17923 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:08.441495 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:08.445842 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:08.447969 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:08.429477 14244 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:58:08.461004 13537 raft_consensus.cc:2368] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10.
I20260709 12:58:08.463472 13691 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:08.463835 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:08.464094 17919 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:08.464679 18093 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:58:08.465687 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 58999.3us
I20260709 12:58:08.504210 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:08.474807 18043 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:08.492406 13688 raft_consensus.cc:2410] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:58:08.493626 13534 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:08.466301 17843 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:08.439275 18059 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.506948 14240 raft_consensus.cc:2368] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:08.507941 13658 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:08.514621 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 111033us
W20260709 12:58:08.515781 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 74372.2us
I20260709 12:58:08.516779 13582 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:08.519057 13537 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:58:08.519397 18059 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:08.520779 13613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1850) took 1046 ms. Trace:
I20260709 12:58:08.521543 13658 leader_election.cc:400] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10.
I20260709 12:58:08.521926 13658 leader_election.cc:403] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:08.521664 18093 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.522714 13658 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:08.524173 17939 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:58:08.528132 17996 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:08.528517 18090 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:08.529461 18006 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:08.530562 13534 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:08.531601 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:08.532689 13615 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:08.533103 18093 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:08.521199 13613 rpcz_store.cc:276] 0709 12:58:07.474378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:07.482011 (+  7633us) service_pool.cc:224] Handling call
0709 12:58:07.483669 (+  1658us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:58:07.493481 (+  9812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:07.493517 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:58:07.494141 (+   624us) log.cc:844] Serialized 52 byte log entry
0709 12:58:07.832756 (+338615us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:07.832812 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:07.832903 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:08.391521 (+558618us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:08.520350 (+128829us) raft_consensus.cc:1685] finished
0709 12:58:08.520372 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:08.520714 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2657,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":31960960,"thread_start_us":337461,"threads_started":1,"wal-append.queue_time_us":533331}
I20260709 12:58:08.508325 17919 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.547973 17843 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:08.548667 17803 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:08.549307 17843 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.549384 17803 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.551703 17729 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:08.509007 17997 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.511777 17908 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.509243 17381 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:08.547608 17919 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:08.555866 17997 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:08.559605 17908 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:08.561808 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:08.579540 13612 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:08.573289 18008 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:58:08.585000 18008 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:08.585592 18008 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:08.586745 18008 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.561385 17584 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:58:08.590147 17584 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:08.590602 17584 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:08.565160 18091 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:58:08.500650 17910 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:58:08.601773 13618 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:58:08.602635 18092 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:58:08.603765 18091 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:58:08.605600 18097 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:58:08.607105 13692 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:08.608422 17960 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:08.610973 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 94321.2us
I20260709 12:58:08.612576 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:08.628559 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:08.634486 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 189834us
I20260709 12:58:08.635236 13615 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:08.637773 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:08.639240 13612 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:08.647012 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:08.657651 17960 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:08.658767 17960 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:58:08.659158 18094 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.224s	user 0.000s	sys 0.002s
I20260709 12:58:08.659385 18077 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:08.652988 17923 consensus_queue.cc:1048] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:58:08.591609 17584 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.662457 17584 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:08.662720 18008 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:08.643774 17996 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:08.628417 18091 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.666095 18091 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.668174 18043 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:58:08.659171 17960 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:08.669627 17908 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:08.676710 14246 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 773) took 1021 ms. Trace:
I20260709 12:58:08.676813 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:08.683274 13508 leader_election.cc:400] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:08.683557 13508 leader_election.cc:403] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:58:08.681265 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.669269 17939 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10.
I20260709 12:58:08.637370 17879 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:08.638132 17862 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:08.696381 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:08.671864 18089 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:58:08.690006 17910 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:08.685182 18041 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:08.694862 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:08.658210 18082 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
I20260709 12:58:08.688711 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:08.649466 17918 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:08.678009 17923 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:08.682478 14246 rpcz_store.cc:276] 0709 12:58:07.655283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:07.655446 (+   163us) service_pool.cc:224] Handling call
0709 12:58:07.655792 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:07.656230 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:58:08.185771 (+529541us) log.cc:844] Serialized 14 byte log entry
0709 12:58:08.484704 (+298933us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:08.504674 (+ 19970us) log.cc:844] Serialized 26 byte log entry
0709 12:58:08.510728 (+  6054us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:58:08.510763 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:08.510837 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:08.676482 (+165645us) raft_consensus.cc:1685] finished
0709 12:58:08.676503 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:08.676650 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":517742,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":2122,"raft.queue_time_us":427,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":7909,"spinlock_wait_cycles":3968,"thread_start_us":714493,"threads_started":2,"wal-append.queue_time_us":411139}
I20260709 12:58:08.668248 18091 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:08.702965 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:08.690932 18077 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.706198 13692 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:08.690774 18087 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:08.732965 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:08.724469 14240 raft_consensus.cc:1240] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:58:08.701877 18041 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:08.736866 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:08.740676 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:08.745461 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:08.739717 17992 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:58:08.769578 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 119224us
I20260709 12:58:08.776765 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:08.743141 17869 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:08.742125 17643 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:08.777532 13691 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:08.752116 17939 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:08.752724 18091 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:08.735760 17923 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.798839 13692 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:08.801192 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:08.801878 13686 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:08.737529 18077 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:08.805612 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 71188.4us
I20260709 12:58:08.817200 13612 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:58:08.818045 13506 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:08.757069 18087 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.825223 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.820626 17919 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:58:08.819519 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.834641 13691 raft_consensus.cc:2410] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:58:08.806567 17918 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:08.836910 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.806147 17996 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:08.791203 17643 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:08.876581 18109 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:58:08.877632 18102 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 12:58:08.878679 18098 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.450s	user 0.000s	sys 0.002s
W20260709 12:58:08.880437 18103 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:58:08.882171 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 638) took 1174 ms. Trace:
I20260709 12:58:08.882916 13684 rpcz_store.cc:276] 0709 12:58:07.707436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:07.707688 (+   252us) service_pool.cc:224] Handling call
0709 12:58:07.729236 (+ 21548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:07.996484 (+267248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:08.881830 (+885346us) spinlock_profiling.cc:243] Waited 74.3 ms on lock 0x7b54002391d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:08.882086 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170830592}
W20260709 12:58:08.882277 18105 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:58:08.887460 18083 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.456s	user 0.000s	sys 0.002s
I20260709 12:58:08.894034 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1853) took 1058 ms. Trace:
I20260709 12:58:08.894518 13611 rpcz_store.cc:276] 0709 12:58:07.835311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:07.835403 (+    92us) service_pool.cc:224] Handling call
0709 12:58:07.850853 (+ 15450us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:58:07.896005 (+ 45152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:07.896040 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:58:07.896710 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 12:58:08.135937 (+239227us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:08.168581 (+ 32644us) log.cc:844] Serialized 14 byte log entry
0709 12:58:08.168805 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:08.168907 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:08.665084 (+496177us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b18002f4ca0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:58:08.886723 (+221639us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:08.893799 (+  7076us) raft_consensus.cc:1685] finished
0709 12:58:08.893821 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:08.893969 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34195,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":18485,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":3935,"spinlock_wait_cycles":106742272,"thread_start_us":238745,"threads_started":1,"wal-append.queue_time_us":479345}
I20260709 12:58:08.853817 18115 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:08.822211 17992 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 12:58:08.899425 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 69452.9us
I20260709 12:58:08.818459 18041 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.927120 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.928206 13611 raft_consensus.cc:2393] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:08.925871 17997 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.930949 17997 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:08.891733 17803 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:08.932870 18087 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:08.934037 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:08.938495 17869 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:08.939234 13691 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:58:08.939075 17869 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.854871 17918 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:08.945917 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 106458us
W20260709 12:58:08.931663 18114 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:58:08.863399 17996 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:08.951946 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:08.952157 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:08.952687 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:08.941125 17757 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:08.963318 13687 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:58:08.971045 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:08.960747 17757 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:08.984413 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:09.006017 17869 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:09.008083 18121 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:09.008942 18121 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.012349 17918 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.016227 13535 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
W20260709 12:58:09.018011 18113 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:58:09.016775 17757 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:09.020162 18116 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:58:09.020877 17757 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:09.020941 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:09.021694 13686 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:08.978878 17997 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.032832 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:09.035787 13687 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:08.980432 17919 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:58:09.035961 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:09.028968 17876 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:09.042537 17923 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:09.045447 18081 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.612s	user 0.004s	sys 0.000s
I20260709 12:58:09.030190 18077 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.048434 17996 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:09.048671 18110 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.178s	user 0.002s	sys 0.000s
I20260709 12:58:09.065016 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:09.066022 18041 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:09.089787 17876 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:09.071401 18095 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.638s	user 0.002s	sys 0.001s
I20260709 12:58:09.102041 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1166) took 1005 ms. Trace:
I20260709 12:58:09.106024 14245 rpcz_store.cc:276] 0709 12:58:08.096388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:08.096912 (+   524us) service_pool.cc:224] Handling call
0709 12:58:08.097292 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:08.097727 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:08.097768 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:08.098332 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:58:08.347081 (+248749us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:08.364273 (+ 17192us) log.cc:844] Serialized 14 byte log entry
0709 12:58:08.364716 (+   443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:08.364873 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:08.482276 (+117403us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b180011a880. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:58:09.069124 (+586848us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:09.101824 (+ 32700us) raft_consensus.cc:1685] finished
0709 12:58:09.101849 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:09.101981 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24093,"prepare.queue_time_us":16097,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":5190,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":30908,"spinlock_wait_cycles":41419008,"thread_start_us":158114,"threads_started":1,"wal-append.queue_time_us":291870}
W20260709 12:58:09.135511 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 168552us
I20260709 12:58:09.099740 17960 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.136580 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:09.136874 13619 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:58:09.138923 17960 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.046906 17381 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:09.140568 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:09.141213 13687 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:09.145035 14244 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:09.151703 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:09.152848 14245 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:09.153098 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:09.154814 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:09.163501 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:09.164220 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:09.101630 18093 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 12:58:09.172420 18123 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:09.104187 18117 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:09.173715 18117 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:09.204515 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 45969.7us
I20260709 12:58:09.200273 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:09.190521 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:09.206162 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:09.215461 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 51997.6us
I20260709 12:58:09.218489 13687 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:09.190587 17996 raft_consensus.cc:2764] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 12:58:09.226799 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61377.7us
I20260709 12:58:09.230636 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:09.221454 17757 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:09.233338 18122 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:09.233615 18054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.017s	sys 0.013s
W20260709 12:58:09.234135 18054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.018s	sys 0.013s
I20260709 12:58:09.218345 17997 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:09.233462 17757 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:09.234678 17997 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:09.235005 17757 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:09.235095 17997 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:09.236685 18123 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:09.237206 18123 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:09.204435 18121 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.234102 18122 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:09.207971 17923 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:09.222728 17869 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:09.249550 17869 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:09.492621 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:09.498693 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:09.499706 13543 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:09.504189 18117 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:09.514680 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:09.518829 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:09.441254 18121 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:09.527210 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:09.466012 17951 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:09.477367 18008 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:09.487882 17869 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:09.474128 17876 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.466269 18077 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:58:09.557806 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:09.564779 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:09.592578 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1753) took 1165 ms. Trace:
W20260709 12:58:09.572115 18124 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:58:09.601537 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:09.588078 18132 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:09.559896 18111 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:09.593050 13538 rpcz_store.cc:276] 0709 12:58:08.426666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:08.448647 (+ 21981us) service_pool.cc:224] Handling call
0709 12:58:08.467437 (+ 18790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:08.558872 (+ 91435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:08.558920 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:08.559621 (+   701us) log.cc:844] Serialized 26 byte log entry
0709 12:58:08.700270 (+140649us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:08.700307 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:09.577909 (+877602us) spinlock_profiling.cc:243] Waited 91 ms on lock 0x7b540006e5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:09.577962 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:09.578009 (+    47us) raft_consensus.cc:1685] finished
0709 12:58:09.578023 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:09.592500 (+ 14477us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1536,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":209205632,"thread_start_us":140190,"threads_started":1,"wal-append.queue_time_us":193512}
W20260709 12:58:09.606331 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 356025us
I20260709 12:58:09.577595 17996 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:09.630837 18127 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:09.631462 18127 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.603567 17923 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:09.566748 17951 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.633757 18121 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:09.652711 18140 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 12:58:09.653791 18129 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 12:58:09.654848 17910 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.953s	user 0.014s	sys 0.000s
I20260709 12:58:09.608300 17862 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:09.637954 17923 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.596933 18104 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:09.634078 18127 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:09.636744 18126 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 12:58:09.609529 18111 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:09.587683 18008 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:09.629107 17960 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:09.672920 17910 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:09.698151 18111 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:09.720170 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:09.721221 18127 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:09.726542 17869 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:09.727623 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:09.735005 18041 consensus_meta.cc:287] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.000s
I20260709 12:58:09.696803 17862 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:09.738095 17757 consensus_meta.cc:287] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.002s
I20260709 12:58:09.672300 18121 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:09.706262 17960 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:09.731710 17997 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.622927 18132 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:09.710876 18104 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:09.697455 17996 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.757823 18127 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.777470 17923 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.766607 13537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1759) took 1302 ms. Trace:
I20260709 12:58:09.759078 17757 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.778995 18127 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.778661 13537 rpcz_store.cc:276] 0709 12:58:08.463602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:08.519852 (+ 56250us) service_pool.cc:224] Handling call
0709 12:58:08.520185 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:08.546403 (+ 26218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:08.546433 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:08.546472 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:08.546503 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:09.766166 (+1219663us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54002832d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:09.766226 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:09.766395 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59873024}
I20260709 12:58:09.780457 17757 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.781211 18132 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:09.782604 18135 log.cc:927] Time spent T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 12:58:09.751266 14211 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:09.755106 18133 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:09.784387 18133 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:09.785481 18133 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:09.786574 18145 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:09.787007 18145 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:09.787318 18145 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:09.794314 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1676) took 1010 ms. Trace:
I20260709 12:58:09.795591 13619 raft_consensus.cc:2368] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:58:09.794785 13685 rpcz_store.cc:276] 0709 12:58:08.784261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:08.784474 (+   213us) service_pool.cc:224] Handling call
0709 12:58:08.801038 (+ 16564us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:58:08.826550 (+ 25512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:08.826583 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:58:08.850778 (+ 24195us) log.cc:844] Serialized 52 byte log entry
0709 12:58:08.968032 (+117254us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:58:08.986063 (+ 18031us) log.cc:844] Serialized 14 byte log entry
0709 12:58:08.986309 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:08.986451 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:09.794039 (+807588us) raft_consensus.cc:1685] finished
0709 12:58:09.794066 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:09.794231 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21392,"prepare.queue_time_us":13443,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":42927,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":37561,"thread_start_us":50129,"threads_started":1,"wal-append.queue_time_us":303065}
W20260709 12:58:09.825513 14209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.017s
W20260709 12:58:09.826033 14209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.000s	sys 0.017s
W20260709 12:58:09.826365 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 579012us
I20260709 12:58:09.803617 17862 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:09.833329 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 211971us
I20260709 12:58:09.804593 18123 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:09.812379 17757 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:09.836792 17757 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:09.837988 17996 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:09.839053 17757 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:10.045204 13658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.054s
W20260709 12:58:10.048841 13658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.007s	sys 0.054s
W20260709 12:58:10.066362 18108 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.257s	user 0.000s	sys 0.009s
I20260709 12:58:10.035483 18052 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.039610 17757 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.039337 17862 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.042785 18122 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.042543 17996 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.169902 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:10.067799 18127 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.200146 13617 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:10.109891 18043 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.116868 17923 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.185848 17869 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:09.849697 18104 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.214594 18151 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.188642 18052 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.221513 18121 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.237541 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.241329 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.246683 13684 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:10.230222 17960 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:58:10.252516 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:10.231856 17869 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.214264 17862 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.202484 17757 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.268141 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.260798 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.268982 13691 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:10.262526 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:10.231832 17997 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.258231 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.270155 13692 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:10.217504 18127 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.222271 18043 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:10.258934 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 205235us
I20260709 12:58:10.273633 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.274376 13684 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:10.215058 17996 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.282099 17757 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.284003 17862 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.225586 17923 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.294905 13692 raft_consensus.cc:2468] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:10.297590 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.298377 13691 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:10.272243 17869 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.303813 17869 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:10.311906 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59535.4us
I20260709 12:58:10.316231 18172 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.317152 17803 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.316819 18172 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.317526 17923 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.317719 17803 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:10.312549 18154 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:58:10.312911 17757 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.324407 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:10.324357 17757 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.330119 17918 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.245826 18151 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.333338 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1622) took 1362 ms. Trace:
I20260709 12:58:10.333896 13535 rpcz_store.cc:276] 0709 12:58:08.970574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:08.970765 (+   191us) service_pool.cc:224] Handling call
0709 12:58:10.333092 (+1362327us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54003ca354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:10.333230 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102312704}
I20260709 12:58:10.336797 17996 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.338564 18008 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.340852 17996 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.341638 17996 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.351966 17960 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.352648 17960 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.357667 18058 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.358609 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:10.360831 18084 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 1.926s	user 0.006s	sys 0.007s
I20260709 12:58:10.362200 14237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 774) took 2698 ms. Trace:
I20260709 12:58:10.362669 14237 rpcz_store.cc:276] 0709 12:58:07.663573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:07.664146 (+   573us) service_pool.cc:224] Handling call
0709 12:58:07.664500 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:07.664990 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:07.665030 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:07.897571 (+232541us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b2c00047778. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:58:07.898054 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:58:08.167652 (+269598us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:58:08.167715 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:08.169093 (+  1378us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:10.360067 (+2190974us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:10.361948 (+  1881us) raft_consensus.cc:1685] finished
0709 12:58:10.361978 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:10.362118 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4401,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":88304768,"thread_start_us":261734,"threads_started":1,"wal-append.queue_time_us":535866}
I20260709 12:58:10.313446 18161 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.365384 18058 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.366211 18161 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.311389 18156 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.368588 18156 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.370780 18170 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:10.371445 18170 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.372706 13689 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:10.373436 17869 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.374109 17869 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.376679 18172 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.377414 18172 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.378001 18172 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.385918 18178 raft_consensus.cc:2764] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:58:10.386281 13693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 654) took 1693 ms. Trace:
I20260709 12:58:10.386967 18176 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.388074 18008 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.388569 18008 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.386766 13693 rpcz_store.cc:276] 0709 12:58:08.692800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:08.693210 (+   410us) service_pool.cc:224] Handling call
0709 12:58:08.693546 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:58:08.693942 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:08.693970 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:58:08.694549 (+   579us) log.cc:844] Serialized 52 byte log entry
0709 12:58:08.776417 (+ 81868us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:08.777668 (+  1251us) log.cc:844] Serialized 14 byte log entry
0709 12:58:10.385708 (+1608040us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b1800245bc0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287
0709 12:58:10.385846 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:10.385922 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:10.385964 (+    42us) raft_consensus.cc:1685] finished
0709 12:58:10.386055 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:10.386211 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2841,"prepare.queue_time_us":2743,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":5923,"raft.queue_time_us":55708,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":5456,"spinlock_wait_cycles":15202560,"thread_start_us":146247,"threads_started":2,"wal-append.queue_time_us":125851,"wal-append.run_cpu_time_us":4406,"wal-append.run_wall_time_us":1241417}
W20260709 12:58:10.392527 18139 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.832s	user 0.003s	sys 0.000s
I20260709 12:58:10.394956 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1680) took 1298 ms. Trace:
I20260709 12:58:10.395439 13686 rpcz_store.cc:276] 0709 12:58:09.096777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:09.096936 (+   159us) service_pool.cc:224] Handling call
0709 12:58:09.097538 (+   602us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:58:09.098180 (+   642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:09.098223 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:58:09.099057 (+   834us) log.cc:844] Serialized 52 byte log entry
0709 12:58:09.133649 (+ 34592us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:09.134026 (+   377us) log.cc:844] Serialized 14 byte log entry
0709 12:58:09.141590 (+  7564us) log.cc:844] Serialized 14 byte log entry
0709 12:58:09.145391 (+  3801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:09.145460 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:09.785040 (+639580us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b180011d400. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:58:09.834506 (+ 49466us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b180011d400. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:58:10.391813 (+557307us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:10.394658 (+  2845us) raft_consensus.cc:1685] finished
0709 12:58:10.394691 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:10.394887 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10449,"prepare.queue_time_us":318,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":83,"raft.queue_time_us":30420,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":34430,"spinlock_wait_cycles":74953472,"thread_start_us":34133,"threads_started":1,"wal-append.queue_time_us":344383}
I20260709 12:58:10.403879 18148 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.405737 18148 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.406737 18144 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:10.414700 18131 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.415246 18132 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.415341 18131 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.416049 18132 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.416514 18125 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:10.416060 18093 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:10.418207 18093 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.422057 18178 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:10.422305 17923 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:10.423174 17923 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.407287 18144 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.455976 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.464797 17862 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.465545 17862 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:10.473140 18155 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:58:10.440256 17910 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.477213 17910 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.440570 17799 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.479182 17799 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.481699 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:10.484053 18054 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.484632 18054 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.486568 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.495268 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:10.497869 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 175751us
I20260709 12:58:10.502156 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:10.500053 13658 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:10.507609 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 77688.6us
I20260709 12:58:10.520751 13617 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:10.530133 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:10.531138 13612 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:10.532001 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.505143 18041 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:10.569624 17803 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.570039 18041 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.570923 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:10.576087 17951 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:10.576282 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:10.577028 13540 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:58:10.577399 18172 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:10.577591 17908 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:10.578917 18111 consensus_meta.cc:287] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.823s	user 0.020s	sys 0.000s
I20260709 12:58:10.579602 17910 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.580611 17799 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.581703 14243 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:58:10.581841 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:10.582646 18043 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.583174 18043 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:10.583630 18043 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.585700 18043 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.585450 13438 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 10, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 10 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:10.586730 17996 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.582647 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.588533 13541 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:10.588737 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.589366 13543 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:10.591563 13541 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:58:10.591663 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 55806.7us
I20260709 12:58:10.578194 17908 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.593727 17908 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.594022 13540 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:10.594835 17862 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.595898 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.579342 18111 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.596200 14211 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:10.599364 13539 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
W20260709 12:58:10.597195 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 66089.9us
I20260709 12:58:10.537297 13610 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 13. Current term is 14. Ops: []
W20260709 12:58:10.597149 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 88722.3us
I20260709 12:58:10.596931 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.600487 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.603452 13509 leader_election.cc:400] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:58:10.603531 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.603973 13509 leader_election.cc:403] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:10.601500 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:10.604219 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.544000 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.587317 13438 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 9, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:10.500010 18006 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:10.533528 17843 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.605669 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:10.611441 18172 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:10.611788 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.520519 18127 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.608525 17381 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.387652 18176 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.524749 18087 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:10.530161 17876 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:10.627617 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:10.629951 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.631130 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.510669 18179 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.633193 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:10.508580 17918 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.631150 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:10.635666 13438 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 9, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 9 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:10.666477 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.536181 17992 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:58:10.624697 17729 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:10.670287 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.633236 17869 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:10.628005 18115 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:58:10.622262 17843 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.678946 17729 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:10.640834 18179 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.679993 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.685688 18054 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.687065 17643 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.626778 17719 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 12:58:10.691963 13437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49892 (request call id 319) took 20413 ms (client timeout 15000 ms). Trace:
I20260709 12:58:10.712422 18117 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:10.626657 17584 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:58:10.713692 18117 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:10.714196 18117 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:10.628528 18138 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:10.716130 18130 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:10.678418 13438 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 7, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:10.699136 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 68050.6us
I20260709 12:58:10.722256 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:10.609866 18125 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:58:10.671424 18059 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.671293 17862 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.621826 18006 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:10.684482 18177 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.362s	user 0.002s	sys 0.000s
I20260709 12:58:10.643036 17876 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:10.671571 18127 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.694294 18111 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:10.724709 13659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.013s	sys 0.025s
I20260709 12:58:10.619114 17381 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.732525 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.785336 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.734053 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.675297 17757 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.694756 17996 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.733012 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:10.703258 17774 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:10.740666 17939 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:10.836012 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:10.740099 18087 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:10.703722 17729 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:10.769045 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:10.841089 14241 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:10.769632 13438 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:10.835566 17774 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.790408 18170 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.786298 13539 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
W20260709 12:58:10.786769 13659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.013s	sys 0.025s
I20260709 12:58:10.724141 17918 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.785828 14240 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:10.790398 18151 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:10.793524 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 60572.4us
I20260709 12:58:10.858613 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.862854 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:10.694371 18148 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.769300 18132 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.769511 18156 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:10.785214 18144 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.843199 17774 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:10.857159 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.859383 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.894775 13540 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:10.740842 18059 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.863258 13543 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:10.851711 17843 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:10.766656 13616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.023s	sys 0.012s
W20260709 12:58:10.913841 13616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.023s	sys 0.013s
I20260709 12:58:10.867138 13582 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:10.868666 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:10.914750 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 47719.2us
I20260709 12:58:10.842581 18111 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.835485 18172 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.843524 18087 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:10.838481 18058 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:10.877637 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:10.843520 17939 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:10.991355 17939 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.896886 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:10.780162 13687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.007s	sys 0.030s
I20260709 12:58:10.847452 17757 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:10.908278 13686 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:58:10.764991 17862 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.910882 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.921262 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:10.861120 18093 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:10.835295 18127 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:10.999364 13543 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:11.004380 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.020182 18087 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:10.909052 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:10.915398 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:10.936293 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.937845 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.939872 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:11.028157 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 112829us
I20260709 12:58:10.943073 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:10.850543 13438 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:10.950495 14242 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.000s	sys 0.012s
W20260709 12:58:11.029306 14242 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.005s	sys 0.012s
I20260709 12:58:10.922812 17918 raft_consensus.cc:1064] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:10.837648 17996 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:10.985320 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 276920us
W20260709 12:58:10.984995 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 370363us
W20260709 12:58:10.996469 13687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.007s	sys 0.031s
I20260709 12:58:11.037896 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:10.998461 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 325600us
I20260709 12:58:11.000207 13691 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:10.944818 18133 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:11.039896 13584 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:10.974227 18156 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.024942 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.028928 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:11.042586 13686 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:10.871049 18170 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:10.942107 13689 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:11.000422 18173 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:10.994065 17803 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.033634 13692 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:10.913092 17876 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.021922 18176 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:10.998812 17757 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.028945 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.041507 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:10.975649 18111 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.046277 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.008870 18005 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:11.074683 17757 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.074820 18182 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:11.106875 18182 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.110015 18182 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.109872 17918 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.044018 18133 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.112191 18133 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:11.112270 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.113891 18059 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.116205 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.116294 17996 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.116959 13686 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.033537 18163 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.118096 18163 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.052403 17951 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.120471 17729 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.120719 17951 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.121706 14245 raft_consensus.cc:2368] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13.
I20260709 12:58:11.122614 17729 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.123152 18163 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.043455 18187 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:11.123813 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.124529 13538 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:11.124094 18187 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.125139 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.126252 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.126789 18187 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.127211 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.127745 18196 raft_consensus.cc:2804] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:11.082062 13438 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 11, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 11 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.072605 17869 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:10.712663 13437 rpcz_store.cc:269] 0709 12:57:50.277944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:50.284611 (+  6667us) service_pool.cc:224] Handling call
0709 12:58:10.691591 (+20406980us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:04.166099 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:04.303478 (+137379us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:04.330843 (+ 27365us) write_op.cc:435] Acquired schema lock
0709 12:58:04.332749 (+  1906us) tablet.cc:662] Decoding operations
0709 12:58:04.463664 (+130915us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:04.463769 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:04.463799 (+    30us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:58:04.465281 (+  1482us) tablet.cc:701] Row locks acquired
0709 12:58:04.465329 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:58:04.465510 (+   181us) write_op.cc:270] Start()
0709 12:58:04.465639 (+   129us) write_op.cc:276] Timestamp: P: 1783601884465445 usec, L: 0
0709 12:58:04.465675 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:58:04.483734 (+ 18059us) log.cc:844] Serialized 7050 byte log entry
0709 12:58:04.832142 (+348408us) op_driver.cc:464] REPLICATION: finished
0709 12:58:04.941430 (+109288us) write_op.cc:301] APPLY: starting
0709 12:58:04.941563 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:05.132082 (+190519us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:05.132218 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:05.146667 (+ 14449us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:05.147018 (+   351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:58:05.147072 (+    54us) write_op.cc:312] APPLY: finished
0709 12:58:05.147551 (+   479us) log.cc:844] Serialized 164 byte log entry
0709 12:58:05.147745 (+   194us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:05.186119 (+ 38374us) write_op.cc:454] Released schema lock
0709 12:58:05.217637 (+ 31518us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:05.319860 (+102223us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:05.325687 (+  5827us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1967484,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":108959,"apply.run_cpu_time_us":27987,"apply.run_wall_time_us":314491,"cfile_cache_hit":70,"cfile_cache_hit_bytes":73080,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6215,"cfile_init":2,"lbm_read_time_us":16072,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":11,"num_ops":18,"prepare.queue_time_us":639956,"prepare.run_cpu_time_us":24920,"prepare.run_wall_time_us":484443,"raft.queue_time_us":215315,"raft.run_cpu_time_us":37610,"raft.run_wall_time_us":91746,"replication_time_us":366272,"spinlock_wait_cycles":19072,"thread_start_us":883971,"threads_started":4,"wal-append.queue_time_us":129319,"wal-append.run_cpu_time_us":41622,"wal-append.run_wall_time_us":1546828}]]}
I20260709 12:58:11.060746 18170 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.064391 18173 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.064682 17803 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.127813 18186 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:11.104205 14241 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:11.136270 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.138619 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.034687 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3008) took 1317 ms. Trace:
I20260709 12:58:11.192353 13616 rpcz_store.cc:276] 0709 12:58:09.717192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:09.719262 (+  2070us) service_pool.cc:224] Handling call
0709 12:58:09.722199 (+  2937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:09.722403 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:58:10.118390 (+395987us) log.cc:844] Serialized 14 byte log entry
0709 12:58:11.034161 (+915771us) log.cc:844] Serialized 14 byte log entry
0709 12:58:11.034354 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:11.034402 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:11.034446 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:11.034519 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:11.034630 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":510115,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2806,"raft.queue_time_us":172143,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":107,"spinlock_wait_cycles":118235392,"thread_start_us":1309442,"threads_started":4,"wal-append.queue_time_us":182404}
I20260709 12:58:11.223856 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.225613 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:11.090077 18185 log.cc:927] Time spent T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:58:11.247478 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.247380 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.248677 13438 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 10, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 10 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:11.250190 13438 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.104758 18006 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.122617 17951 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.223807 18186 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:11.086941 18184 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:58:11.050096 18156 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
W20260709 12:58:11.089401 18188 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:58:11.228765 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:11.242920 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 334032us
W20260709 12:58:11.231176 18145 consensus_meta.cc:287] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.433s	user 0.020s	sys 0.000s
I20260709 12:58:11.160055 17996 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:11.227253 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:11.227166 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:11.245854 13534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.004s	sys 0.004s
W20260709 12:58:11.275353 13534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.004s	sys 0.004s
W20260709 12:58:11.246151 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 206330us
I20260709 12:58:11.084148 18111 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.084156 17862 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.076750 18179 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.097187 18127 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.082588 17381 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.262006 18186 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.256596 13438 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.074923 18122 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.168s
I20260709 12:58:11.289984 18149 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.271487 18187 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:11.265849 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:11.182636 18087 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:11.245733 17869 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:11.270169 18171 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.943s	user 0.004s	sys 0.006s
I20260709 12:58:11.270525 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.271468 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.072434 18117 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.272666 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.105137 17774 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.278689 18168 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:58:11.282202 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 253317us
I20260709 12:58:11.283707 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.282768 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.041905 18172 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:11.280048 18198 log.cc:927] Time spent T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.227s	user 0.001s	sys 0.001s
I20260709 12:58:11.290210 14238 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:11.295863 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:11.304021 18104 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:11.331946 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:11.336645 13658 leader_election.cc:400] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13.
I20260709 12:58:11.290822 18149 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.299906 13686 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:11.304209 13438 catalog_manager.cc:5697] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 1 to 10, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 10 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.309289 14242 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:11.288729 18111 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:11.311597 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.317355 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.291622 18127 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.345759 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.345949 18127 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.346544 13686 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:11.346235 13438 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.347497 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.349982 18015 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:11.350385 18015 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:11.351241 18015 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.353802 18015 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:11.352993 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.356003 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.356627 18181 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:11.357092 18181 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.357447 18181 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:11.357767 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.357975 18209 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.356622 13686 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:11.359746 18170 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.359727 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.361318 18156 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.361467 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.362140 13686 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:11.362984 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.244671 18202 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:11.344069 18151 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:11.364457 18151 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.364838 18151 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:11.365016 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.365442 17774 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.365653 13686 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:11.332801 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:11.360226 18175 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.024s	user 0.003s	sys 0.000s
I20260709 12:58:11.304138 18006 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:11.367388 18006 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.367792 18006 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:11.369213 17869 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.334022 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:11.322795 18167 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.966s	user 0.003s	sys 0.000s
I20260709 12:58:11.363070 18117 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.363615 18202 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.366562 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.347755 13438 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.329450 18168 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:11.339378 18173 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.413631 13690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 662) took 1178 ms. Trace:
I20260709 12:58:11.304085 18087 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.366612 13538 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:11.278292 18145 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.376792 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.373260 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:11.379536 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45593.9us
I20260709 12:58:11.358440 18209 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.329598 18205 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:11.378070 18196 raft_consensus.cc:697] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:11.383260 13438 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 8 to 9, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.388214 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.388636 17862 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:11.323467 18122 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:58:11.401772 18202 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.356330 18211 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.387686 17757 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.430330 14240 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 12: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:11.420712 18087 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.324333 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:11.414227 13690 rpcz_store.cc:276] 0709 12:58:10.234713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:10.234831 (+   118us) service_pool.cc:224] Handling call
0709 12:58:10.242242 (+  7411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:10.263643 (+ 21401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:10.263697 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:10.284251 (+ 20554us) log.cc:844] Serialized 26 byte log entry
0709 12:58:10.620344 (+336093us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:58:10.634216 (+ 13872us) log.cc:844] Serialized 14 byte log entry
0709 12:58:10.671330 (+ 37114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:10.671412 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:10.719756 (+ 48344us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b180001c420. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:58:11.413352 (+693596us) raft_consensus.cc:1685] finished
0709 12:58:11.413442 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:11.413569 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11269,"prepare.queue_time_us":13869,"prepare.run_cpu_time_us":1205,"prepare.run_wall_time_us":10784,"raft.queue_time_us":210062,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":6507,"spinlock_wait_cycles":41076224,"thread_start_us":335707,"threads_started":1,"wal-append.queue_time_us":567988}
I20260709 12:58:11.428263 17996 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:11.418469 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.343233 18111 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.421167 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:11.424925 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 48247.5us
W20260709 12:58:11.425526 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52337.9us
I20260709 12:58:11.412132 18168 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:11.420317 18171 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:58:11.309782 18187 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.429947 13438 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 9, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.343360 18104 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.420851 18127 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.436082 13582 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.430608 18077 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:58:11.437459 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 113233us
I20260709 12:58:11.435158 18111 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:11.439126 18196 consensus_queue.cc:237] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.440553 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.431726 18211 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.441654 18213 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:11.442624 17923 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.445497 18145 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.487811 17923 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:11.488364 17923 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:11.488682 17862 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:11.489182 17923 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.445789 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.446521 17757 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:11.430969 18193 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:58:11.448350 13438 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:11.492877 13438 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.494369 13438 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 8, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 8 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.450490 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.496214 13690 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:11.497247 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.436734 18122 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:11.453563 17803 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.454921 18172 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.500396 13692 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:58:11.500996 18149 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.503146 13438 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:11.427413 18205 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.459897 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:11.465112 18115 consensus_meta.cc:287] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.763s	user 0.010s	sys 0.000s
I20260709 12:58:11.480720 18127 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:11.484493 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 44015us
I20260709 12:58:11.514845 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:11.511885 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52059us
I20260709 12:58:11.505939 13438 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:11.467128 18202 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:11.519058 18202 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.523833 18211 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.525229 18008 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.530470 18104 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.532647 18209 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.534101 18181 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.536365 18181 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.538552 18151 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.541054 18187 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.543020 18205 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.545636 18127 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.547326 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.458345 18077 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.552371 13690 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:11.548077 13691 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:11.551285 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.571035 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.573764 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.548390 18149 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:11.576418 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.547847 18127 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.579481 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.556658 18115 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:58:11.578347 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.583508 14241 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:11.543304 18131 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.556510 18170 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:11.574107 13506 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:11.572757 18209 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:11.588425 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:11.578955 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 142944us
I20260709 12:58:11.582177 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.589538 13582 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.556910 18077 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.578994 18132 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:11.579689 18149 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.591506 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:11.583338 18005 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:58:11.598320 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.629577 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.585268 18127 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.630116 13612 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:11.636756 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:11.636404 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.638792 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.594638 18149 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:11.648451 18133 consensus_meta.cc:287] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.536s	user 0.006s	sys 0.006s
I20260709 12:58:11.648902 18133 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.650970 18133 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.641037 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.652213 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.652550 13617 raft_consensus.cc:2410] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:58:11.653451 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.634407 18186 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.655714 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.656044 18087 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.658977 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.659288 13689 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:11.659479 18202 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.659787 18151 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.659957 13612 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:11.638129 13541 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:11.642993 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.643702 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.661618 13542 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:11.644022 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.639432 13611 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:11.662508 13539 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:58:11.663002 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.663723 18186 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.664248 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.664805 13611 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:11.665342 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.665512 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.666064 13617 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:11.666549 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.666755 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.596774 18115 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:11.645538 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.645687 18122 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:11.668641 18122 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:11.669049 18122 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:11.590348 18131 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:11.672281 18131 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:11.672700 18131 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.672923 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.673512 13539 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:11.674710 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:11.674962 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.674719 18006 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.595165 18211 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:11.676586 13686 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:11.680447 18127 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.681879 18006 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.682698 18127 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.675525 13536 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:11.683568 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.684334 13538 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:11.687738 13686 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:11.687781 18122 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:11.690161 18174 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 1.367s	user 0.000s	sys 0.004s
I20260709 12:58:11.691372 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.692027 13536 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:11.671568 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.694590 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.695482 18205 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:11.695883 18205 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.695744 18149 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.696342 18205 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:11.696861 18216 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:58:11.699781 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.700709 13536 raft_consensus.cc:2410] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:58:11.702489 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:11.703028 13536 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:11.593809 18132 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:11.704806 18132 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.706095 18132 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:11.706516 18205 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.708933 18131 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.712121 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.722285 13536 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:11.726529 13540 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
W20260709 12:58:11.728425 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 69538.8us
I20260709 12:58:11.729184 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:11.730737 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:11.730883 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:11.732637 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:11.733174 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:11.737923 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.731580 13538 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:11.741807 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.759325 13617 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:11.762017 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.728128 18157 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:11.770010 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.773949 18213 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:11.775281 18205 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.782243 18210 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.731091 18186 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:11.785797 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:11.790927 13617 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.787766 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.791090 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.792516 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:11.782272 18199 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.694s	user 0.005s	sys 0.003s
I20260709 12:58:11.794337 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:11.725073 18111 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:11.767359 18122 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.795966 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.804469 18149 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.805634 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.784793 18157 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.782894 18210 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.806452 13619 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:11.792371 18196 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:11.792143 18213 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:11.801597 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.812578 17643 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:11.815172 17643 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.815603 13611 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:11.795861 18202 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:11.817600 13618 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:11.818818 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.819691 13691 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:11.825887 18111 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.830231 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.832903 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.812098 18157 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:11.809234 18186 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:11.815909 18168 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.848896 13691 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:11.850661 13506 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:11.851298 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.851727 18144 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:11.852409 18144 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:11.853603 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.855085 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.853520 18144 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.856401 13690 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:11.860373 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:11.817549 18077 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.865898 18077 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.842110 18131 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.842504 18186 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.870626 13684 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:58:11.871856 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.869051 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.876318 13690 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:11.874953 13611 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:11.878101 13506 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:11.879757 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3014) took 1378 ms. Trace:
I20260709 12:58:11.880746 13614 rpcz_store.cc:276] 0709 12:58:10.501419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:10.501523 (+   104us) service_pool.cc:224] Handling call
0709 12:58:10.520320 (+ 18797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:10.703073 (+182753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:10.703121 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:10.703182 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:10.703231 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:11.879492 (+1176261us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540023c8d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:11.879552 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:11.879691 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419804928}
I20260709 12:58:11.882754 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:11.874842 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.885130 13684 raft_consensus.cc:1240] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:11.885670 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.886853 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.888003 14244 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1183) took 1347 ms. Trace:
I20260709 12:58:11.887243 13684 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:11.888489 14244 rpcz_store.cc:276] 0709 12:58:10.530021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:10.530154 (+   133us) service_pool.cc:224] Handling call
0709 12:58:10.530460 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:10.739446 (+208986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:10.739477 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:10.739517 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:10.739556 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:11.877129 (+1137573us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400601654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:11.877192 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:11.877351 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480169216}
I20260709 12:58:11.874049 18186 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:11.893164 18170 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:11.893918 18210 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:11.876875 18043 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:11.895339 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.874408 18111 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:11.875830 18213 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:11.884348 18131 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:11.899887 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:11.892562 18202 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:11.903295 13684 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:58:11.894299 18196 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:11.906872 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:11.896165 13611 raft_consensus.cc:1240] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:11.896960 18170 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.899431 17643 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.902038 17757 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:11.899300 18043 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:11.901008 18111 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.902153 18151 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:11.924511 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:11.945291 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:11.908762 18202 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.949118 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:11.950042 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:11.949164 13538 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:58:11.908473 18196 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:11.902419 18213 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:11.909829 18217 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:11.911729 18131 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:11.962852 18041 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:11.974315 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 888) took 1399 ms. Trace:
I20260709 12:58:11.949494 18043 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:11.962443 18151 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.981729 18151 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.974699 13534 rpcz_store.cc:276] 0709 12:58:10.574904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:10.581166 (+  6262us) service_pool.cc:224] Handling call
0709 12:58:10.581409 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:10.720690 (+139281us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:58:10.721022 (+   332us) log.cc:844] Serialized 14 byte log entry
0709 12:58:11.275777 (+554755us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:11.275843 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:58:11.275886 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:11.974104 (+698218us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540028c154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:11.974145 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:11.974260 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46266,"prepare.queue_time_us":534,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":149,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2781,"spinlock_wait_cycles":319815040,"thread_start_us":554433,"threads_started":1,"wal-append.queue_time_us":519582}
I20260709 12:58:11.967088 18187 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:11.990504 18187 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:11.993065 18187 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.994733 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.996007 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.996249 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.996353 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.963686 18202 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:12.001863 18192 log.cc:927] Time spent T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.710s	user 0.001s	sys 0.000s
I20260709 12:58:12.003332 18052 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.914927 18130 consensus_queue.cc:1059] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:58:11.915074 18170 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:11.971012 18180 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.628s	user 0.000s	sys 0.013s
I20260709 12:58:11.967139 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:11.999378 18087 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.969846 18196 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:12.030495 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:11.945765 17774 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:58:12.009702 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:11.955904 18111 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.014160 13540 raft_consensus.cc:3055] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:12.040287 13540 raft_consensus.cc:740] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:11.954388 17757 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:12.033612 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:12.048883 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:12.050331 17381 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:12.034404 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:12.013964 18215 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:58:12.052767 18122 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:12.053283 18122 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.056318 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.056983 13610 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:12.058729 17960 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:11.915258 17923 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:12.035626 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:11.968149 18213 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:12.010203 18217 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:12.041294 13540 consensus_queue.cc:260] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.068935 13617 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:12.038637 18149 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:12.073335 13688 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:58:12.033016 18211 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 12:58:12.069015 18226 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:58:12.070391 18168 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.091377 13540 raft_consensus.cc:3060] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:12.084847 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:12.091586 18131 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.080847 17757 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.084129 18217 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:12.094570 18223 log.cc:927] Time spent T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:58:12.107769 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40872 (request call id 670) took 1022 ms. Trace:
I20260709 12:58:12.116671 13689 rpcz_store.cc:276] 0709 12:58:11.085756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:11.225364 (+139608us) service_pool.cc:224] Handling call
0709 12:58:12.107570 (+882206us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b540031d154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:12.107699 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994950400}
I20260709 12:58:12.108852 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.110424 13542 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:12.110085 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:12.121681 13542 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:58:12.124691 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 157647us
W20260709 12:58:12.136668 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 87936.5us
I20260709 12:58:12.137444 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:12.141716 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 68026.4us
I20260709 12:58:12.139993 13658 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.139163 13542 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.109189 18161 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:12.152122 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.147964 13540 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:12.132395 18210 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:12.152940 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.133147 17996 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:12.155207 14246 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:12.146696 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.157008 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.158485 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.159811 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.161037 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.159219 13543 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:12.137077 18008 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:12.167081 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.171823 13536 raft_consensus.cc:2393] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:12.168646 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.096163 18149 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.174310 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:12.103904 18202 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.189091 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.176649 13509 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:12.178113 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.179205 13685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 3201) took 1301 ms. Trace:
I20260709 12:58:12.181084 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:12.190549 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 151518us
I20260709 12:58:12.190593 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.147285 18122 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.190984 13685 rpcz_store.cc:276] 0709 12:58:10.878256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:10.878389 (+   133us) service_pool.cc:224] Handling call
0709 12:58:10.933547 (+ 55158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:10.990379 (+ 56832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:10.990427 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:10.990487 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:10.990531 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:12.178863 (+1188332us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b54002963d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:12.179019 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:12.179143 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130014464}
I20260709 12:58:12.147821 18187 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:12.165916 17996 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:12.148053 18151 raft_consensus.cc:2764] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:58:12.193459 13537 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:12.170696 18015 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:58:12.141328 18145 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:58:12.201434 13508 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.206533 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:12.176741 18161 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:12.171514 18008 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:12.207880 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.209456 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.211349 13613 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:12.197444 18149 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:12.189424 18148 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:12.174994 18210 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:12.187085 18196 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.188292 18087 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:12.216907 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.199326 17996 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:12.262156 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 179819us
I20260709 12:58:12.263269 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:12.224920 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.199018 18122 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:58:12.214964 18235 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:12.218155 18008 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:12.226642 18149 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:12.199637 18181 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:12.278437 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.285187 18211 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:12.285816 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.297587 13685 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:12.288038 18235 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:12.302373 18235 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.306437 13542 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:12.281898 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 113333us
I20260709 12:58:12.223869 18232 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:12.215845 18207 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:58:12.310138 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.226631 18043 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.273852 18148 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:58:12.318521 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.293743 18196 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.317766 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.321548 13537 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
W20260709 12:58:12.321045 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 96197.6us
I20260709 12:58:12.324536 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:12.324537 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 184614us
I20260709 12:58:12.325155 13508 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.325611 13615 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:12.297313 18170 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.326606 13612 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:12.327523 13582 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:12.328338 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.333226 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.316424 17951 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:12.341593 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.341938 13618 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:12.297508 18217 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.342681 18196 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:12.347661 18148 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:12.348114 18148 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:12.348248 18217 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.348914 18148 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.350279 18148 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:12.350574 18043 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:12.356546 18202 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:12.314591 18210 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.357580 18170 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.359176 18111 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.361501 18111 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:12.362315 18157 consensus_meta.cc:287] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 12:58:12.362730 18157 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.363317 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:12.343302 18213 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.365163 18157 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:12.365998 18213 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.367420 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.368544 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:12.369056 13611 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.370093 18181 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:12.370632 18181 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:12.372059 18052 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:12.372423 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.372573 18052 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.373011 13540 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:12.373682 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.374240 13534 raft_consensus.cc:3060] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:12.374666 18052 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:12.375739 13541 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:12.376641 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.377215 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.380091 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:12.381994 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:12.384563 13536 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:58:12.385073 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:12.385784 13615 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:12.322050 18232 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:12.390405 13534 raft_consensus.cc:2468] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
W20260709 12:58:12.392210 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 122657us
I20260709 12:58:12.393101 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.451371 14239 raft_consensus.cc:2368] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13.
I20260709 12:58:12.452263 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.393496 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.452865 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.453207 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.453738 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:12.394134 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.394811 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.452890 13689 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:12.399052 13538 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:12.399801 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.456537 14242 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 12: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:12.457279 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:12.457858 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:12.401577 13543 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:58:12.458909 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.404305 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.406286 18181 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:12.407716 13542 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:58:12.460522 13542 pending_rounds.cc:85] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Aborting all ops after (but not including) 5
I20260709 12:58:12.460830 13542 pending_rounds.cc:107] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:58:12.461136 13542 raft_consensus.cc:2889] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:12.464345 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.466403 13688 raft_consensus.cc:2368] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
W20260709 12:58:12.468098 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 145184us
I20260709 12:58:12.470285 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.473639 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.476826 18093 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:12.477538 18093 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:58:12.471987 18121 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:58:12.475683 17719 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:58:12.481276 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.412127 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:12.483947 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.414386 17996 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.375771 18235 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.491987 17381 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:12.415728 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.417089 18077 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:12.418870 17960 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:12.420912 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:12.424719 18186 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 12:58:12.427604 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.427848 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.503796 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.428385 14246 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:58:12.433512 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:12.434755 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:12.504940 13541 raft_consensus.cc:2393] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 12: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:58:12.505399 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70709us
I20260709 12:58:12.506341 13659 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.436655 13616 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:12.440852 13611 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:12.442679 13540 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:12.397358 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.400511 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.410609 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:12.413739 18230 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.236s	user 0.004s	sys 0.000s
I20260709 12:58:12.513857 18144 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:12.516201 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.516505 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.516988 13540 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:12.517331 13539 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.520473 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:12.504863 18077 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:12.480939 17719 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:58:12.490568 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 47819.2us
I20260709 12:58:12.427659 18008 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.429109 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:12.529094 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 100062us
I20260709 12:58:12.460291 18157 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:58:12.491214 17876 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:12.525137 17719 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:12.523289 18237 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:12.542819 18237 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:12.542991 18235 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.543274 18237 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.543130 17719 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.544320 18168 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:12.544631 17719 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.544814 18168 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:12.545223 18168 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:12.548019 18157 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:12.548197 17996 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:12.504650 17960 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:12.508626 17774 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:12.523281 17951 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:12.548496 18157 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:12.529987 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.527850 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.508088 18186 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.508152 18143 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:58:12.557571 13535 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.557471 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:12.557664 13541 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.536437 18131 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:12.580228 18179 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:12.580714 18179 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:12.581105 18179 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.581007 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.551704 18008 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:12.586813 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.587785 13536 raft_consensus.cc:2368] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13.
I20260709 12:58:12.594393 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:12.489130 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.625312 13542 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:12.625530 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:12.625705 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.626497 13534 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:12.558444 18145 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:58:12.612164 18104 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:12.630559 18077 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.632007 18077 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:12.634102 18144 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:12.634552 18144 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:12.635707 18143 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:12.636140 18143 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:12.636444 18143 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:12.613837 18157 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.574719 18131 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:12.660673 18131 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.663267 18157 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:12.668114 13536 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:58:12.621132 17774 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:12.617851 18186 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:12.649425 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 62285us
I20260709 12:58:12.656344 17876 consensus_queue.cc:1059] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:12.674295 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:12.674670 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 194549us
W20260709 12:58:12.678045 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 265988us
W20260709 12:58:12.674899 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 79392.6us
I20260709 12:58:12.689723 13619 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:58:12.696084 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:12.697124 13618 raft_consensus.cc:2368] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15.
I20260709 12:58:12.700461 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:12.676101 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 169868us
I20260709 12:58:12.712306 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.713203 13693 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:12.738902 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.742947 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.741514 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.741781 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.744537 13687 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:12.740816 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.742097 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.744978 13688 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:12.740244 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.677134 18131 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:12.744691 13540 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:12.755472 13537 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:12.756920 13534 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:12.745122 13508 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:12.751505 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:12.757903 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.759763 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:12.760390 14238 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:12.771090 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.679976 17774 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:12.785583 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:12.790699 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:12.791836 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:12.808840 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 63440.2us
I20260709 12:58:12.809876 14209 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:12.816676 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 106808us
I20260709 12:58:12.817648 13584 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:12.819594 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.820400 13685 raft_consensus.cc:2410] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:58:12.821393 13617 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:12.821714 13542 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:12.822567 13536 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: []
W20260709 12:58:12.823843 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 78801.1us
I20260709 12:58:12.824142 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:12.825294 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:12.831492 13539 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:12.792577 18205 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:12.837824 13509 leader_election.cc:400] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15.
I20260709 12:58:12.838145 13509 leader_election.cc:403] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:58:12.809167 18093 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:12.840242 13584 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:12.840713 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.849251 13693 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:12.842700 13539 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:58:12.836628 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:12.858826 14271 heartbeater.cc:646] Failed to heartbeat to 127.12.30.126:40047 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.30.126:40047 timed out after 15.000s (SENT)
I20260709 12:58:12.830200 18041 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:12.860134 18093 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.834297 18240 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:12.791375 17960 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.833179 18130 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:12.830521 18043 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:12.791988 18202 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:58:12.875039 13689 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:12.875249 13535 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:12.878000 13541 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:12.872877 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:12.829458 18111 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:12.818303 18186 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:58:12.882671 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.876611 18196 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:12.888294 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.888595 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:12.898136 13692 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:58:12.836853 18205 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:12.840895 17997 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:12.871243 18232 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15.
I20260709 12:58:12.872020 17951 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:12.885215 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.941664 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.876899 18146 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:12.928319 13614 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:12.929728 13689 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:58:12.840659 18008 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:58:12.932462 13693 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:12.945365 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:12.936246 13692 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:58:12.940454 13618 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:12.855744 18217 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:58:12.881628 18170 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:58:12.897872 18238 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:12.931094 18087 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:12.967504 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.883836 18181 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:58:12.976774 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.976848 13687 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:58:12.978535 13690 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:58:12.978650 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.979460 13691 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:12.976279 18168 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.981838 18168 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:12.984900 13692 raft_consensus.cc:1275] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:12.986593 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:12.893004 18240 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:12.990036 18144 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:12.887777 17774 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.001055 17774 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:12.967162 18146 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:13.009742 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:13.012590 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:13.018239 13685 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:13.031194 14238 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:13.057252 14243 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:58:13.058460 17876 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:13.061307 18237 consensus_meta.cc:287] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.517s	user 0.013s	sys 0.022s
I20260709 12:58:13.061800 18237 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.065251 18093 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:13.080407 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 258322us
I20260709 12:58:13.080407 18211 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:13.089051 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.007905 17960 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:13.097972 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:13.113283 13536 raft_consensus.cc:606] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:13.113651 13536 raft_consensus.cc:614] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:13.113880 13691 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:13.105450 18240 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:13.106632 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.025202 18205 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.109701 18143 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.083983 18211 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:13.025007 18241 consensus_queue.cc:1175] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" responder_term: 13 status { last_received { term: 12 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:58:13.009233 18146 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.115641 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.126893 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:13.128212 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 376749us
I20260709 12:58:13.082222 18087 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:58:13.097043 18148 consensus_meta.cc:287] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.746s	user 0.022s	sys 0.015s
I20260709 12:58:13.133807 18043 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:13.136356 18157 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:58:13.162317 13691 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:58:13.137341 18162 consensus_queue.cc:1059] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:13.138881 14211 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:13.145553 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:13.119976 18237 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:13.121030 18041 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:13.171764 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.188288 13691 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 13. Current term is 14. Ops: []
W20260709 12:58:13.216518 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 83607.5us
I20260709 12:58:13.218964 18264 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:13.219856 18264 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:13.220291 18264 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:13.220651 18264 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:13.224685 18242 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:13.228015 18162 consensus_queue.cc:1243] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:58:13.230774 18179 consensus_meta.cc:287] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.013s
I20260709 12:58:13.231138 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:13.231220 18179 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.232270 18041 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.233886 18242 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:13.234347 18144 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:13.235038 18238 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:13.239037 18157 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:13.239777 17757 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:13.240123 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:13.267865 18077 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.635s	user 0.017s	sys 0.020s
I20260709 12:58:13.246073 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:13.246781 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:13.268908 18254 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:13.248860 17658 raft_consensus.cc:2804] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:13.249963 18043 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.250638 18264 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.219190 18093 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:58:13.209313 18237 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:58:13.139024 18008 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:13.258126 13693 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:58:13.235824 18211 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.261118 18205 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:13.255736 18266 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:58:13.267259 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:13.255882 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 84190us
I20260709 12:58:13.294138 18254 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:13.347958 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:13.348675 13618 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:13.274224 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:13.216080 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 387645us
I20260709 12:58:13.351351 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.244233 18087 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:13.290509 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 51016us
I20260709 12:58:13.331688 18179 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:13.271642 18242 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:13.274088 18238 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:13.289338 18157 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:13.292954 17923 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:13.346169 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.270279 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:13.388635 13541 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:13.347077 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:13.391088 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 44094.1us
I20260709 12:58:13.392655 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.354375 13582 leader_election.cc:400] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:13.394876 13582 leader_election.cc:403] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:58:13.248260 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 109556us
I20260709 12:58:13.395823 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.397279 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.360330 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:13.360903 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:13.342608 17658 raft_consensus.cc:697] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:58:13.366760 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.368059 18111 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:13.371660 13540 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:13.378418 13535 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:58:13.404351 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 50170.3us
I20260709 12:58:13.383864 18264 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.386122 18254 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.401404 14211 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.291895 18258 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:58:13.405298 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:13.407491 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 61437.9us
I20260709 12:58:13.408597 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.410804 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.412026 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:13.411194 14246 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: [11.10-12.11]
I20260709 12:58:13.386360 17774 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:13.418421 14211 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.419390 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:13.420284 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.373373 18145 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:13.425143 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:13.347024 18008 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.414480 18202 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:13.407455 18168 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:13.402189 18196 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:13.415743 18257 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:13.406723 18259 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:13.407879 18256 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:13.417092 18213 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:13.417131 18266 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:13.420857 18217 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:13.416888 18237 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:13.421777 17719 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:13.444423 14245 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:13.429697 18104 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:13.446154 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.433620 18153 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:13.431757 18265 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:13.456836 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.417424 18179 consensus_queue.cc:1059] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:58:13.459239 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 50769.8us
I20260709 12:58:13.438014 18145 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:13.463778 18145 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.465399 14237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:13.467191 13658 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.466167 14237 raft_consensus.cc:606] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.469806 14237 raft_consensus.cc:614] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:13.423957 18261 consensus_queue.cc:1059] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:58:13.439436 18262 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:58:13.471047 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 49030.1us
I20260709 12:58:13.476620 14246 raft_consensus.cc:1240] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:13.475845 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:13.456894 18254 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:13.478858 14243 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:13.479297 14239 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:13.452044 18202 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:13.489588 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.483110 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.520788 18262 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:13.470681 17996 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:13.524536 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.471061 18260 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:58:13.526379 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59321.4us
I20260709 12:58:13.527827 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:13.531522 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.532549 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:13.481666 18259 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:13.477259 18256 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:13.445513 18157 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.437793 18242 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.466166 18237 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.545840 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:13.546406 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:13.555258 13535 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:13.540180 13582 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.457520 18266 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.562120 18259 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:13.566257 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:13.567782 17960 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:13.545614 18258 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:13.538061 18262 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:13.570266 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:13.537752 18146 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:13.539510 18260 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:13.525085 18077 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:13.456197 18238 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.542099 17923 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:13.541946 18186 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:13.568338 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:13.540637 18217 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:13.540956 18275 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:13.473119 17658 consensus_queue.cc:237] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.537290 17997 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:58:13.609581 18242 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:13.606788 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.538193 18213 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:58:13.481392 18111 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.617408 18260 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:13.604105 18157 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:13.596213 18276 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:13.658562 13613 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:13.659982 18238 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:13.660800 18273 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:58:13.605043 13614 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:13.696246 18280 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:13.609823 18262 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:13.700141 18262 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:13.681730 14244 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:58:13.682045 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 75351.6us
I20260709 12:58:13.702487 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.662042 18217 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.683753 13618 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:13.618250 13535 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:13.552470 18196 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.705817 18217 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.707372 18276 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:13.660051 18271 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:58:13.707903 18276 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.710283 18276 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.712834 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:13.712904 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:13.714071 18260 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.716184 13616 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:13.719866 18265 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:58:13.722901 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 180518us
I20260709 12:58:13.605018 17960 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.693053 17908 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:13.694188 18237 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.704084 18153 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:13.685117 18281 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:13.725704 13614 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:13.685117 18235 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:13.650336 18077 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:13.738914 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 195891us
I20260709 12:58:13.742600 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:13.609390 18258 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.691859 18008 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.613183 18202 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.747943 13509 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:13.749459 13658 leader_election.cc:400] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13.
I20260709 12:58:13.729851 18260 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.753209 13658 leader_election.cc:403] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:58:13.754580 17960 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:13.736179 17908 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:13.756939 18235 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:13.757125 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:13.757586 18235 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.759203 18266 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:13.759790 18235 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:13.752910 18272 log.cc:927] Time spent T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
W20260709 12:58:13.750425 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.765532 18077 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:13.768152 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:13.769093 13535 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:13.780951 18181 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:13.781611 18181 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:13.784314 18235 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:13.785108 18235 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:13.786062 18235 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.787885 18258 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:13.690668 18256 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:13.796016 18269 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 12:58:13.797545 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.749454 18281 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.798985 13508 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:13.799772 18264 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:13.800194 18264 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:58:13.802765 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 53443.7us
I20260709 12:58:13.803296 14211 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:13.808323 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:13.809186 13615 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:13.810771 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:13.800935 18005 consensus_queue.cc:1059] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:58:13.815722 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:13.821113 17951 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:13.821658 17951 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:13.822677 17951 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.823770 17951 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:13.828912 18258 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:13.831578 18258 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:13.832121 18258 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:13.833344 18258 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.835021 18258 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:13.835908 18104 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:13.836509 18104 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:13.836988 18104 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:13.843598 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:13.844262 13539 raft_consensus.cc:606] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:13.844257 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.846968 18168 raft_consensus.cc:3055] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:13.861534 14240 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: [11.10-12.11]
I20260709 12:58:13.804345 18261 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:13.804766 17960 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:13.813897 18153 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:58:13.818849 17658 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:13.756893 17908 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.801083 18264 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:13.877547 18282 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
I20260709 12:58:13.804374 18281 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:13.804466 18265 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13.
I20260709 12:58:13.864233 18236 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:13.954011 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:13.905119 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:13.847888 18168 raft_consensus.cc:740] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:13.956987 13617 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 14: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:13.957794 18168 consensus_queue.cc:260] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.958849 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:13.959067 18168 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:13.959452 13685 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:13.919711 17960 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.961560 17960 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:13.961901 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:13.962495 13539 raft_consensus.cc:606] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:13.962807 13539 raft_consensus.cc:614] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:13.967006 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:13.967180 18264 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:13.967628 13539 raft_consensus.cc:606] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:13.968050 13539 raft_consensus.cc:614] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:58:13.968194 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.969146 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:58:13.970954 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:13.971326 18157 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:13.972138 17658 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:13.973408 17658 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:13.889371 18143 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:13.980580 18181 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.950182 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:13.892195 18261 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:13.783725 18262 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:13.971895 18157 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:13.987324 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.932389 14242 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 12:58:13.954027 13616 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:13.991235 14242 pending_rounds.cc:85] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 10
I20260709 12:58:13.991614 14242 pending_rounds.cc:107] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 12:58:13.992049 14242 raft_consensus.cc:2889] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:58:14.000183 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:13.982882 13691 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
W20260709 12:58:14.013473 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 169860us
I20260709 12:58:14.014549 13658 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:13.940801 18241 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:14.013587 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.952562 18236 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:14.033540 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:14.004904 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:14.033295 13611 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:13.985014 17908 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:13.981904 18181 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:14.020327 18258 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:14.057729 18041 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:14.036914 18288 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:14.063828 13685 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:14.035181 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:13.995869 18261 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:14.046433 18265 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:14.047864 18252 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:58:14.072336 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.080747 13616 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:14.032193 18254 consensus_meta.cc:287] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
I20260709 12:58:14.073194 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.020618 18281 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:14.074796 13615 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:14.075546 13614 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:14.083719 13619 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:14.019470 18241 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:14.075901 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.089015 13618 raft_consensus.cc:2393] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:14.089208 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.075570 13617 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:14.032514 17960 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:14.077358 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.078138 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.045773 18238 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:58:14.096308 13612 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:14.097143 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 62064.9us
I20260709 12:58:14.097601 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.038586 18236 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:14.098167 13658 leader_election.cc:400] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13.
I20260709 12:58:14.102715 13658 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:14.103684 13610 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
W20260709 12:58:14.039575 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 223910us
I20260709 12:58:14.094517 17960 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:14.100193 18241 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.110581 13613 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:14.112668 13543 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
W20260709 12:58:14.125871 13582 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.731s (SENT)
I20260709 12:58:14.126381 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:14.138940 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:14.140595 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:14.142022 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:14.120604 17960 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:14.090123 18104 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.147528 17951 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:14.147619 18266 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:14.130796 18292 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:14.120409 18265 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:14.153863 18093 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:14.147691 18008 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:14.176137 18143 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:58:14.183137 17381 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 12:58:14.176206 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 44290.6us
I20260709 12:58:14.133699 18297 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:14.193048 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:14.193943 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:14.196393 18299 log.cc:927] Time spent T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:58:14.196321 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:14.160425 18235 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:14.197613 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:14.148046 18237 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:14.189517 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:14.156898 18241 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:14.203320 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.286006 13616 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:14.148254 18242 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.204770 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:14.290839 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 164986us
I20260709 12:58:14.136129 18277 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:58:14.196689 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:14.295581 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.296934 13540 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:14.283697 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.198726 18093 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:14.320209 14238 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:14.159667 18288 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
I20260709 12:58:14.319684 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:14.329775 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:14.330454 13685 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:14.332671 13689 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:14.333797 13689 raft_consensus.cc:1240] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
W20260709 12:58:14.342090 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.343784 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:14.317564 18271 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:58:14.344947 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:14.342826 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:14.317633 18008 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.349663 13691 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:14.350104 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:14.362833 13685 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:14.364354 13686 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:58:14.370308 13686 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:14.371701 13686 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:14.375581 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:14.375841 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:14.376407 13690 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:14.378118 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:14.382457 18309 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 12:58:14.383179 18294 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:58:14.383324 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:14.384059 18264 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:14.384505 18264 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:14.384889 18264 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:14.385569 18259 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:14.387187 18259 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:14.387749 18259 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:14.388535 13687 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:14.389552 13692 raft_consensus.cc:1240] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:58:14.383802 18237 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:14.331413 18262 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:14.341045 18292 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:14.388439 18306 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 12:58:14.404027 13684 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:58:14.287835 18235 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.404390 18146 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:14.321262 18242 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.348184 18295 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:14.344126 17757 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:58:14.363333 18211 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:58:14.374130 18186 consensus_queue.cc:1048] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 12:58:14.463521 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.437417 13691 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
W20260709 12:58:14.474685 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 269767us
I20260709 12:58:14.476166 13535 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:14.477624 18314 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:14.478248 18314 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:14.480866 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49176us
W20260709 12:58:14.481076 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.435834 18295 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:14.431170 17719 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 12:58:14.485481 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 100868us
I20260709 12:58:14.437211 18277 consensus_queue.cc:1059] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:58:14.393445 18008 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.490811 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:14.483860 18314 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.424908 18292 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.162376 18041 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Committing config change with OpId 11.11: config changed from index 5 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:14.510659 13693 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:58:14.515023 18311 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:58:14.515837 18235 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:14.404474 18265 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.423316 18093 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.493867 18295 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:14.544196 18288 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:14.546618 18327 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:14.547073 18327 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:14.547276 18008 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:14.546943 18288 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.548092 18327 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.549204 18288 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:14.549510 18327 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:14.555477 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:14.534250 18260 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:14.569151 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:14.539377 18104 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:14.540303 18314 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.500073 18261 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.577533 18320 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.581013 18320 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.583274 18157 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:14.587968 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:14.588923 13690 raft_consensus.cc:2410] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:58:14.597659 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:14.598531 13536 raft_consensus.cc:2393] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:14.600847 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3047) took 1037 ms. Trace:
I20260709 12:58:14.601226 13610 rpcz_store.cc:276] 0709 12:58:13.563473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:13.563608 (+   135us) service_pool.cc:224] Handling call
0709 12:58:13.564179 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:14.104166 (+539987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:14.600626 (+496460us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b540013abd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:14.600791 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240312704}
I20260709 12:58:14.603137 13543 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:14.606464 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1205) took 1195 ms. Trace:
I20260709 12:58:14.606961 14242 rpcz_store.cc:276] 0709 12:58:13.411520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:13.413043 (+  1523us) service_pool.cc:224] Handling call
0709 12:58:13.426841 (+ 13798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:13.992583 (+565742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:14.606246 (+613663us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54005ce754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:14.606410 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161017216}
I20260709 12:58:14.608554 17381 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:58:14.610387 18278 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.826s	user 0.000s	sys 0.004s
I20260709 12:58:14.612900 18235 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:14.613266 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.613430 18235 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.583318 18320 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.615634 18235 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.635284 13616 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15. Ops: []
W20260709 12:58:14.637635 18236 consensus_meta.cc:287] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.000s
I20260709 12:58:14.638142 18236 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.639868 13538 raft_consensus.cc:2368] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
W20260709 12:58:14.606383 17908 consensus_meta.cc:287] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.000s
I20260709 12:58:14.644767 18295 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.601672 18292 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:14.618196 18205 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:14.569443 18260 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:14.622238 18264 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.624320 18242 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.625157 13541 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:58:14.622262 13689 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:14.679262 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.626509 18261 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:14.626950 18311 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:58:14.627449 18314 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.630916 13611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3050) took 1066 ms. Trace:
I20260709 12:58:14.698324 13611 rpcz_store.cc:276] 0709 12:58:13.564689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:13.565572 (+   883us) service_pool.cc:224] Handling call
0709 12:58:13.565911 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:14.034516 (+468605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:14.630633 (+596117us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54002b3754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:14.630834 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073642880}
I20260709 12:58:14.631871 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:14.649389 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:14.700199 13616 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:14.656669 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:14.660364 14239 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:58:14.662238 13543 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:58:14.701365 14239 pending_rounds.cc:85] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 10
I20260709 12:58:14.701649 14239 pending_rounds.cc:107] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:58:14.666401 14238 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:14.701879 14239 raft_consensus.cc:2889] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:14.668784 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.664184 18205 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:58:14.663859 18181 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.593s	user 0.008s	sys 0.003s
I20260709 12:58:14.675441 18260 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:14.692586 18104 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.695068 18324 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.695982 18077 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.696910 14245 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:14.702884 13613 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:14.664428 18304 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.682976 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.667606 18196 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:14.711580 13543 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:58:14.716410 18264 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.718536 13693 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:14.797067 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.798375 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.798709 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:14.799026 14237 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:14.799367 14246 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:14.773216 18258 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:14.720647 18170 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:14.721364 18217 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:14.674870 18146 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:58:14.728432 13616 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
W20260709 12:58:14.835927 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.839624 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:14.840456 13692 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:58:14.803244 18258 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.724844 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:14.843499 13611 raft_consensus.cc:2368] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:58:14.678360 14242 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 5 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:14.680450 18277 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:14.793510 18298 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.805758 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.730820 13536 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 5 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:14.853410 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:14.807972 18259 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:14.772810 18324 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.808244 13508 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:14.858528 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.810024 14240 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:14.810312 14246 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:14.729740 18295 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:14.819986 18304 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.821054 18236 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:14.824180 17960 consensus_meta.cc:287] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.647s	user 0.008s	sys 0.003s
I20260709 12:58:14.825376 18196 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.794759 18077 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.830971 18153 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:14.806833 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.865311 13659 leader_election.cc:400] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:58:14.868845 13659 leader_election.cc:403] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 12:58:14.872651 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 64499.8us
I20260709 12:58:14.873631 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:14.874400 13508 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:14.854214 18320 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.857491 18265 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.881611 14245 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:14.773744 18104 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.857837 18330 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.868553 18153 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:14.867502 18254 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:14.893152 13692 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:14.892288 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:14.927652 13506 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:14.928586 18242 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:14.929177 18242 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:14.929984 18242 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.880189 13538 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:14.861685 18259 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.934587 18259 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.876108 18264 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:14.935322 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:14.857246 17908 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:58:14.878535 18240 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:14.868000 18262 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:58:14.944921 13542 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:14.883292 18236 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:14.945953 18236 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:14.904407 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:14.948381 13688 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:14.883502 18144 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:14.858594 18298 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.949514 18144 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:14.950024 18144 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:14.951548 18298 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:14.868678 17960 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.874833 18277 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.954339 17960 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.954921 18277 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:14.882798 17381 consensus_queue.cc:1059] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:58:14.955519 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:14.956543 17381 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:14.957086 17381 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:14.957582 17381 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:14.957962 13619 raft_consensus.cc:2368] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15.
I20260709 12:58:14.957775 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:14.960228 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:14.960798 14246 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:14.960583 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:14.961565 13688 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:14.963555 13506 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:14.947288 18236 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.966580 17381 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.969589 14246 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:14.972486 18181 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:14.973223 18181 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:14.956828 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:14.978654 13540 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:14.979547 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:14.970546 17381 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:14.985610 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:14.986747 13688 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:58:14.980106 18181 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:14.980041 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:14.989529 13611 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:14.991068 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:14.992106 18181 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:14.992553 18181 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:14.992942 18181 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:15.000883 18181 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:15.003638 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 109476us
I20260709 12:58:15.033023 18324 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:15.034097 18335 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:58:15.034441 18304 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:15.036881 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.039094 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:15.040102 18318 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:15.039968 17960 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:15.039287 18260 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.043960 18260 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:14.888012 18265 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.260515 18265 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:14.916359 18237 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.261198 18237 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.263252 18237 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:14.898468 18330 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:14.883634 18320 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.263788 13686 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:15.265501 18320 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.265729 18330 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:15.266593 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.267544 14243 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:14.907757 18314 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:15.056545 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:15.057588 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:15.065507 18323 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:15.268832 13539 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:15.066788 18327 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:15.069186 18196 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.069967 18258 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.071496 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:15.072085 18077 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:15.271128 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 199712us
I20260709 12:58:15.073900 18104 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:15.074884 14245 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:58:15.074771 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:15.033035 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:15.078778 13613 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:15.081478 13692 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:15.082930 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:15.083379 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:15.084782 13684 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:15.084872 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.087514 13617 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:15.091034 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.091553 13687 raft_consensus.cc:1275] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:58:15.094313 14212 leader_election.cc:400] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15.
I20260709 12:58:15.276041 13686 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:15.276178 14212 leader_election.cc:403] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:58:15.276760 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 182589us
I20260709 12:58:15.276057 13687 pending_rounds.cc:85] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 9
I20260709 12:58:15.278752 13687 pending_rounds.cc:107] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:58:15.278898 13687 raft_consensus.cc:2889] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:15.095257 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:15.279939 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 223464us
I20260709 12:58:15.096619 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:15.013549 13537 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:15.099440 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:15.102239 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:15.100739 18322 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:15.138127 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:15.288513 13615 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:15.289620 13619 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:15.138522 13688 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:15.155791 13614 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:15.100932 18318 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:15.169960 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:15.107084 17908 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:15.011284 18144 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.181704 18262 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:15.182554 17960 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.194384 18181 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:15.185670 18336 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.291615 18252 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15.
I20260709 12:58:15.256666 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:15.330698 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:15.331498 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 75029.3us
I20260709 12:58:15.332410 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:15.300526 18236 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:14.918921 18254 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:15.283809 13613 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:15.284099 13618 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:15.291095 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:15.291061 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:15.299114 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:15.308295 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:15.279974 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 222458us
I20260709 12:58:15.337381 13615 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:15.304803 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:15.306205 13506 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:15.272115 18196 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:15.256111 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.322726 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.341046 13693 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:15.321664 17960 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.297019 18320 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:15.300734 17774 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.333642 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:15.300464 18286 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.334165 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:15.334849 13539 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15. Ops: []
W20260709 12:58:15.337080 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 167216us
I20260709 12:58:15.338306 13619 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:15.314806 13616 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:15.319351 18264 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
W20260709 12:58:15.351433 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:15.301136 18322 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.301381 18327 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:58:15.361744 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:15.335847 18181 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:15.364468 18286 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.360805 17774 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.366706 18258 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:15.334345 18336 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.368670 18144 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.369350 18254 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.359637 18314 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:15.355242 18320 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:58:15.353780 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 48170.6us
I20260709 12:58:15.372550 17381 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:15.445304 17381 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:15.445777 17381 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:15.424914 18241 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:15.336051 18236 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:15.458096 18236 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:15.458289 13619 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:15.459086 17381 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.385069 18041 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:58:15.461241 17381 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.320616 18262 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.334823 18330 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:58:15.422886 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:15.423278 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:15.385087 18145 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:15.359391 18259 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:15.382274 18196 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:15.424366 18335 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:58:15.433984 18240 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:58:15.444384 18314 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:15.424608 18162 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:15.349014 18237 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:15.432245 18143 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.451653 13506 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:15.533895 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 98968.3us
I20260709 12:58:15.452380 17774 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:15.540562 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 88980.9us
I20260709 12:58:15.540758 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:15.541272 13506 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:15.448360 18320 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:15.457607 17960 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:15.459012 13690 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:15.383232 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1811) took 1187 ms. Trace:
I20260709 12:58:15.464883 13612 raft_consensus.cc:2393] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 10: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:15.421173 18264 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 12:58:15.465898 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 132378us
W20260709 12:58:15.502192 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 69914.4us
I20260709 12:58:15.421057 18261 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:15.599443 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51063.7us
I20260709 12:58:15.600395 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:15.601323 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:15.523579 18314 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:15.520533 18196 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:15.532023 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 174148us
I20260709 12:58:15.434197 18258 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:15.473094 18093 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:15.541610 14239 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:15.535548 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:15.543177 13541 rpcz_store.cc:276] 0709 12:58:14.196029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:14.197006 (+   977us) service_pool.cc:224] Handling call
0709 12:58:14.197399 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:14.678068 (+480669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:15.382998 (+704930us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54000ab254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:15.383178 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983209344}
W20260709 12:58:15.533490 18348 log.cc:927] Time spent T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:58:15.466127 18241 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.468533 18145 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:58:15.603408 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 52321.6us
W20260709 12:58:15.627863 18350 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:58:15.629266 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:15.630194 18346 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:58:15.632148 18041 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:15.634644 18314 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.641125 18314 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:15.646003 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 690) took 1310 ms. Trace:
I20260709 12:58:15.646567 13689 rpcz_store.cc:276] 0709 12:58:14.335875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:14.380751 (+ 44876us) service_pool.cc:224] Handling call
0709 12:58:14.381118 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:14.678330 (+297212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:15.645635 (+967305us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540031d154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:15.645916 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553779072}
I20260709 12:58:15.654935 18361 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:15.656958 18336 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.602248 13658 leader_election.cc:400] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:58:15.607481 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.657570 13658 leader_election.cc:403] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:15.657857 13691 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:15.522235 18162 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:58:15.658326 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 56227.6us
I20260709 12:58:15.659212 18162 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.602419 18235 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:15.660452 18235 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:15.660836 18235 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:15.661401 18262 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.664832 18322 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.545171 18259 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:15.604002 18237 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.549394 18242 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:15.667157 18237 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.670902 18235 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:15.671429 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:15.674715 18314 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:15.675235 18314 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:15.620826 18335 log.cc:927] Time spent T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:58:15.676895 18286 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:15.613026 18261 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.618902 14245 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:15.681620 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:15.667503 18242 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:15.686028 13542 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:58:15.686270 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:15.666191 18259 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.664382 18205 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.689251 13619 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:15.689592 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.619204 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:15.707224 13692 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:15.677301 18314 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.708478 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:15.716182 14244 raft_consensus.cc:2393] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 11: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
W20260709 12:58:15.712803 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 93673.4us
I20260709 12:58:15.718214 13689 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:15.714205 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:15.715265 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:15.721026 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:15.723472 13684 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:58:15.725149 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:15.725649 13684 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:15.618340 18258 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:15.726227 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:15.731846 14238 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:15.732649 14239 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:15.734207 13535 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:15.739434 13692 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 16.
I20260709 12:58:15.740464 13689 raft_consensus.cc:1240] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:58:15.744299 13690 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:15.746554 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:15.658634 18266 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:58:15.756853 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:15.614262 18283 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:58:15.750608 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:15.751433 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.762883 14239 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:15.767113 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:15.767807 13684 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:15.768426 14244 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:15.714804 18265 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.727816 18356 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:15.748251 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:15.543545 18143 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.726846 18241 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:15.778264 13690 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:15.771512 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:15.717497 18168 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:58:15.717535 18322 consensus_queue.cc:1059] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 12:58:15.788730 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 431885us
I20260709 12:58:15.728171 18104 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:58:15.800346 18336 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:15.800872 18336 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:15.801292 18336 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:15.627897 18145 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.797457 18143 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:15.810714 18181 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:58:15.820617 18242 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:15.824527 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:15.829007 18162 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:15.786243 18146 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:15.833320 18146 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:15.779502 18265 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.791206 18356 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:15.850543 18356 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:15.719699 18361 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
W20260709 12:58:15.863400 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 73991.3us
I20260709 12:58:15.865530 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:15.851873 18148 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:58:15.910477 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:15.873526 18213 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:58:15.911396 18148 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:58:15.935513 18358 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:58:15.875370 18352 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
W20260709 12:58:15.992362 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 271758us
I20260709 12:58:15.995702 13508 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.000519 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.011015 13693 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:15.989833 18235 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.012714 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.022493 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.023311 13538 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
W20260709 12:58:16.024572 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 300741us
I20260709 12:58:16.027391 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:16.027736 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.028713 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.028942 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:16.029353 13538 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:16.029752 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.030475 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:16.031015 13613 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:16.031177 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:16.028875 13617 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:16.036361 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.037249 13538 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:16.037711 13613 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:16.039992 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:16.041100 13612 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:16.040033 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.042249 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.042505 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.025476 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.007437 18265 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:16.048749 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.065801 13543 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:16.066704 17381 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.067425 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.048584 13538 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:16.069355 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.070158 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.070660 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.071830 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.072222 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.070283 13539 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:16.073768 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.073956 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.081200 13541 raft_consensus.cc:1240] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:58:16.061767 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:16.067247 17381 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.102638 18181 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.103483 18181 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.106164 18181 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:16.108739 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 382902us
I20260709 12:58:16.008919 18261 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.016376 18298 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:16.111826 14237 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:58:16.117327 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 343222us
I20260709 12:58:16.122675 17381 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:16.131250 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 404226us
I20260709 12:58:16.061702 18111 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.062830 18146 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:16.138789 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:16.108461 18144 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:16.142908 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:16.145036 13658 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.152263 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:16.133695 18351 log.cc:927] Time spent T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:58:16.162783 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.156199 13690 raft_consensus.cc:2393] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:16.164372 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:16.163426 13536 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:16.158411 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.167179 13506 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.167457 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:16.160621 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:16.170929 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.177021 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.180629 18363 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:16.181361 18363 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:16.182343 18363 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.136080 18181 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:16.038000 18258 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.136691 18298 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:58:16.155640 18320 consensus_meta.cc:287] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.004s
I20260709 12:58:16.040109 18148 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.200917 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:16.211890 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.168804 18217 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:16.127369 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.235770 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.155375 18261 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:58:16.156909 17658 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:58:16.162251 18295 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:58:16.208359 18298 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:16.261734 18242 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:58:16.251036 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 123739us
I20260709 12:58:16.159358 18327 consensus_queue.cc:1048] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 12:58:16.168720 18144 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.160128 18111 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.241815 18143 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:16.303865 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:16.304070 13610 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:58:16.263442 18093 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.184265 18181 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.270231 18241 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:16.273118 18361 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.286825 18304 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:58:16.262928 18264 consensus_queue.cc:1059] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:58:16.286816 18242 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:58:16.428835 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:16.276703 18375 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.349663 18148 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:16.293190 18217 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:16.235716 18320 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:16.438858 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 271472us
W20260709 12:58:16.436674 18236 consensus_meta.cc:287] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.976s	user 0.011s	sys 0.000s
I20260709 12:58:16.352434 18241 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:16.464429 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.237615 18153 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.350070 18354 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:16.498889 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.438416 18359 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:16.500244 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 333112us
I20260709 12:58:16.505734 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.506647 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.352105 18093 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.455962 18368 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.523873 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.506294 14243 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:16.506258 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:16.498118 18336 consensus_meta.cc:287] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.002s
I20260709 12:58:16.534185 13689 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:58:16.506493 18241 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:16.541241 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:16.542006 13689 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:16.503525 18236 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.543520 18242 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.545063 18181 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.546056 18181 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:16.546625 18236 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.511076 18370 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.548987 13534 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:16.516551 18356 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:16.530131 18153 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:16.441874 18361 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.530069 17908 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:16.529670 18261 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:16.575536 18365 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
I20260709 12:58:16.575970 18322 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.551981 18235 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:16.567340 18372 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.560869 18236 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:16.463897 18375 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:16.597224 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 98428.9us
I20260709 12:58:16.517925 18354 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.607438 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.622087 13658 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.622759 14238 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:16.623642 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.538605 18368 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.627339 14238 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:58:16.628481 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:16.539747 18336 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.630070 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.595517 17908 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:16.595589 18262 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:16.636826 18391 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:58:16.637285 18262 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:16.638072 18386 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:58:16.595219 18388 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:16.636694 18395 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:58:16.614609 18235 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:16.618427 18380 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.571628 18217 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.661259 18205 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:16.619654 18322 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.649369 18144 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.651579 18361 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:16.654112 18407 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:16.655040 18354 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.642103 18304 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:16.692132 18304 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:16.692703 18304 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:16.625784 18258 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:16.658284 18265 raft_consensus.cc:3055] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:16.659083 18111 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:16.672636 18379 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:16.672998 18381 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.605753 18261 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.626147 18236 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.673012 18093 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:16.680181 18385 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.062s	user 0.000s	sys 0.002s
I20260709 12:58:16.680975 18401 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:16.682075 18322 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:16.683187 18406 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:16.689522 18262 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.691133 18405 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:16.644359 18394 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:58:16.647145 18235 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:16.630504 18143 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:16.568274 18370 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.659981 18380 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:16.722078 18398 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 12:58:16.724089 18241 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:16.729418 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 105842us
I20260709 12:58:16.622008 18372 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.536938 18359 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.700173 18265 raft_consensus.cc:740] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:16.716055 18258 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:16.742033 18405 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.751643 13686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1746) took 1028 ms. Trace:
I20260709 12:58:16.717491 18381 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:16.764484 18162 consensus_meta.cc:287] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.935s	user 0.014s	sys 0.003s
I20260709 12:58:16.762982 13686 rpcz_store.cc:276] 0709 12:58:15.723347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.726185 (+  2838us) service_pool.cc:224] Handling call
0709 12:58:15.758739 (+ 32554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:15.994922 (+236183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:15.994969 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:15.995505 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:58:16.237846 (+242341us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:16.348564 (+110718us) log.cc:844] Serialized 14 byte log entry
0709 12:58:16.516143 (+167579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:16.516225 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:16.751426 (+235201us) raft_consensus.cc:1685] finished
0709 12:58:16.751450 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:16.751574 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95016,"prepare.queue_time_us":269484,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":5561,"raft.queue_time_us":627862,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":9133,"thread_start_us":632069,"threads_started":4,"wal-append.queue_time_us":556166}
I20260709 12:58:16.765512 18381 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.765661 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:16.740283 18320 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:16.767454 13686 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:16.769654 18359 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:16.769842 18393 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:58:16.770654 18378 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:58:16.771346 18375 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:16.771189 18143 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.772279 18368 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.773185 18241 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.773875 18370 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:16.774571 18396 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:58:16.748620 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.791528 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 710) took 1066 ms. Trace:
W20260709 12:58:16.807607 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59081.1us
I20260709 12:58:16.808630 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.808442 13684 rpcz_store.cc:276] 0709 12:58:15.724671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.768777 (+ 44106us) service_pool.cc:224] Handling call
0709 12:58:15.769230 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:16.227672 (+458442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:16.227705 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:16.228245 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:58:16.439152 (+210907us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:58:16.439193 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:16.439259 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:16.721427 (+282168us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:16.786291 (+ 64864us) raft_consensus.cc:1685] finished
0709 12:58:16.786313 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:16.791461 (+  5148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6460,"raft.queue_time_us":332740,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"spinlock_wait_cycles":16830208,"thread_start_us":661956,"threads_started":2,"wal-append.queue_time_us":289180}
I20260709 12:58:16.766518 18405 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:16.777902 18043 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.786123 18407 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.765897 18336 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.814522 18375 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:16.815115 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:16.817090 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:16.817305 18402 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:58:16.817785 13686 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:16.827504 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:16.828388 13684 raft_consensus.cc:2393] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:16.807778 18241 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.778246 18258 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.837884 13536 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:16.836022 18389 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.842096 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.842252 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.842252 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.845292 13536 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:16.828893 18375 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:16.848495 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1719) took 1151 ms. Trace:
I20260709 12:58:16.848851 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:16.850582 13542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1715) took 1247 ms. Trace:
I20260709 12:58:16.848912 13535 rpcz_store.cc:276] 0709 12:58:15.697388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.698370 (+   982us) service_pool.cc:224] Handling call
0709 12:58:15.729151 (+ 30781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:15.734620 (+  5469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:16.848237 (+1113617us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b54000b86d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:16.848439 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10900864}
I20260709 12:58:16.850960 13542 rpcz_store.cc:276] 0709 12:58:15.603256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.604549 (+  1293us) service_pool.cc:224] Handling call
0709 12:58:15.604970 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:15.707080 (+102110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:16.850290 (+1143210us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b54003cb754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:16.850489 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185841920}
I20260709 12:58:16.845973 18370 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:16.829887 18407 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:16.853662 18411 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:58:16.795243 18265 consensus_queue.cc:260] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.848552 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.841331 18241 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.857913 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.859757 18412 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:16.860396 18412 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:16.858886 18320 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:16.862165 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:16.862813 13686 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:16.863994 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:16.840946 18389 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.853762 18370 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:16.868621 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.870404 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.870800 14240 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:16.858551 13536 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:16.869859 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.871294 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:16.872130 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:16.869292 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.837602 18043 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:16.877367 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.861259 18412 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.854143 18375 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.882519 17719 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:16.883054 17719 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.886550 18145 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:16.875811 13508 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:16.858772 18236 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:16.900920 13508 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.901110 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:16.876456 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.904022 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:16.870047 18265 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:16.874256 18370 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:16.873656 18148 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:16.910949 18403 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:58:16.911989 18043 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:58:16.862550 17908 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.861860 18320 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.915819 13508 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:16.916276 18392 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:58:16.914644 18148 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.924806 17908 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:16.904305 18093 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:16.924609 18236 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:16.927242 13539 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:16.905915 18401 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:16.905810 18367 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:16.933358 13543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1717) took 1301 ms. Trace:
I20260709 12:58:16.935376 13543 rpcz_store.cc:276] 0709 12:58:15.631861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.681377 (+ 49516us) service_pool.cc:224] Handling call
0709 12:58:16.933188 (+1251811us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54000b90d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:16.933289 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":881846656}
I20260709 12:58:16.941157 18148 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.940045 14244 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:16.939738 18153 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:16.945674 18153 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:16.929898 18236 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:16.948958 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.931025 18401 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:16.948379 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.953071 17960 raft_consensus.cc:3055] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:16.954368 18258 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.957572 18237 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:16.958757 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:16.958928 13534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1721) took 1214 ms. Trace:
I20260709 12:58:16.935146 18367 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:16.963892 18401 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:16.968019 18367 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:16.967794 13534 rpcz_store.cc:276] 0709 12:58:15.744273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.744419 (+   146us) service_pool.cc:224] Handling call
0709 12:58:16.958726 (+1214307us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b540028d554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:16.958866 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843838336}
I20260709 12:58:16.966109 18144 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:16.970168 18144 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:16.971447 18241 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.964994 17960 raft_consensus.cc:740] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:16.938668 18380 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:16.972690 18261 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:16.972594 17960 consensus_queue.cc:260] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.976930 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:16.977567 14239 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:16.978673 13508 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:16.971210 18144 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.980898 18407 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:16.950902 18236 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:58:16.983356 18298 consensus_meta.cc:287] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.001s
I20260709 12:58:16.984227 18241 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:16.988130 18241 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:16.988004 18298 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:16.988768 18241 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:17.000774 18388 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:17.011783 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.016149 13690 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:17.017088 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:17.020138 18372 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.020874 18367 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.021283 18370 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.023650 18145 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:17.022701 18241 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.024277 18367 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.025591 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.026170 14239 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:17.026154 18304 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.028236 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.028929 13690 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:17.029784 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:16.899617 14242 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:17.023332 18370 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.035851 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.034147 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.037668 14239 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:17.032099 18368 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:17.039245 13508 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.046582 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:17.052011 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:17.052717 13690 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:17.060068 13582 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.731s (SENT)
I20260709 12:58:17.067605 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.036832 18205 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.069033 18144 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:17.071010 18144 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.071695 18205 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.073167 18144 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.074334 18205 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.066030 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.038411 18367 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:17.024277 18145 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.076030 17960 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:17.076769 18241 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.079540 18153 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.084821 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:17.085460 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.087435 13690 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:17.051054 18370 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:17.103984 18370 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:17.064070 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:17.083777 18412 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.095402 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.067211 18304 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.082820 18235 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.102557 14242 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:17.093016 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:17.106278 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 93825.1us
I20260709 12:58:17.107136 13690 raft_consensus.cc:1240] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:17.112788 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.113410 13690 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:17.114254 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.115286 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.109596 14239 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 15: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:17.090502 18211 consensus_queue.cc:1059] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 12:58:17.116840 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 52945.9us
I20260709 12:58:17.115839 13690 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:17.117303 14242 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:17.102154 18367 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:17.118688 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.119460 14211 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:17.104808 18372 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.121062 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.097726 18401 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.124128 18372 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.124469 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:17.109609 18412 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.126799 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.128647 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:17.129232 18145 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.120766 18327 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:17.130292 17876 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:17.123363 18407 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.118292 18181 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.133720 13691 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:58:17.134478 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:17.135656 18330 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:17.135185 18367 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.137666 18372 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:17.213475 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 79063.7us
I20260709 12:58:17.204788 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.164361 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.168556 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.126430 18162 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:17.216286 14243 raft_consensus.cc:1240] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:58:17.077716 13540 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:17.369000 13540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 934) took 1321 ms. Trace:
I20260709 12:58:17.369519 13540 rpcz_store.cc:276] 0709 12:58:16.047197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:16.074972 (+ 27775us) service_pool.cc:224] Handling call
0709 12:58:17.077388 (+1002416us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:58:17.077538 (+   150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:17.368466 (+290928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:17.368759 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82859648}
I20260709 12:58:17.221896 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.371222 18371 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.220129 14239 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:17.372232 18371 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.377391 14238 raft_consensus.cc:2393] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:17.377465 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:17.308372 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:17.180253 17997 raft_consensus.cc:3055] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:17.396026 17997 raft_consensus.cc:740] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:17.398533 17997 consensus_queue.cc:260] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.218926 18168 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:17.153530 18235 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.216323 18181 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.158917 18236 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.176128 18298 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.164955 18320 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.138126 18370 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.167860 18375 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.162452 18389 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:17.154776 18390 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.482s	user 0.003s	sys 0.000s
I20260709 12:58:17.154681 18401 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.311196 13508 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.134271 18405 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.444607 18298 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:17.445338 18298 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:17.446735 18405 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.448096 13686 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:17.450707 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:17.451511 13686 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:17.142019 18407 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:17.144402 18412 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.462472 17960 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:58:17.303319 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 218351us
I20260709 12:58:17.335227 13687 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:58:17.156474 13693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1740) took 1441 ms (client timeout 1731 ms). Trace:
I20260709 12:58:17.339093 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:17.145112 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1745) took 1422 ms. Trace:
I20260709 12:58:17.344326 18111 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:17.349665 17774 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.350828 13691 pending_rounds.cc:85] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 8
I20260709 12:58:17.351693 18361 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.358929 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.360345 18410 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.361145 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.364290 18162 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:17.363739 18368 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.468194 17774 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.468566 18162 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.468906 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.366252 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.469858 14242 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:17.471196 17774 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:17.386632 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 58671.8us
I20260709 12:58:17.388089 18314 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.471506 18410 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.465783 13688 rpcz_store.cc:276] 0709 12:58:15.723089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.723368 (+   279us) service_pool.cc:224] Handling call
0709 12:58:15.723629 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:16.131872 (+408243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:16.131908 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:16.149418 (+ 17510us) log.cc:844] Serialized 26 byte log entry
0709 12:58:16.434926 (+285508us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:17.029145 (+594219us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b18004388e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf
0709 12:58:17.029292 (+   147us) log.cc:844] Serialized 14 byte log entry
0709 12:58:17.029521 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:17.144803 (+115282us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540015ead4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:17.144855 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:17.144902 (+    47us) raft_consensus.cc:1685] finished
0709 12:58:17.144914 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:17.145055 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6022,"prepare.queue_time_us":116500,"prepare.run_cpu_time_us":2046,"prepare.run_wall_time_us":12452,"raft.queue_time_us":273483,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":5217,"spinlock_wait_cycles":280195200,"thread_start_us":626352,"threads_started":3,"wal-append.queue_time_us":457041}
I20260709 12:58:17.403517 18181 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.472728 13506 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:17.472477 18314 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.474324 18410 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.474532 13688 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:58:17.475327 14237 raft_consensus.cc:2410] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:58:17.475507 18314 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.409999 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:17.476823 18314 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:58:17.477349 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 138866us
I20260709 12:58:17.478626 13692 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:17.479179 18104 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:17.412573 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.480072 18361 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.413268 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.416311 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.416352 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.482520 18361 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.425436 17997 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:17.433050 18330 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:17.443581 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 132750us
I20260709 12:58:17.403487 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:17.451908 14212 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:17.398787 14246 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 737) took 2325 ms (client timeout 1968 ms). Trace:
I20260709 12:58:17.443735 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1931) took 1720 ms. Trace:
I20260709 12:58:17.458484 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:17.459883 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.460099 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.468317 18368 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.365199 14245 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 829) took 1648 ms. Trace:
W20260709 12:58:17.476172 13657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:17.471213 18162 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.476069 13691 pending_rounds.cc:107] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:58:17.409853 18236 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.475808 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.494081 18254 consensus_queue.cc:1059] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:58:17.495487 18254 raft_consensus.cc:3055] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:58:17.505304 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53730.2us
I20260709 12:58:17.488668 14245 rpcz_store.cc:276] 0709 12:58:15.716126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.716340 (+   214us) service_pool.cc:224] Handling call
0709 12:58:15.716617 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:15.716835 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:58:15.747115 (+ 30280us) log.cc:844] Serialized 14 byte log entry
0709 12:58:16.941513 (+1194398us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b18002fbf00. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 12:58:17.160233 (+218720us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18002fbf00. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 12:58:17.160528 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 12:58:17.364046 (+203518us) log.cc:844] Serialized 14 byte log entry
0709 12:58:17.364421 (+   375us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:17.364502 (+    81us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:58:17.364646 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:17.364756 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:17.364949 (+   193us) inbound_call.cc:177] Queueing success response
0709 12:58:17.421167 (+ 56218us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b4800210b58. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
Metrics: {"mutex_wait_us":1398486,"prepare.queue_time_us":1474213,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":142,"raft.queue_time_us":34934,"raft.run_cpu_time_us":3763,"raft.run_wall_time_us":65181,"spinlock_wait_cycles":20791808,"thread_start_us":202339,"threads_started":1,"wal-append.queue_time_us":107395}
W20260709 12:58:17.476539 13693 rpcz_store.cc:269] 0709 12:58:15.714922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.715039 (+   117us) service_pool.cc:224] Handling call
0709 12:58:17.156277 (+1441238us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b540042d1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:17.156393 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16874880}
W20260709 12:58:17.487587 14246 rpcz_store.cc:269] 0709 12:58:15.073042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.074401 (+  1359us) service_pool.cc:224] Handling call
0709 12:58:17.398127 (+2323726us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54005f1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:17.398482 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4481200000}
I20260709 12:58:17.490496 13691 raft_consensus.cc:2889] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:17.507725 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.502019 13582 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:17.464459 17960 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.446753 18298 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.515848 18314 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:17.516450 18236 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:17.516819 18277 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:17.505498 17719 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.518399 13690 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:58:17.523671 18407 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.526897 18168 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:17.530045 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.529069 13582 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:17.506251 18254 raft_consensus.cc:740] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:17.513608 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.548631 13534 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:17.536738 18314 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.487663 13619 rpcz_store.cc:276] 0709 12:58:15.723269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.724105 (+   836us) service_pool.cc:224] Handling call
0709 12:58:15.724369 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:15.724655 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:15.724682 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:15.732952 (+  8270us) log.cc:844] Serialized 26 byte log entry
0709 12:58:16.100270 (+367318us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b18003da920. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:58:16.279053 (+178783us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:16.279102 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:16.279180 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:17.154233 (+875053us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:17.443237 (+289004us) raft_consensus.cc:1685] finished
0709 12:58:17.443286 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:17.443510 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27819,"raft.queue_time_us":3335,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":15060,"spinlock_wait_cycles":8332928,"thread_start_us":178529,"threads_started":1,"wal-append.queue_time_us":900523}
I20260709 12:58:17.512457 18361 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:17.569931 13685 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:17.578724 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.552392 18314 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:17.585565 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.578060 18330 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.588886 14239 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:17.597278 13536 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:17.575942 18236 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.605057 13540 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:17.607854 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.633208 13690 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:17.613983 18168 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:17.604441 18361 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.641839 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:17.643338 13537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1714) took 2055 ms (client timeout 1559 ms). Trace:
W20260709 12:58:17.643944 13537 rpcz_store.cc:269] 0709 12:58:15.555803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.555944 (+   141us) service_pool.cc:224] Handling call
0709 12:58:17.588471 (+2032527us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b5400003054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:17.611371 (+ 22900us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1516088064}
W20260709 12:58:17.644469 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52332.5us
I20260709 12:58:17.646762 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:17.648303 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.650310 18435 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:17.659440 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:17.660961 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.661652 14238 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:17.663447 18211 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:17.663880 18211 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:17.664336 18211 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:17.664554 18041 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.665262 18041 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.666699 13691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45312 (request call id 3301) took 1069 ms. Trace:
I20260709 12:58:17.667070 13691 rpcz_store.cc:276] 0709 12:58:16.597306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:16.960666 (+363360us) service_pool.cc:224] Handling call
0709 12:58:16.961083 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:17.509492 (+548409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:17.666405 (+156913us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002eccd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:17.666636 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395581696}
I20260709 12:58:17.667567 18041 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.668329 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.683145 14240 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:17.669596 18382 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.684438 18382 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.687651 18382 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.689605 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:17.690481 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:17.695632 18377 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:17.696413 18377 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.698940 18377 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:17.700556 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 53961.5us
I20260709 12:58:17.701488 13687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1743) took 1978 ms. Trace:
I20260709 12:58:17.701902 13687 rpcz_store.cc:276] 0709 12:58:15.722546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:15.722997 (+   451us) service_pool.cc:224] Handling call
0709 12:58:17.333937 (+1610940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:17.464860 (+130923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:17.701263 (+236403us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540042d1d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:17.701405 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3703690112}
I20260709 12:58:17.702656 18314 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.705312 18241 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:17.706230 18235 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:17.706779 18235 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:17.669128 13691 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 5 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:17.734637 18361 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.639775 18168 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:17.735273 18168 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:17.737350 18330 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:17.741793 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:17.743774 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.744447 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.658886 18410 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.681433 17719 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.681890 17960 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.682024 13693 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:17.707883 18235 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.708478 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.681849 18104 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:17.711748 13540 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:17.704291 14242 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:17.723929 18355 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.721693 18314 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.726351 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.727631 14237 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:58:17.728510 14245 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:58:17.728734 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:17.729640 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:17.782406 14239 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:17.712618 18371 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.731817 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.677327 18236 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.708887 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:17.756656 17719 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.717718 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.757228 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.713675 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.758116 18410 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.758690 18168 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.715955 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:17.765904 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 236968us
I20260709 12:58:17.658594 18435 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:17.788182 13685 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:17.788645 13686 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:17.714442 18368 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.760982 18211 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:17.790022 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 58302.2us
W20260709 12:58:17.807381 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 105317us
I20260709 12:58:17.777194 18111 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:58:17.775321 18104 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:17.810642 13534 raft_consensus.cc:2368] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15.
W20260709 12:58:17.808106 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 90669.5us
I20260709 12:58:17.800518 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.813894 18041 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:17.794068 18236 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:17.803597 18435 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:17.796521 18355 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.828220 13619 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:17.829834 13508 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.831645 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.832626 13534 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:17.833496 18168 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.834872 18041 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.837236 18041 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.839277 18407 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:17.839834 13658 leader_election.cc:400] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15.
I20260709 12:58:17.840188 13658 leader_election.cc:403] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:58:17.829607 18368 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:58:17.842818 18370 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:17.843374 18370 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:17.843613 18181 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:17.831933 14246 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:17.844195 17774 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:17.844506 18370 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.846005 17774 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:17.849274 18410 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.852319 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:17.853132 17960 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:17.853381 13618 raft_consensus.cc:2368] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:58:17.853703 17960 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.860690 18314 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:58:17.842532 18407 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:17.785672 18277 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:58:17.865957 18368 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:17.868919 18368 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:17.819623 18236 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:17.877005 14238 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:17.808753 18330 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:58:17.885494 13508 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.875630 18236 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:17.827980 17997 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:58:17.873687 17960 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:17.888689 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:17.913105 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.918869 13619 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:17.939529 18410 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:17.945250 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 72052.7us
W20260709 12:58:17.950502 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 55769.8us
I20260709 12:58:17.954134 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.954536 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.954944 13615 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:17.955884 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.956060 18320 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:17.956694 18320 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:17.957083 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:17.958138 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.957803 18320 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.959301 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:17.957777 13612 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:58:17.943169 18254 consensus_queue.cc:260] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:17.964695 17719 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:17.973209 13688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40872 (request call id 716) took 1146 ms. Trace:
W20260709 12:58:17.965165 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 86665.2us
I20260709 12:58:17.980167 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:17.981076 13540 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:17.982476 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:17.982959 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:17.983620 13538 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:58:17.986726 18367 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:17.968806 18211 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.972380 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.977067 14239 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:17.978120 18355 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:17.989998 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:17.978535 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:17.979105 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:17.990032 14243 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:58:17.989531 13688 rpcz_store.cc:276] 0709 12:58:16.826891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:17.475581 (+648690us) service_pool.cc:224] Handling call
0709 12:58:17.490698 (+ 15117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:17.576418 (+ 85720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:17.576458 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:17.582144 (+  5686us) log.cc:844] Serialized 26 byte log entry
0709 12:58:17.805031 (+222887us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:58:17.805086 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:17.805172 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:17.972982 (+167810us) raft_consensus.cc:1685] finished
0709 12:58:17.973002 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:17.973149 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23726,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":1735,"thread_start_us":209584,"threads_started":1,"wal-append.queue_time_us":363295}
I20260709 12:58:17.998154 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:17.999876 13538 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:58:18.001884 13613 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:18.002880 18407 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:18.003309 18407 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:18.003647 18407 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:18.003742 18435 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.004976 13611 raft_consensus.cc:1240] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:17.995553 17719 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:18.009215 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:18.022400 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.023150 13611 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:18.033285 13619 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:58:17.982519 18445 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:58:18.041429 13612 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:18.011041 18330 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:58:18.051615 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 226281us
I20260709 12:58:18.021903 18277 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:18.043350 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.017640 17719 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:18.086261 18266 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:58:18.086706 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 66845.9us
I20260709 12:58:18.086800 18266 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:18.088157 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:18.088943 18382 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:18.090524 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:18.091302 18382 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:18.092171 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:18.094031 18295 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:18.095081 18295 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:18.095460 18295 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:18.100256 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.101280 18257 consensus_queue.cc:1059] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:18.102377 18257 raft_consensus.cc:3055] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:18.102707 18257 raft_consensus.cc:740] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:18.103639 18257 consensus_queue.cc:260] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.105480 18257 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:18.116277 18435 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:18.117972 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.118498 18181 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15.
I20260709 12:58:18.119470 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.120221 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.121160 13616 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:58:18.119525 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.121541 13616 pending_rounds.cc:85] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 8
I20260709 12:58:18.121809 13616 pending_rounds.cc:107] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:58:18.122030 13616 raft_consensus.cc:2889] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:18.088234 18266 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.136325 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.137687 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.137796 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.140019 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:18.140146 18277 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:18.141233 18277 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.142956 13687 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 12:58:18.143640 18447 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:58:18.045166 18367 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:58:18.126871 17719 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:18.149433 18445 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:18.149629 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.160248 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.152726 13691 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:58:18.091833 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:18.156656 18298 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:18.157056 18254 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:18.173250 18298 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.160291 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:18.111510 18407 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.171706 13685 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:18.153369 18153 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:18.173755 13541 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:58:18.179049 13611 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:18.170948 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.165740 13688 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:18.180534 14239 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:58:18.181627 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.182603 18298 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.185940 18367 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:18.186375 18367 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:18.188267 18153 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.190673 18153 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.180589 13540 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 12: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:18.193790 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.194669 13509 leader_election.cc:400] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11.
I20260709 12:58:18.195214 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.195484 13618 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:58:18.198391 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.200131 13612 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:18.180629 18445 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.147609 18104 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.205816 13616 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:18.169819 18236 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.218060 13538 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:18.218914 18266 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:18.220234 13689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1755) took 1392 ms. Trace:
I20260709 12:58:18.221237 18104 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:18.220654 13689 rpcz_store.cc:276] 0709 12:58:16.827718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:17.480724 (+653006us) service_pool.cc:224] Handling call
0709 12:58:17.759415 (+278691us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b5800160298. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 12:58:17.759719 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:17.760397 (+   678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:17.760436 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:17.761022 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:58:17.994256 (+233234us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:58:18.080615 (+ 86359us) log.cc:844] Serialized 14 byte log entry
0709 12:58:18.080863 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:18.080960 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:18.142869 (+ 61909us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:18.219845 (+ 76976us) raft_consensus.cc:1685] finished
0709 12:58:18.219870 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:18.220044 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86177,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":2510,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":58094,"spinlock_wait_cycles":98645120,"thread_start_us":232650,"threads_started":1,"wal-append.queue_time_us":324902}
I20260709 12:58:18.216830 17719 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.225816 13691 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:18.227229 17719 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:18.227985 18410 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:18.211616 13506 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:18.202783 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:18.241760 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 288307us
W20260709 12:58:18.242827 13582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:18.219143 18236 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:18.266770 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 175022us
I20260709 12:58:18.267590 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.240355 18298 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:18.243777 18153 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:18.274840 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:18.275897 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.273365 13541 raft_consensus.cc:1240] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:58:18.266010 17997 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:18.300802 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:18.325186 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:18.325343 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 50634.3us
W20260709 12:58:18.328003 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 48854.9us
I20260709 12:58:18.348762 13691 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:18.274418 18377 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
W20260709 12:58:18.353755 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 155103us
I20260709 12:58:18.329223 18298 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:18.278344 17960 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:58:18.329506 18320 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:58:18.336745 18314 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:58:18.355734 18445 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.356732 18298 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:18.338510 17997 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.375792 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:18.377374 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:18.377979 18298 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:18.391954 18435 raft_consensus.cc:2764] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:58:18.402114 13685 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:18.403821 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.405246 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.417117 18295 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.418573 18295 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:18.420760 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51400.3us
I20260709 12:58:18.422964 13584 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:18.424019 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:18.429903 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 68288.7us
I20260709 12:58:18.384831 18382 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:18.431638 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.432302 18382 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.437582 18382 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:18.375612 18454 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:58:18.432848 18463 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:18.439544 17997 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:18.434574 13687 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:18.444089 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:18.449286 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.384111 18407 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:18.450695 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:18.452123 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.459357 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.465087 13693 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:18.465287 18314 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:18.466462 14211 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:18.465108 13509 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.469120 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:18.435128 18448 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
I20260709 12:58:18.482456 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.490370 13584 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:18.493626 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.380146 18111 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:58:18.497315 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:18.497946 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.498898 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:18.499552 13540 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:18.508239 18440 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:58:18.509256 18440 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:18.509619 18440 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:18.510507 18440 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.512178 18320 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:18.512107 18236 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:18.512178 13540 raft_consensus.cc:2468] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:18.512684 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:18.515028 14245 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:18.512151 18439 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:18.501058 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.524411 13534 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:18.511783 18440 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:18.480657 18241 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:18.536091 13538 raft_consensus.cc:2393] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:18.537719 13537 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 15: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:18.545249 14211 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.549816 18104 raft_consensus.cc:2804] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:18.550344 18104 raft_consensus.cc:697] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
W20260709 12:58:18.552696 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 62431.6us
I20260709 12:58:18.553839 13584 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:18.551242 18104 consensus_queue.cc:237] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.558923 13689 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:18.541273 18241 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:18.555665 13584 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:18.559851 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.564232 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:18.574947 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 171530us
W20260709 12:58:18.576742 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 111703us
I20260709 12:58:18.577766 13509 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.579273 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.564396 18439 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:18.580258 18266 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:18.582605 18266 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:18.582996 18266 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:18.587036 14245 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:18.561764 18241 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:18.593230 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:18.580964 18041 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:18.605401 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:18.605868 18445 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:58:18.606102 18295 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:58:18.609710 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 210132us
W20260709 12:58:18.617542 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 61996.2us
I20260709 12:58:18.621443 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:18.621296 18211 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:18.622376 18435 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:18.622781 18435 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:18.601297 18371 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:18.580610 18235 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 12:58:18.632824 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 195567us
I20260709 12:58:18.618094 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.623170 18435 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:18.627045 18407 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:18.618696 18295 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:18.644660 18295 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:18.648931 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:18.649308 18461 log.cc:927] Time spent T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:58:18.657512 18298 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.662452 18295 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.638315 18371 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:58:18.659687 13438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41440 (request call id 209) took 21149 ms (client timeout 15000 ms). Trace:
I20260709 12:58:18.671702 17774 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:18.672528 18472 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:58:18.678574 18465 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:18.679283 18465 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.681514 18465 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.683409 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.699965 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.695941 18277 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:18.710125 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.711020 13612 raft_consensus.cc:2368] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:58:18.696995 18298 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.768898 13684 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:58:18.769394 13659 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:18.697655 18465 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:58:18.786281 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.788318 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:18.791563 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 179304us
I20260709 12:58:18.800335 14209 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.801142 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:18.810442 18367 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.811700 18367 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:18.812258 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.813033 13618 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 12: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:18.813882 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.814617 13614 raft_consensus.cc:2393] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:18.795130 17960 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:18.815447 17960 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.816311 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.817528 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.817934 17960 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.789844 18277 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:18.819486 18277 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.820673 13617 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:18.821852 18277 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.824657 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.825912 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.826514 13610 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:18.826877 17997 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:18.827486 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.832350 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.832540 13440 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:18.811627 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.810925 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.801870 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:18.833089 13617 raft_consensus.cc:2393] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 9: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:18.852659 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:18.852929 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.853822 13612 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:18.859862 14238 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:58:18.868690 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:18.856658 18168 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:18.798802 18266 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.873320 13440 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 3 to 11. New cstate: current_term: 11 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:18.874949 18266 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:18.876089 18111 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:18.667385 13438 rpcz_store.cc:269] 0709 12:57:57.648825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:57.691480 (+ 42655us) service_pool.cc:224] Handling call
0709 12:58:18.659426 (+20967946us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:11.971726 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:11.972765 (+  1039us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:11.972842 (+    77us) write_op.cc:435] Acquired schema lock
0709 12:58:11.972875 (+    33us) tablet.cc:662] Decoding operations
0709 12:58:11.975315 (+  2440us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:11.975470 (+   155us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:11.975516 (+    46us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:58:12.144608 (+169092us) tablet.cc:701] Row locks acquired
0709 12:58:12.166470 (+ 21862us) write_op.cc:260] PREPARE: finished
0709 12:58:12.199945 (+ 33475us) write_op.cc:270] Start()
0709 12:58:12.225746 (+ 25801us) write_op.cc:276] Timestamp: P: 1783601892193346 usec, L: 0
0709 12:58:12.260809 (+ 35063us) op_driver.cc:348] REPLICATION: starting
0709 12:58:12.347641 (+ 86832us) log.cc:844] Serialized 7816 byte log entry
0709 12:58:13.254947 (+907306us) op_driver.cc:464] REPLICATION: finished
0709 12:58:13.543958 (+289011us) write_op.cc:301] APPLY: starting
0709 12:58:13.554404 (+ 10446us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:13.938385 (+383981us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:13.938504 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:14.489882 (+551378us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:14.533564 (+ 43682us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 12:58:14.545089 (+ 11525us) write_op.cc:312] APPLY: finished
0709 12:58:14.595729 (+ 50640us) log.cc:844] Serialized 180 byte log entry
0709 12:58:14.880585 (+284856us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:14.913036 (+ 32451us) write_op.cc:454] Released schema lock
0709 12:58:14.919613 (+  6577us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:15.036270 (+116657us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:15.036319 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4769285,"reported_tablets":35,"spinlock_wait_cycles":9088,"child_traces":[["op",{"apply.queue_time_us":223033,"apply.run_cpu_time_us":159839,"apply.run_wall_time_us":1502185,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87647,"num_ops":20,"prepare.queue_time_us":177746,"prepare.run_cpu_time_us":39628,"prepare.run_wall_time_us":642505,"raft.queue_time_us":577124,"raft.run_cpu_time_us":13393,"raft.run_wall_time_us":166839,"replication_time_us":993942,"thread_start_us":1103417,"threads_started":5,"wal-append.queue_time_us":606585,"wal-append.run_cpu_time_us":31841,"wal-append.run_wall_time_us":2524767}]]}
I20260709 12:58:18.876650 18111 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.877619 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.827451 17997 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:18.878973 18111 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:18.879541 18111 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:18.880474 17997 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:18.880971 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45285.3us
I20260709 12:58:18.884251 13692 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:58:18.875571 13440 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:18.887288 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:18.888397 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:18.888562 13541 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:18.892515 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:18.893215 14245 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:18.897239 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:58:18.897886 18474 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:58:18.898291 13537 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:18.802892 18371 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.915318 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:18.917002 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 124690us
I20260709 12:58:18.868022 18370 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:18.923273 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.930034 18295 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:18.934449 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:18.942508 14246 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:58:18.949314 14239 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:18.950845 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.952221 13616 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:18.952266 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.942926 14246 pending_rounds.cc:85] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 9
I20260709 12:58:18.955690 14246 pending_rounds.cc:107] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:58:18.955866 14246 raft_consensus.cc:2889] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:18.960940 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.961593 13617 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:18.921504 13440 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 7 to 9, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 9 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:18.945964 18446 log.cc:927] Time spent T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:58:18.933221 18370 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:18.896328 18111 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.971284 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.969805 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.976912 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:18.985287 13618 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:58:18.978062 14239 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:58:18.987673 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70048.7us
I20260709 12:58:18.992137 13440 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 2 to 10, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:19.026495 14245 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:19.039229 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:18.976606 18370 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:19.062757 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:19.062412 18412 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:19.074790 18477 log.cc:927] Time spent T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:58:19.075425 18041 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:19.074404 18412 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:18.976756 18235 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:58:19.080897 14211 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:19.020995 18153 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 12:58:19.091478 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 54645us
I20260709 12:58:19.020804 18320 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:58:19.096819 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:19.097589 13614 raft_consensus.cc:2368] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:58:19.077826 18412 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:19.095566 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:19.098536 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:19.098879 18254 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:19.098712 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.099327 18254 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:19.099696 18254 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:19.077775 18235 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:58:19.100162 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.063334 18241 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:19.106797 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:19.112602 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 89701.8us
I20260709 12:58:19.112884 13617 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:19.117290 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.075908 18041 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:19.120253 13537 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:58:19.121847 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:19.123730 18480 log.cc:927] Time spent T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260709 12:58:19.124959 18459 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:58:19.130733 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.124859 18241 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:19.136952 13617 raft_consensus.cc:3055] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:19.138548 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.085953 18314 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:58:19.092136 17774 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:58:19.141033 17774 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:19.148511 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.151093 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.137351 13617 raft_consensus.cc:740] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:19.154569 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 66981us
I20260709 12:58:19.116756 18235 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:58:19.157671 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 106738us
I20260709 12:58:19.158949 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:19.121019 18277 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:19.121296 18320 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:19.162695 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.170949 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.174105 13618 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:19.179177 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:19.176254 18277 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.181423 18277 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:19.182497 18304 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:19.196621 18111 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:19.197067 14241 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:19.197832 13440 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 5 to 8, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 8 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:19.199208 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:19.199973 13440 catalog_manager.cc:5697] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 8 to 9, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 9 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:19.200567 18153 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:19.200819 18298 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:19.201426 18298 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:19.202416 18298 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:19.203275 13440 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 6 to 7, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 7 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:19.206279 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.188436 18235 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:19.188642 13617 consensus_queue.cc:260] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.200523 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.209127 13539 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:19.210229 18304 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:19.210448 18277 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:19.211213 13617 raft_consensus.cc:3060] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:19.211066 18277 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.211661 14239 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:58:19.213295 18277 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:19.210436 13440 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 1 to 8. New cstate: current_term: 8 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:19.216426 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:19.228361 13617 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:19.228796 14238 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:19.218719 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.229900 14246 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:19.234992 18143 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:19.236564 18304 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:58:19.240782 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:19.242197 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:19.244602 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:58:19.256323 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:19.256924 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:19.258257 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:19.303164 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:19.290104 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:19.259027 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 128957us
I20260709 12:58:19.306727 13685 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:19.282776 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.261442 13541 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
W20260709 12:58:19.279672 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61035.5us
W20260709 12:58:19.275130 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:19.308333 13659 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.259069 14246 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12.
I20260709 12:58:19.272434 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:19.321388 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 142543us
I20260709 12:58:19.321844 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:19.322414 13685 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:19.322968 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:19.326000 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:19.326643 14245 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:19.326671 13541 raft_consensus.cc:1240] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:58:19.326155 13534 raft_consensus.cc:2393] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:19.323540 13615 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:19.329104 13536 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:19.329726 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:19.329828 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 124788us
I20260709 12:58:19.343076 13543 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:19.338174 18104 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:58:19.347345 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.273979 18304 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:19.348781 13538 raft_consensus.cc:1275] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:19.348914 13614 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:19.259768 18412 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:19.349539 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.354363 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:19.322150 18367 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:19.354792 13614 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:19.364912 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.332705 18439 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:19.329602 18266 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:19.389637 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 117308us
I20260709 12:58:19.338443 18453 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:19.391081 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:19.369763 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:19.364708 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:19.395349 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 65701.6us
I20260709 12:58:19.398540 14241 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:19.353688 18382 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:19.409662 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:19.446671 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 138444us
I20260709 12:58:19.365199 18410 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:19.445385 13685 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
W20260709 12:58:19.429024 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 78559.3us
I20260709 12:58:19.349968 18465 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:58:19.391091 17997 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:19.282022 18041 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.457489 13659 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:19.459129 18435 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.783s	user 0.017s	sys 0.005s
I20260709 12:58:19.451884 13614 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:19.469120 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 59665.7us
I20260709 12:58:19.476456 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:19.520809 13614 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:58:19.524514 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 48139.7us
W20260709 12:58:19.543677 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 81080.2us
I20260709 12:58:19.369383 18304 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.408735 18439 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.439985 18495 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.125s
I20260709 12:58:19.472421 17997 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:19.448345 18382 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.553068 18512 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:58:19.556113 18492 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:19.634989 13659 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:19.703619 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 275527us
W20260709 12:58:19.718042 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 325704us
I20260709 12:58:19.736536 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:19.637012 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 69371us
W20260709 12:58:19.637401 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:19.523880 18407 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:19.552229 18111 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:58:19.644896 18458 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.002s
I20260709 12:58:19.645040 18490 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:58:19.764964 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 205233us
W20260709 12:58:19.645990 18497 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:58:19.544139 18041 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:19.480005 18435 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:19.543694 18266 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:19.652386 13536 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 15: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:19.409811 18453 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.474323 18410 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.552037 18168 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:19.745216 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:19.791719 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 46598.7us
I20260709 12:58:19.755115 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:19.780939 13584 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:19.749240 18111 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:19.811160 18382 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:19.811740 18111 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:19.813028 18439 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:19.813736 18487 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:19.814293 18487 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.818415 18304 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:19.828248 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:19.828400 18512 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:19.828935 18512 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:19.830418 18410 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:19.830935 18512 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:19.832243 18489 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:19.832826 18489 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:19.773401 18370 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.000s
I20260709 12:58:19.791240 18168 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:19.797303 18235 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:19.797859 18453 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:19.799379 18407 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:19.800418 18181 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:58:19.801208 18496 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
I20260709 12:58:19.781674 17997 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:19.809098 14241 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:58:19.809378 18412 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:19.799456 18493 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.270s	user 0.000s	sys 0.000s
W20260709 12:58:19.928248 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 147402us
I20260709 12:58:19.975965 13614 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:58:20.126881 13614 pending_rounds.cc:85] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 10
I20260709 12:58:19.928311 18489 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:19.776851 18507 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:19.943861 18439 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:20.127082 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:20.078647 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 207697us
I20260709 12:58:19.943742 18168 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:20.127171 13614 pending_rounds.cc:107] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:58:20.157853 13614 raft_consensus.cc:2889] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:58:20.121021 13610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.000s	sys 0.030s
W20260709 12:58:20.158653 13610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.001s	sys 0.031s
I20260709 12:58:20.067107 18382 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.095300 18435 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:20.163599 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 611803us
I20260709 12:58:20.173096 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.177893 13584 leader_election.cc:400] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12.
I20260709 12:58:19.943974 18370 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:20.159772 18371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.009s	sys 0.012s
I20260709 12:58:20.178215 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:20.157281 18254 consensus_meta.cc:287] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 1.057s	user 0.012s	sys 0.000s
I20260709 12:58:20.094939 18407 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.133010 18235 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:20.178315 13584 leader_election.cc:403] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:20.173486 18487 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:20.189182 18371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.009s	sys 0.012s
I20260709 12:58:20.172722 18512 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:20.192521 14244 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:20.213748 13541 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:20.210134 18407 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:20.212808 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 45216.6us
I20260709 12:58:20.216452 13584 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:20.219277 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:20.197574 18439 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:20.224274 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:20.226254 13659 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.232941 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59162 (request call id 847) took 1063 ms. Trace:
I20260709 12:58:20.198545 18518 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:20.178539 18382 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.241823 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.156514 18507 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.234488 13613 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:20.251392 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:20.250190 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:20.214787 18435 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:20.220587 18211 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12.
I20260709 12:58:20.222188 18454 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:20.206408 18254 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.241423 14240 rpcz_store.cc:276] 0709 12:58:19.169350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.169529 (+   179us) service_pool.cc:224] Handling call
0709 12:58:19.170038 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:19.221545 (+ 51507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:19.221580 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:19.232211 (+ 10631us) log.cc:844] Serialized 26 byte log entry
0709 12:58:19.280234 (+ 48023us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:19.280285 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:19.280374 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:20.232749 (+952375us) raft_consensus.cc:1685] finished
0709 12:58:20.232770 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.232886 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39662,"raft.queue_time_us":41546,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":7005,"spinlock_wait_cycles":18304,"thread_start_us":47599,"threads_started":1,"wal-append.queue_time_us":134877}
I20260709 12:58:20.246658 18518 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:20.271409 13612 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:20.257341 18435 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.281121 18519 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:20.284058 18235 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.220413 18512 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.290724 18512 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:20.294315 18370 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.260974 18487 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:20.297734 18487 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:20.298910 18487 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.270692 18454 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:20.304263 18439 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.274734 18449 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.327613 14238 raft_consensus.cc:2368] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15.
W20260709 12:58:20.329687 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 103535us
I20260709 12:58:20.359426 13659 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.334496 18535 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:20.276852 18518 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:20.336344 18525 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:20.342653 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3135) took 1012 ms. Trace:
I20260709 12:58:20.344754 18254 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.344933 18507 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:20.347107 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:20.369131 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.351403 18382 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:20.331810 13618 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.004s	sys 0.002s
W20260709 12:58:20.375613 13618 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.005s	sys 0.002s
I20260709 12:58:20.364428 18511 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:20.362320 18266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.008s	sys 0.023s
I20260709 12:58:20.363885 13610 rpcz_store.cc:276] 0709 12:58:19.330398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.330514 (+   116us) service_pool.cc:224] Handling call
0709 12:58:19.330792 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:19.396291 (+ 65499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:19.396327 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:19.439675 (+ 43348us) log.cc:844] Serialized 26 byte log entry
0709 12:58:20.159130 (+719455us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:20.159188 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.159283 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:20.342418 (+183135us) raft_consensus.cc:1685] finished
0709 12:58:20.342442 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.342584 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37168,"raft.queue_time_us":34621,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":27811,"thread_start_us":692075,"threads_started":2,"wal-append.queue_time_us":866642}
I20260709 12:58:20.376611 18528 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:58:20.376925 18266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.008s	sys 0.023s
I20260709 12:58:20.376904 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.377308 18528 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:20.365926 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:20.328747 13582 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:41755 timed out after 1.744s (SENT) [suppressed 11 similar messages]
I20260709 12:58:20.377971 18528 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:20.377956 18266 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:20.380015 13618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3132) took 1136 ms. Trace:
I20260709 12:58:20.381773 18507 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.382534 18507 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.385241 18507 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.381034 13618 rpcz_store.cc:276] 0709 12:58:19.243934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.244754 (+   820us) service_pool.cc:224] Handling call
0709 12:58:19.245134 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:19.269493 (+ 24359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:19.269531 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:19.286497 (+ 16966us) log.cc:844] Serialized 26 byte log entry
0709 12:58:19.728310 (+441813us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:58:20.376099 (+647789us) log.cc:844] Serialized 14 byte log entry
0709 12:58:20.379570 (+  3471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.379667 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:20.379723 (+    56us) raft_consensus.cc:1685] finished
0709 12:58:20.379740 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.379946 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33357,"prepare.queue_time_us":563879,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":80894,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":23614,"thread_start_us":997218,"threads_started":2,"wal-append.queue_time_us":1011874}
W20260709 12:58:20.398272 18529 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.003s
I20260709 12:58:20.387167 18528 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:20.425601 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 48795.1us
I20260709 12:58:20.426695 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:20.431072 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 256781us
W20260709 12:58:20.431810 14210 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.017s	sys 0.018s
W20260709 12:58:20.432196 14210 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.017s	sys 0.020s
I20260709 12:58:20.439339 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.410943 18453 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.368369 18449 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:20.448479 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 309494us
I20260709 12:58:20.454388 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.455469 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.458288 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.445567 18528 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:20.472481 14240 raft_consensus.cc:2410] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:58:20.443347 18511 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:20.476536 18041 consensus_meta.cc:287] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.002s
I20260709 12:58:20.477192 18168 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:20.478228 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 46726.1us
I20260709 12:58:20.474283 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1737) took 1345 ms. Trace:
I20260709 12:58:20.454038 18295 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:20.500294 13691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.008s	sys 0.031s
I20260709 12:58:20.455355 18266 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:20.501477 13539 rpcz_store.cc:276] 0709 12:58:19.128907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.129027 (+   120us) service_pool.cc:224] Handling call
0709 12:58:19.129381 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:19.219352 (+ 89971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.473959 (+1254607us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b54003b7c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:20.474210 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177489280}
I20260709 12:58:20.506136 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:20.502131 13691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.008s	sys 0.031s
I20260709 12:58:20.480427 18535 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:20.512789 14244 raft_consensus.cc:2410] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:58:20.451856 18453 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.513193 13542 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:20.513829 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:20.480322 18463 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.516721 13537 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:20.516978 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.516361 18463 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.519960 18449 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.510246 18295 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.522404 18295 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.522918 18542 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.523442 18542 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.524286 13659 leader_election.cc:400] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 11. Current term is 12.
I20260709 12:58:20.525509 18542 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.525796 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.507803 18531 consensus_queue.cc:1048] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:58:20.528471 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.531066 13617 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
W20260709 12:58:20.533042 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 204318us
I20260709 12:58:20.453374 18454 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.532871 18266 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:20.559620 14246 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:20.563405 13619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3134) took 1318 ms. Trace:
I20260709 12:58:20.564358 13619 rpcz_store.cc:276] 0709 12:58:19.244587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.245517 (+   930us) service_pool.cc:224] Handling call
0709 12:58:19.245805 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:19.260557 (+ 14752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:19.260602 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:19.261132 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 12:58:20.259403 (+998271us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b1800497c80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:58:20.444708 (+185305us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:58:20.479833 (+ 35125us) log.cc:844] Serialized 14 byte log entry
0709 12:58:20.480158 (+   325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.480234 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:20.560372 (+ 80138us) raft_consensus.cc:1685] finished
0709 12:58:20.563112 (+  2740us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.563309 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11377,"prepare.queue_time_us":36151,"raft.queue_time_us":28676,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":25789,"spinlock_wait_cycles":54646784,"thread_start_us":180850,"threads_started":1,"wal-append.queue_time_us":1247648}
I20260709 12:58:20.567983 13535 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:20.573035 18168 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:20.584728 18548 log.cc:927] Time spent T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:58:20.586958 18518 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.589772 18518 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.593262 13615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 961) took 1242 ms. Trace:
I20260709 12:58:20.593839 13615 rpcz_store.cc:276] 0709 12:58:19.350997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.360960 (+  9963us) service_pool.cc:224] Handling call
0709 12:58:19.361185 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:19.681344 (+320159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:19.681382 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:58:19.683300 (+  1918us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:19.683342 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.592972 (+909630us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b540026e154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:20.593023 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.593182 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735830784}
I20260709 12:58:20.561574 18266 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.598204 18266 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:20.599601 14238 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59176 (request call id 758) took 1322 ms. Trace:
I20260709 12:58:20.600030 14238 rpcz_store.cc:276] 0709 12:58:19.277251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.277363 (+   112us) service_pool.cc:224] Handling call
0709 12:58:20.599357 (+1321994us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400497ad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:20.599509 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332287872}
W20260709 12:58:20.607419 18513 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.862s	user 0.008s	sys 0.008s
I20260709 12:58:20.608800 13616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56786 (request call id 3136) took 1278 ms. Trace:
I20260709 12:58:20.609192 13616 rpcz_store.cc:276] 0709 12:58:19.330717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.330838 (+   121us) service_pool.cc:224] Handling call
0709 12:58:19.331103 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:19.455227 (+124124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:19.455307 (+    80us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:19.456359 (+  1052us) log.cc:844] Serialized 26 byte log entry
0709 12:58:19.460918 (+  4559us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:58:19.677227 (+216309us) log.cc:844] Serialized 14 byte log entry
0709 12:58:19.699122 (+ 21895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:19.699227 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:20.606621 (+907394us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:20.608480 (+  1859us) raft_consensus.cc:1685] finished
0709 12:58:20.608597 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.608740 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36223,"prepare.queue_time_us":831161,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":30,"raft.queue_time_us":172080,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":29392,"spinlock_wait_cycles":67019008,"thread_start_us":301346,"threads_started":3,"wal-append.queue_time_us":181687}
W20260709 12:58:20.611657 13693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45328 (request call id 1777) took 2016 ms (client timeout 1744 ms). Trace:
W20260709 12:58:20.612433 13693 rpcz_store.cc:269] 0709 12:58:18.595088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:18.648616 (+ 53528us) service_pool.cc:224] Handling call
0709 12:58:20.611268 (+1962652us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400159d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:20.611521 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3613645696}
I20260709 12:58:20.613148 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1284) took 1451 ms. Trace:
I20260709 12:58:20.616684 14242 rpcz_store.cc:276] 0709 12:58:19.161486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.161621 (+   135us) service_pool.cc:224] Handling call
0709 12:58:19.161899 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:19.196640 (+ 34741us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:19.196669 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:20.366325 (+1169656us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b2c00047778. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:58:20.368436 (+  2111us) log.cc:844] Serialized 26 byte log entry
0709 12:58:20.535067 (+166631us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:20.535120 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.535215 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:20.612670 (+ 77455us) raft_consensus.cc:1685] finished
0709 12:58:20.612705 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.613043 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21707,"raft.queue_time_us":34792,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":49409792,"thread_start_us":141687,"threads_started":1,"wal-append.queue_time_us":218510}
I20260709 12:58:20.609727 18525 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:20.623842 18518 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.633273 18518 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:20.638151 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:20.643909 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:20.644732 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 142564us
I20260709 12:58:20.648393 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:20.651103 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:20.653510 13542 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:20.656976 13616 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:20.658192 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:20.658999 13610 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:20.637490 18519 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:20.660607 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:20.680629 13618 raft_consensus.cc:2393] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 16: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:20.682154 13659 leader_election.cc:336] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.538s (SENT)
I20260709 12:58:20.682612 13659 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.643875 17997 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.660607 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.680277 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.652545 17960 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.658124 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.656517 18505 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:20.693157 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.690724 18412 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:58:20.694622 18553 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:58:20.696113 18520 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:20.699323 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:20.700178 18491 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.700767 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.700773 18491 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.702713 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.703706 18491 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.705305 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.708590 18435 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.692873 18518 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:20.718286 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 60739.9us
W20260709 12:58:20.699398 18551 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:58:20.723744 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.692570 18554 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:20.726137 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.689797 17997 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.734822 18236 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:20.736799 13542 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:20.741175 14211 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:20.742223 18491 raft_consensus.cc:2804] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:20.743005 18491 raft_consensus.cc:697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:20.745289 18554 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.753764 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.734849 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:20.724444 18505 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:20.770793 17997 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.775949 18111 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:20.685590 18519 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.778890 18519 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.710238 17960 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.771893 18454 consensus_queue.cc:1059] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:58:20.781232 17960 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.762146 18518 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:20.775597 18266 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.782601 13687 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:20.785290 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.785301 18266 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:20.787362 18266 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:20.787896 18266 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.792946 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.793532 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:20.793735 18511 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:20.794749 18511 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:20.795661 18511 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.798395 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:20.801005 18503 log.cc:927] Time spent T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.124s	user 0.005s	sys 0.010s
I20260709 12:58:20.801388 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.793519 18266 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:20.803674 18492 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:20.803694 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.804188 18492 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:20.804646 18492 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:20.803951 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1869) took 1457 ms. Trace:
I20260709 12:58:20.805248 13541 rpcz_store.cc:276] 0709 12:58:19.346297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.349018 (+  2721us) service_pool.cc:224] Handling call
0709 12:58:19.349264 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:20.214202 (+864938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.803726 (+589524us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54003b8154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:20.803875 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1987729280}
I20260709 12:58:20.808061 18236 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:20.808557 18236 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.810781 18236 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.810876 18440 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:20.811352 18491 consensus_queue.cc:237] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.811949 18440 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.812469 18440 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:20.814302 13659 leader_election.cc:336] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.708s (SENT)
I20260709 12:58:20.814805 18435 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:20.815018 13659 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:20.803587 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 68845.2us
I20260709 12:58:20.818238 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.819149 13541 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:20.820178 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.821090 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.821874 18454 raft_consensus.cc:3055] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:20.822311 18454 raft_consensus.cc:740] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:20.823146 18454 consensus_queue.cc:260] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.824506 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.824968 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.815236 18435 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:20.826056 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 64441.5us
I20260709 12:58:20.826640 18435 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.827945 18435 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:20.828689 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 43495.1us
I20260709 12:58:20.829443 13584 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.830590 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:20.831601 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:20.831854 18518 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:20.835028 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:20.835297 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:20.814248 18492 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.836586 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.838904 13542 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:20.839480 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:20.832597 18518 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.844580 13537 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 15: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:20.845089 14240 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:20.772116 18554 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.847978 13658 leader_election.cc:400] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15.
I20260709 12:58:20.848353 13658 leader_election.cc:403] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:20.860455 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.861795 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:20.862499 13611 raft_consensus.cc:2368] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 15. Current term is 16.
I20260709 12:58:20.837927 18181 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:20.814539 18440 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.864850 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:20.867022 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.875301 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:20.876269 13618 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:20.877465 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.877902 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:20.846184 18507 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:20.880754 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.883069 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.886379 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.881222 13619 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:20.887943 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.871654 18465 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.890669 18465 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.894207 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45566 (request call id 970) took 1548 ms. Trace:
I20260709 12:58:20.894773 13536 rpcz_store.cc:276] 0709 12:58:19.345883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.347080 (+  1197us) service_pool.cc:224] Handling call
0709 12:58:20.893963 (+1546883us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b540028bc54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:20.894128 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699353472}
I20260709 12:58:20.895843 18181 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.896507 18181 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.897400 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 972) took 1531 ms. Trace:
I20260709 12:58:20.898885 18181 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.849141 18525 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.897912 13538 rpcz_store.cc:276] 0709 12:58:19.366188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.366432 (+   244us) service_pool.cc:224] Handling call
0709 12:58:20.476014 (+1109582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:20.479653 (+  3639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:20.479683 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:20.504275 (+ 24592us) log.cc:844] Serialized 26 byte log entry
0709 12:58:20.644558 (+140283us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:58:20.644596 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:20.644655 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:20.703400 (+ 58745us) raft_consensus.cc:1685] finished
0709 12:58:20.703419 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:20.897052 (+193633us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003b7c50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:20.897317 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3243,"raft.queue_time_us":4778,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3770,"spinlock_wait_cycles":2551032192,"thread_start_us":125565,"threads_started":1,"wal-append.queue_time_us":141982}
I20260709 12:58:20.855439 18314 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:20.871984 18143 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:20.908188 18314 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.908677 18143 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.910422 18314 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.910610 18143 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:20.861562 17997 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:20.911402 17997 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.912508 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.913888 14212 leader_election.cc:400] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 15. Current term is 16.
I20260709 12:58:20.914004 13689 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:20.914233 14212 leader_election.cc:403] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:58:20.915867 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:20.916150 18320 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:20.917202 18519 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:20.917550 18519 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:20.917842 17719 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:58:20.917976 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:20.918412 18440 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:20.918799 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:20.919638 13659 leader_election.cc:336] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Timed out: RequestConsensusVote RPC to 127.12.30.66:39367 timed out after 1.759s (SENT)
I20260709 12:58:20.870491 18111 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:20.880450 13508 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.907534 13610 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:20.922639 13688 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:58:20.923070 13688 pending_rounds.cc:85] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:58:20.923295 13688 pending_rounds.cc:107] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:58:20.923522 13688 raft_consensus.cc:2889] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:20.925468 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:20.926061 13537 raft_consensus.cc:3060] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:20.847333 18528 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:20.927524 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:20.862788 18266 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:20.928485 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.929111 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:20.929641 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.930884 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.929965 13541 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:20.942894 13536 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 15: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:20.943573 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.944517 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:20.932216 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 51847.8us
I20260709 12:58:20.929057 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.861951 18501 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:20.863476 18277 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:20.953583 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.873682 18412 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:58:20.945174 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:20.869911 18512 raft_consensus.cc:3060] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:58:20.946398 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.947357 18111 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:20.949150 18492 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.879581 18544 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:20.856907 18518 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.960532 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:20.971675 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.022703 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:21.025327 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 52511.6us
I20260709 12:58:20.936555 18536 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:20.899770 18525 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:21.035250 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 115619us
I20260709 12:58:20.961167 18181 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.039832 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:20.951017 18528 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.976670 18544 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.031735 18536 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:21.032871 18527 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:58:20.954200 18266 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:20.918406 18519 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.100693 18519 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:21.034194 17774 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:20.957858 18277 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:20.960860 18111 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.037863 18304 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:21.076089 18494 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.080034 18298 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:21.080127 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.082451 18528 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:21.084152 18561 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:21.089049 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.051257 18314 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.094457 18463 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.095394 18525 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:20.975404 18501 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.109777 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.111024 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.111742 13611 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 12:58:21.111503 13617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56786 (request call id 3127) took 1877 ms (client timeout 1673 ms). Trace:
I20260709 12:58:21.126591 13537 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
W20260709 12:58:21.132786 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:21.133461 13619 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:21.181049 13619 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:21.183759 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.185216 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.185913 13616 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:21.186867 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.141840 18254 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:21.158501 13617 rpcz_store.cc:269] 0709 12:58:19.234542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.256682 (+ 22140us) service_pool.cc:224] Handling call
0709 12:58:21.111245 (+1854563us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002fc6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:21.111383 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2876367744}
I20260709 12:58:21.138346 17960 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:21.188598 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.182981 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.219530 13534 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:21.133034 17774 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.226649 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.219412 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:21.226696 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.180609 18490 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:21.189844 18463 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.146014 18298 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.144315 18494 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.195782 18528 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.192469 18143 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:21.235504 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 150243us
I20260709 12:58:21.237850 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.155699 18314 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.239289 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.192324 18412 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:21.218839 18382 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:21.234414 13619 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.195868 18525 consensus_queue.cc:1059] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:58:21.268029 13689 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:58:21.153961 18561 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.234216 18507 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:21.225275 17960 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.221448 18254 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.276409 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:21.279660 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 54033.2us
I20260709 12:58:21.284253 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.289183 18277 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:21.289300 18565 log.cc:927] Time spent T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:58:21.290469 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.290743 18298 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.292974 18465 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.294000 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.296877 14244 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 12:58:21.297732 18564 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:58:21.320744 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.307461 13619 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:21.309900 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.310207 13687 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:58:21.271489 18236 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.237473 18463 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.324494 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.281950 18575 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:21.272599 18143 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.316401 18528 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:21.278029 18412 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.320052 18512 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15.
I20260709 12:58:21.331774 18236 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.456548 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:21.457206 14244 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:21.458082 18412 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.458246 18236 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.458714 18412 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:21.459942 18143 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.460281 18463 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.460559 18544 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:21.320966 18518 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.462323 18544 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.462783 18544 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:21.463426 18572 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 12:58:21.462806 17774 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.464918 18544 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.468441 18505 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:21.468935 18505 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:21.469928 18505 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.471084 18505 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:21.473146 14244 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:21.475404 17960 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.476990 18314 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:21.483309 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 158112us
I20260709 12:58:21.281168 18382 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:21.501588 17719 consensus_meta.cc:287] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.004s
I20260709 12:58:21.502113 17719 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 15. Current term is 16.
I20260709 12:58:21.502408 18501 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.332455 18573 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.181s
I20260709 12:58:21.504415 18573 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:21.504822 18573 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:21.506953 18435 consensus_meta.cc:287] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.679s	user 0.005s	sys 0.004s
I20260709 12:58:21.507678 13617 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:21.445256 18525 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:21.509840 14240 raft_consensus.cc:2468] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:58:21.340374 18536 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:21.510075 13616 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:21.457242 18453 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.458506 18254 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.341226 18571 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.508616 18494 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:21.526404 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:21.511746 18525 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:21.528697 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.537223 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.539523 13691 raft_consensus.cc:2468] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:21.540766 13508 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:21.527585 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:21.529574 13543 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:21.543437 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:21.545420 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.547094 14209 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.547889 13619 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:58:21.548379 14209 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.544942 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.548389 13619 pending_rounds.cc:85] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 10
I20260709 12:58:21.549533 13619 pending_rounds.cc:107] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:58:21.549576 13543 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:21.549966 13619 raft_consensus.cc:2889] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:21.551851 18501 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:21.552731 18501 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:21.552977 18412 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.555657 14209 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.556128 13509 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:21.562817 18236 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:58:21.561657 18277 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.555043 18254 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.527240 17719 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:21.567780 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.544270 18525 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:21.531375 18445 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:21.578161 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.546051 18143 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.546576 18494 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.581307 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.585108 18330 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:21.583613 13534 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:58:21.582543 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.531847 18320 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:58:21.586840 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.587111 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.561095 18463 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.589216 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.589395 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.600741 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.553759 18501 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.591979 13619 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:21.584626 18546 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:21.580745 18495 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.584076 18041 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:21.603205 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.603471 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.605070 13615 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:21.607129 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.583542 17719 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.592799 18554 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:21.611732 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.600857 18440 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:21.604900 18465 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:21.617460 18304 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:21.615059 18502 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:21.612345 18518 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:58:21.606413 18568 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 12:58:21.603494 18535 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:21.622270 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.617952 18304 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:21.625030 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.626192 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.626734 13613 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:21.627538 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.612010 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.591696 18254 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:58:21.628645 17719 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.629194 18266 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:21.631076 18578 log.cc:927] Time spent T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.335s	user 0.003s	sys 0.000s
I20260709 12:58:21.634761 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.649796 13611 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:21.639686 13618 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:21.604084 18463 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.615254 13615 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:21.590608 18445 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.645380 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.643968 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:21.653527 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.653937 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:21.608882 18546 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:21.639211 18304 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.656823 13542 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:21.656752 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.657047 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.660431 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.661093 13540 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:21.661384 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:21.662508 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.662878 18111 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.654472 18153 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:21.663756 18153 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:21.664331 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:21.664758 13583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:21.664960 13616 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:21.665753 18153 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.665956 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.666648 13613 raft_consensus.cc:2410] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:58:21.667457 18463 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.658726 17774 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
W20260709 12:58:21.668942 18569 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:58:21.664397 18304 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.673368 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:21.677328 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.654626 13619 raft_consensus.cc:2376] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11. Re-sending same reply.
I20260709 12:58:21.678714 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.680330 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.680183 13509 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:21.672127 18382 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.679258 18153 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:21.658576 18298 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:21.685710 18298 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:21.686113 18298 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:21.691216 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.694199 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.694387 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.695585 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.695747 13687 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:21.696183 13686 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:21.697136 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.697597 13686 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:21.700187 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:21.694988 18236 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:21.701695 18236 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:21.702126 18236 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:21.689599 17774 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:21.705502 14212 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.705991 17774 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:21.706422 17774 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:21.706784 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:21.709426 18254 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:21.710263 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.710839 18254 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:21.713798 18445 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.711678 18254 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:21.719408 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:21.707670 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.724444 13619 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:21.726189 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.728536 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.729918 13613 raft_consensus.cc:1240] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:21.730005 18576 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:21.731277 18320 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:21.731690 18320 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:21.732841 18320 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:21.733970 18574 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.449s	user 0.001s	sys 0.000s
I20260709 12:58:21.767199 18236 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:21.732831 18567 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.437s	user 0.000s	sys 0.003s
I20260709 12:58:21.724249 18254 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:21.744463 17719 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:21.779320 13619 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:21.780158 13613 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:58:21.783257 13619 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:21.785631 13509 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:21.786773 18370 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:58:21.787261 18370 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:21.788121 18370 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.795600 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.797735 18181 raft_consensus.cc:3055] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:21.798189 18181 raft_consensus.cc:740] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:21.799060 18181 consensus_queue.cc:260] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.800228 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:21.801880 18563 log.cc:927] Time spent T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.662s	user 0.003s	sys 0.012s
I20260709 12:58:21.802445 18454 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:21.803690 18153 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.804255 18153 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.804913 18561 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.806345 18153 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.809321 18181 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:21.838112 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.834878 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:21.813937 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.760790 18546 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.778092 18320 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:21.819541 18573 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.822619 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:21.856696 13618 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:21.825042 18525 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.828192 17997 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:21.828426 18488 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:58:21.833305 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:21.832861 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.837648 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.777187 18254 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:21.839053 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:21.736982 18445 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:58:21.839155 17774 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.853111 18236 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:21.859494 18266 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:21.855082 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.862527 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.862962 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.866029 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.866590 13686 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:21.869001 13687 raft_consensus.cc:1240] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13. Ops: []
W20260709 12:58:21.840322 18519 consensus_meta.cc:287] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.712s	user 0.012s	sys 0.000s
I20260709 12:58:21.871269 13691 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:58:21.876271 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 62426.7us
I20260709 12:58:21.872499 18491 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:21.882102 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.882679 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.884562 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.880728 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.910328 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:21.941995 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 61385.5us
I20260709 12:58:21.883416 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:21.778002 17719 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:21.902746 17774 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:21.904578 18570 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.440s	user 0.000s	sys 0.004s
I20260709 12:58:21.905800 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.907197 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.870230 18412 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:58:21.900704 18266 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:21.908506 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.910328 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.910454 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.911267 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.911813 13508 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.912966 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.913589 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:21.916241 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:21.918815 13689 raft_consensus.cc:1275] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:21.922617 13616 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:21.922668 13686 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:58:21.881325 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:21.926199 18298 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:21.978897 13693 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:21.935705 18573 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:21.955835 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:21.956471 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56540.3us
W20260709 12:58:21.957388 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 74052.2us
I20260709 12:58:21.940052 13688 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:21.943532 13659 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:21.882050 18304 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:21.966626 17719 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:21.980182 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:21.975347 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 45961.8us
I20260709 12:58:21.932814 18525 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:21.982998 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 71253us
I20260709 12:58:21.969483 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.003892 18454 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.004204 13509 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:22.006129 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.007762 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:22.009847 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.010483 13618 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:22.007547 18143 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:22.014648 18143 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:22.015121 18143 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:21.997325 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53876.2us
I20260709 12:58:22.026175 13659 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:21.971117 18576 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:22.027148 18576 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:22.027647 18576 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:21.968684 18581 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:22.036834 13508 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.037999 18490 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:22.057780 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 295798us
I20260709 12:58:21.982264 18266 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:22.065634 13535 raft_consensus.cc:1240] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16. Ops: []
W20260709 12:58:22.066972 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 86858.7us
I20260709 12:58:22.078150 13584 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:22.078493 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 120104us
I20260709 12:58:22.079643 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:22.068091 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.082832 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:21.996033 17774 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:21.973768 17960 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:58:22.088171 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.097769 13584 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:22.098613 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:22.086935 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.102706 13618 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 16.
I20260709 12:58:21.968201 18491 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.123222 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:22.063464 18382 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:22.124794 13584 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:22.127374 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.127749 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.084755 18355 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:22.129388 13614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39902 (request call id 964) took 1286 ms. Trace:
I20260709 12:58:22.102470 17997 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.130887 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.116474 18525 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:22.135610 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:22.140599 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.085232 18581 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.133406 13614 rpcz_store.cc:276] 0709 12:58:20.842859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:20.843731 (+   872us) service_pool.cc:224] Handling call
0709 12:58:20.844109 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:20.881574 (+ 37465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:20.881603 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:20.913411 (+ 31808us) log.cc:844] Serialized 26 byte log entry
0709 12:58:21.229162 (+315751us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:58:21.229213 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:21.229278 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:21.904008 (+674730us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:22.119857 (+215849us) raft_consensus.cc:1685] finished
0709 12:58:22.119878 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:22.129313 (+  9435us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36791,"raft.queue_time_us":40080,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":8611,"spinlock_wait_cycles":36597632,"thread_start_us":314625,"threads_started":1,"wal-append.queue_time_us":407584}
I20260709 12:58:22.113137 13506 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.116252 18528 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:22.116711 18463 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:22.129336 18518 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:22.129842 18502 raft_consensus.cc:2804] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:22.128717 18382 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:22.143630 18111 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:22.120437 18435 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:58:22.140060 18508 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:22.192160 18412 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:22.175299 18528 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:58:22.192521 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 79443.5us
I20260709 12:58:22.192744 18412 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:22.193399 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.193835 18412 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.194660 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.195574 18445 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:22.196297 18445 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:22.196671 18445 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:22.196873 18500 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:22.197074 18445 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:22.197636 18502 raft_consensus.cc:697] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:22.198745 18502 consensus_queue.cc:237] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.184288 18111 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:22.208473 18111 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:22.209242 18491 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.221684 13534 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:22.224484 18254 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.159971 18488 consensus_queue.cc:1048] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:58:22.225466 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:22.150434 18535 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.148819 18525 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:22.149040 18505 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.161826 18454 raft_consensus.cc:2764] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:58:22.153684 18494 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:22.178267 18382 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:22.190261 18581 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.215379 18445 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.240713 13534 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:22.264256 14243 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:58:22.249974 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.255614 14239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:22.265985 14239 raft_consensus.cc:606] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:22.266463 14239 raft_consensus.cc:614] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:22.246088 18525 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:22.279245 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 260486us
I20260709 12:58:22.279836 18576 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:22.369560 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 226650us
I20260709 12:58:22.411942 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:22.413766 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:22.423043 18266 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.424171 13539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45566 (request call id 982) took 1764 ms. Trace:
I20260709 12:58:22.464321 13688 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
W20260709 12:58:22.424835 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 337992us
I20260709 12:58:22.426519 18330 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:22.462496 17719 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.465615 18330 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.466569 18576 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.431252 14239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d73171142594388972ed22eae44f95f"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:22.432255 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:22.433386 18143 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:22.439672 18181 consensus_meta.cc:287] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.007s
W20260709 12:58:22.440587 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 313267us
I20260709 12:58:22.472359 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.442695 18445 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.445123 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:22.461594 18320 consensus_meta.cc:287] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.008s
W20260709 12:58:22.377473 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 240951us
I20260709 12:58:22.467800 17719 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:22.493139 18528 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:22.493702 18528 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:22.494207 18585 log.cc:927] Time spent T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.356s	user 0.003s	sys 0.000s
I20260709 12:58:22.470275 14239 raft_consensus.cc:606] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.474228 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:22.468779 14240 raft_consensus.cc:1240] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 12:58:22.495952 14239 raft_consensus.cc:614] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:22.471517 18603 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:22.502529 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.504151 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.475546 18143 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.476478 18581 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:22.477547 18266 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:22.478142 13684 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:22.468570 18330 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.464479 13539 rpcz_store.cc:276] 0709 12:58:20.660257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:20.710224 (+ 49967us) service_pool.cc:224] Handling call
0709 12:58:20.723747 (+ 13523us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:58:20.761345 (+ 37598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:20.761378 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:58:20.774183 (+ 12805us) log.cc:844] Serialized 52 byte log entry
0709 12:58:21.792466 (+1018283us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b1800469300. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 12:58:21.982407 (+189941us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:58:22.355934 (+373527us) log.cc:844] Serialized 14 byte log entry
0709 12:58:22.423610 (+ 67676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:22.423711 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:22.423768 (+    57us) raft_consensus.cc:1685] finished
0709 12:58:22.423881 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:22.424091 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58213,"prepare.queue_time_us":440341,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":87773,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":2670,"spinlock_wait_cycles":22646400,"thread_start_us":480607,"threads_started":2,"wal-append.queue_time_us":1370223}
I20260709 12:58:22.480042 18602 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:22.480470 18566 consensus_queue.cc:1059] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:22.446858 18111 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.492132 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:22.533463 13691 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:22.492705 18601 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:22.475409 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:22.497028 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 1972) took 1313 ms. Trace:
I20260709 12:58:22.500861 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.501506 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.501844 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.507730 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.523054 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.527627 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.531195 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.529599 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:22.555567 13688 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:22.557076 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.557775 13614 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:22.560541 13610 rpcz_store.cc:276] 0709 12:58:21.183620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:21.711197 (+527577us) service_pool.cc:224] Handling call
0709 12:58:21.711517 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:21.711822 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:21.711851 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:21.714525 (+  2674us) log.cc:844] Serialized 26 byte log entry
0709 12:58:21.833518 (+118993us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:58:22.135892 (+302374us) log.cc:844] Serialized 14 byte log entry
0709 12:58:22.136118 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:22.136215 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:22.493911 (+357696us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b24002b5bd8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b70409d 00007fa02e33610e 00007fa02e34d88c 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221
0709 12:58:22.496688 (+  2777us) raft_consensus.cc:1685] finished
0709 12:58:22.496797 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:22.496956 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7441,"prepare.queue_time_us":150252,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":11293,"raft.queue_time_us":3324,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":9819,"spinlock_wait_cycles":74496768,"thread_start_us":377829,"threads_started":2,"wal-append.queue_time_us":343014}
I20260709 12:58:22.523869 18603 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.571484 18601 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.574021 18603 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:22.574713 18501 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.574914 18111 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.508272 18528 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:22.578634 13611 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:22.586159 14239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:22.587781 18601 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:22.588338 18254 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.588990 18566 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:22.594914 13688 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
W20260709 12:58:22.597196 18588 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:58:22.598568 14239 raft_consensus.cc:606] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:22.605384 14239 raft_consensus.cc:614] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:58:22.601895 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 72386.2us
I20260709 12:58:22.606496 13691 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:22.603670 13614 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:22.608273 13506 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:22.608482 14212 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:22.598713 18573 consensus_meta.cc:287] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.002s
I20260709 12:58:22.610201 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:22.615058 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:22.613317 18519 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:22.632047 18519 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:22.633086 18519 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.613514 18505 consensus_queue.cc:1059] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:22.620723 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.635077 18505 consensus_queue.cc:1243] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:58:22.635376 18581 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.635609 18505 raft_consensus.cc:3055] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:22.636040 18505 raft_consensus.cc:740] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:22.636490 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.637296 18581 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:22.636879 18505 consensus_queue.cc:260] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.638226 18505 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:22.638919 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.602911 18566 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:22.640005 18528 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:22.640949 18566 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.642194 18566 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:22.604786 17997 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:22.622011 14245 raft_consensus.cc:1240] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:22.622149 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:22.622973 18435 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:22.623369 18606 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:22.624130 18501 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:22.658468 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.648293 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.630848 18599 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:22.533388 18602 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:22.624861 14241 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:22.678248 18143 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:22.679013 18143 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.649096 13612 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:58:22.662679 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.626845 14239 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:22.687326 13617 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:22.661096 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.687893 13610 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:22.656090 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.663475 13619 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:22.667868 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.684813 18573 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:22.669842 14245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:22.662113 18435 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:22.669139 13613 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:22.718138 14245 raft_consensus.cc:606] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:22.719194 14245 raft_consensus.cc:614] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:22.693068 13509 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:22.666481 18606 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.668627 18525 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.667973 18501 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:22.727545 13685 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:58:22.686600 18602 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:22.686868 18599 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:22.719656 18435 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:22.745169 18525 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:22.745437 18606 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.747404 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.739626 13610 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:22.716259 18528 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:58:22.734973 18586 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:22.734709 17997 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:22.763175 17997 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.718525 18111 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:58:22.769969 18599 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.774673 18501 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.779618 18528 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:22.780148 18528 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:22.781210 18528 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.782840 18528 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:22.769582 13617 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:22.753371 18143 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:22.821817 18586 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:22.841045 18586 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.860035 13617 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:22.862039 13618 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:22.819502 18581 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:22.864399 18581 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.867027 18581 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:22.823993 18382 consensus_meta.cc:287] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.003s
I20260709 12:58:22.869040 18382 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:22.871613 18382 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:22.863548 17997 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:22.872465 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.885401 18602 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.904763 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:22.913393 13617 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:22.916353 18602 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:22.915797 18435 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:22.940486 18435 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:22.942286 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:22.967265 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 238815us
I20260709 12:58:22.971052 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:22.971405 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49526 (request call id 1889) took 1025 ms. Trace:
I20260709 12:58:22.972087 18561 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:22.972236 13535 rpcz_store.cc:276] 0709 12:58:21.946103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:21.946290 (+   187us) service_pool.cc:224] Handling call
0709 12:58:21.946610 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:22.085422 (+138812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:22.971142 (+885720us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540007e4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:22.971341 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273256192}
I20260709 12:58:22.975415 18143 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:22.979604 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:22.993695 13690 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:58:23.002460 13691 raft_consensus.cc:1240] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:23.010378 13535 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87a8c869a6b840e591423340bc32af00"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:23.011524 13535 raft_consensus.cc:606] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:23.021695 14240 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59164 (request call id 1310) took 1227 ms. Trace:
I20260709 12:58:23.025143 14240 rpcz_store.cc:276] 0709 12:58:21.794385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:21.794515 (+   130us) service_pool.cc:224] Handling call
0709 12:58:21.794846 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:22.496201 (+701355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:23.021190 (+524989us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b540069e954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:23.021593 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549305088}
W20260709 12:58:23.027689 18587 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.605s	user 0.010s	sys 0.010s
I20260709 12:58:23.047091 13610 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:58:23.064224 13616 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16. Ops: []
W20260709 12:58:23.066748 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:23.072822 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.077008 13687 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:23.082823 18298 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.075527 18236 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:58:23.087754 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:23.094856 14245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a315f47e47184081bd93e74c72dba511"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:23.095523 14245 raft_consensus.cc:606] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:23.099920 13619 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:23.102059 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.142450 13508 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.145339 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:23.146543 13688 raft_consensus.cc:2410] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:58:23.150424 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:23.151468 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.153220 13689 raft_consensus.cc:2393] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:23.154065 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:23.145694 17719 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.155493 17719 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.159433 18536 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:58:23.141970 18621 log.cc:927] Time spent T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:58:23.167590 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.169078 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.172354 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:23.174993 18598 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:23.175499 18598 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:23.145809 18277 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:23.187513 13692 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 11: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:23.189124 18111 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:23.192054 18382 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.208709 17719 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.166836 18235 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:58:23.207624 18612 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.470s	user 0.004s	sys 0.000s
I20260709 12:58:23.157132 18617 consensus_queue.cc:1059] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:58:23.186719 18511 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.164404 18304 consensus_queue.cc:1059] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:58:23.206988 18500 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:23.232465 18304 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:23.232924 18304 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:23.207046 18525 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:23.233613 18525 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.234016 18304 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.235942 18525 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:23.232502 18500 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.235489 18304 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:23.242973 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:23.249817 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 86623.1us
I20260709 12:58:23.224186 18382 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.241291 18511 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.258745 18500 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:23.272925 18619 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:58:23.281279 18314 raft_consensus.cc:3055] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:23.281774 18314 raft_consensus.cc:740] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:23.282687 18314 consensus_queue.cc:260] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.283140 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:23.284135 18382 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:23.281730 18511 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:23.319563 18490 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:23.320241 18490 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.323312 18490 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:23.347338 18435 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.348001 18435 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.350329 18435 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:23.357266 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:23.377923 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:23.392650 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:23.394548 13684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45328 (request call id 1810) took 1534 ms. Trace:
I20260709 12:58:23.394995 13684 rpcz_store.cc:276] 0709 12:58:21.860537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:21.860838 (+   301us) service_pool.cc:224] Handling call
0709 12:58:21.861239 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:58:22.523875 (+662636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:23.394202 (+870327us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b5400002154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:23.394478 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418207488}
I20260709 12:58:23.415315 17774 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:23.415930 17774 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.418380 18511 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:23.418947 18511 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.420418 17774 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.420991 18511 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:23.421468 18435 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:23.421881 14243 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59176 (request call id 765) took 1513 ms. Trace:
I20260709 12:58:23.421932 18435 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.422363 14243 rpcz_store.cc:276] 0709 12:58:21.908691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:21.915011 (+  6320us) service_pool.cc:224] Handling call
0709 12:58:21.915311 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:22.264718 (+349407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:23.421621 (+1156903us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54007102d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:23.421823 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789347584}
I20260709 12:58:23.424139 18298 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:23.427484 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.428437 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:23.428758 18586 raft_consensus.cc:993] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:23.429266 18586 raft_consensus.cc:1081] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:23.429859 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.430698 13540 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:23.432953 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:23.428843 13539 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:23.423424 18519 consensus_queue.cc:1059] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:23.438280 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:23.436800 18519 raft_consensus.cc:3055] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:23.439810 18604 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:23.439891 18519 raft_consensus.cc:740] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:23.440364 18604 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:23.440795 18604 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:23.441080 18519 consensus_queue.cc:260] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.442354 18519 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:23.444310 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:23.448354 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:23.448635 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:23.450011 17774 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:23.450441 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:23.450726 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:58:23.450613 17774 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.452950 17774 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:23.454308 18573 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:23.454775 18573 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.455235 18573 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:23.455855 18604 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.484777 13611 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:58:23.491392 18511 consensus_queue.cc:1059] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:23.465894 13540 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:23.494406 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:23.471760 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.501670 13543 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:23.471405 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:23.467017 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.468961 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.503083 13688 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:23.469532 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.474000 13617 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:23.481329 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:23.482285 13616 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:58:23.464773 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:23.494266 13692 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:58:23.467890 13689 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:23.509666 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:23.510598 13541 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 12: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:23.472844 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:23.516296 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:23.532708 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 43981.5us
I20260709 12:58:23.538682 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:23.539371 13612 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
W20260709 12:58:23.549487 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 76718.4us
I20260709 12:58:23.550581 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:23.553594 13534 raft_consensus.cc:1240] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:23.557823 18235 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:23.559083 18235 consensus_queue.cc:1243] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:58:23.539793 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:23.565055 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:23.566529 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:23.569974 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:23.573534 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.574132 13684 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:23.567695 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.575615 14242 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:58:23.582617 13536 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:23.584349 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:23.415381 18622 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:23.593513 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:23.494381 18604 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.592520 14209 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.584846 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.467708 18561 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:23.596396 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.454754 18435 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.602327 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.558715 18320 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:58:23.575289 18528 raft_consensus.cc:3055] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:23.552096 17997 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.592864 18382 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:58:23.610347 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 55710.7us
I20260709 12:58:23.610483 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:23.563297 13690 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87a8c869a6b840e591423340bc32af00"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:23.558882 18576 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:23.589190 18602 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:23.604463 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:23.632661 13540 raft_consensus.cc:2410] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:58:23.627568 13535 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:23.627496 13690 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:23.633965 13690 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:23.464257 18598 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.606093 18525 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:23.631803 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.614621 18528 raft_consensus.cc:740] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:23.596287 18511 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:58:23.616156 18604 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.641129 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.606482 18235 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:58:23.650270 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.628170 18640 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:23.609622 18561 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.662667 13617 pending_rounds.cc:85] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 8
I20260709 12:58:23.665872 13617 pending_rounds.cc:107] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:58:23.666239 13617 raft_consensus.cc:2889] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:23.631950 18602 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:23.672969 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.605123 18622 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.628695 17997 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.635293 18576 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.679208 18602 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:23.745520 18649 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:23.745662 18573 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.746307 18649 consensus_queue.cc:1059] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:23.736249 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.747731 18561 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.739760 18277 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:23.748715 18277 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.750867 18277 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.753499 13542 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:23.754297 18622 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:23.755576 18528 consensus_queue.cc:260] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.757236 18528 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:23.682262 18525 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:23.759451 18525 raft_consensus.cc:3055] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:23.759938 18525 raft_consensus.cc:740] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:23.760990 18525 consensus_queue.cc:260] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.751999 18111 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:58:23.763530 18647 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:23.764115 18647 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:23.765190 18647 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.766511 18647 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:23.767777 18602 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.669062 18382 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:23.770658 18382 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:23.775677 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:23.775977 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.775399 13543 raft_consensus.cc:2468] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:23.777431 13541 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:23.780572 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:23.784111 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 106261us
I20260709 12:58:23.786782 13684 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:58:23.787225 13684 pending_rounds.cc:85] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 9
I20260709 12:58:23.787477 13684 pending_rounds.cc:107] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:58:23.787740 13684 raft_consensus.cc:2889] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:23.800906 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:23.804523 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:23.806493 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.814806 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.817863 13658 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:23.664036 18604 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:23.791388 18641 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:58:23.836225 13509 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:23.838886 18445 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:23.857594 18314 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:23.864645 18634 log.cc:927] Time spent T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:58:23.866410 18602 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:23.867071 18602 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:23.873587 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.874727 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:23.875327 13693 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:23.878253 18576 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:23.881742 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:23.892927 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 300491us
W20260709 12:58:23.895123 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 58955.7us
I20260709 12:58:23.896368 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:23.903190 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 334058us
I20260709 12:58:23.909957 17719 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:58:23.892601 18587 log.cc:927] Time spent T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:58:23.924669 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 121765us
I20260709 12:58:23.926779 13610 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:23.919456 13536 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:23.906805 13690 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.834483 18646 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:23.927850 13692 raft_consensus.cc:1240] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:23.929111 13690 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:23.931281 18511 consensus_queue.cc:1243] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:58:23.934237 13506 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:23.937570 18277 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:23.938989 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:23.940003 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:23.938234 18277 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:23.914083 18445 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:23.913686 18617 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:23.932181 18646 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:23.963447 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:23.972922 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:58:23.979701 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 160663us
I20260709 12:58:23.982151 13534 raft_consensus.cc:1240] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:23.955065 18511 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:23.949347 18652 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:23.976862 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:23.966470 18445 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:23.955408 18489 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:58:23.971864 18646 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:23.968964 18536 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:58:24.040446 18604 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.042059 18604 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:24.011830 18511 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.043782 18277 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:24.044114 18667 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:24.044781 18511 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:24.045053 18653 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:24.045790 18511 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:24.011547 18445 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:24.069651 13616 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:24.071581 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:24.073200 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 96521.2us
I20260709 12:58:24.062670 18657 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:24.075734 18277 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:24.075805 18658 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:24.086575 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:24.077327 13584 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:24.079581 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 99113us
I20260709 12:58:24.088793 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.076364 18657 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:24.091802 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.044806 18667 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.094909 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.114806 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:24.099347 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 141138us
I20260709 12:58:24.100117 18617 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:24.090046 18658 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.118434 18658 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:24.109763 18643 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:24.118970 18643 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:24.045784 18653 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.119938 18382 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:24.119850 18643 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.120905 18382 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:24.121209 18643 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:24.121639 18382 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:24.121963 18653 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.090888 18111 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:58:24.108562 18656 log.cc:927] Time spent T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:58:24.116429 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.087759 18277 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.140338 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:24.099447 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 151109us
I20260709 12:58:24.141379 18669 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:24.141628 18528 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:24.093060 18670 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:24.142547 13689 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:58:24.142913 13689 pending_rounds.cc:85] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 9
I20260709 12:58:24.143131 13689 pending_rounds.cc:107] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:58:24.143370 13689 raft_consensus.cc:2889] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:24.113003 18657 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.144913 18657 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:24.141829 13506 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.148692 18320 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:58:24.149608 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:24.149816 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:24.150460 13534 raft_consensus.cc:3055] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:24.150790 13534 raft_consensus.cc:740] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:24.115509 18667 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.151558 13534 consensus_queue.cc:260] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.153424 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:24.168764 13536 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:24.169047 13584 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:24.155238 13534 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:24.171242 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.159536 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:24.171869 13692 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 10.
I20260709 12:58:24.159907 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:24.172771 13658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:24.173492 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.166797 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.176137 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:24.151868 13506 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:24.185431 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:24.186077 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:24.186180 13616 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:24.186779 13614 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:24.195390 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:24.184507 13658 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
W20260709 12:58:24.198697 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 46890.2us
I20260709 12:58:24.115487 18646 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.232257 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
W20260709 12:58:24.231619 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46915.2us
I20260709 12:58:24.223819 18382 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:24.244287 13539 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:24.189080 18660 raft_consensus.cc:2804] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:24.192849 18525 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:24.153626 18277 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.238828 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:24.150460 18489 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:24.288161 13612 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 15: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:24.243539 18382 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:24.238519 18235 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:24.184043 18667 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:24.344942 18667 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.346547 18445 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.347119 18667 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.349942 18445 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.350461 17997 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.351992 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:24.353377 18235 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:24.353905 18235 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:24.355592 18488 raft_consensus.cc:993] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:24.356318 18488 raft_consensus.cc:1081] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
W20260709 12:58:24.358886 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 59353.8us
I20260709 12:58:24.361159 13509 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.363071 18236 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:24.366132 18236 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:24.367089 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.371558 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:24.373580 13539 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:58:24.252391 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:24.374764 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:24.259189 13690 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:24.261344 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.265429 13537 raft_consensus.cc:2410] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:58:24.379056 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:24.232585 18511 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:24.280404 18646 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.292814 18652 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:24.293792 18573 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.288924 18660 raft_consensus.cc:697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:24.227697 13534 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:24.300766 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:24.301647 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.301956 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:24.246044 18622 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:24.266374 18674 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:24.379693 14240 raft_consensus.cc:3060] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:24.385367 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:24.387984 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:24.401367 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:24.333971 18382 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:24.406245 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:24.407204 13658 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:24.411201 18660 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.409381 14243 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 10.
I20260709 12:58:24.420677 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:24.422254 13693 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:58:24.424168 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 171844us
I20260709 12:58:24.497884 14245 raft_consensus.cc:1275] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:58:24.501464 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 94375.7us
I20260709 12:58:24.505307 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.507230 14240 raft_consensus.cc:2468] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:24.522847 18643 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:24.523476 18643 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:24.523942 18643 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:24.540046 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.408900 18511 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:24.543823 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:24.547581 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 380853us
W20260709 12:58:24.551589 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 358545us
I20260709 12:58:24.552718 13659 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:24.558220 18511 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.500387 18654 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:24.560672 18654 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:24.533279 18111 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:58:24.561584 18654 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.564029 18670 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:24.564456 18670 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:24.564831 18670 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:24.543170 18641 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:58:24.568280 13693 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:24.533806 18602 raft_consensus.cc:2804] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:24.585201 14211 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.570384 18525 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:24.615931 13537 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:24.616631 13617 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 12:58:24.617974 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 382180us
I20260709 12:58:24.631909 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.595729 18671 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:24.605407 18655 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:24.649962 18649 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:24.618947 18648 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:24.645404 18657 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.000s
I20260709 12:58:24.617494 18111 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:24.693091 18111 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.642028 18528 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:24.649000 18646 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:24.669912 14212 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:24.660000 18666 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:24.642026 18653 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:24.631599 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 270495us
W20260709 12:58:24.684988 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:24.680703 18680 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:24.691854 18648 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.704028 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:24.710745 18643 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.625442 18602 raft_consensus.cc:697] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:24.712669 18602 consensus_queue.cc:237] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.643198 18661 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:24.651731 18573 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:24.742187 18657 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:24.695886 18111 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.767943 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:24.697019 18679 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:24.776809 13610 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:24.652456 18671 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.742110 18502 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:58:24.794471 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 422214us
I20260709 12:58:24.764153 18236 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:24.802445 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 181030us
I20260709 12:58:24.696254 18646 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:24.808463 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 184394us
I20260709 12:58:24.683727 18525 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:58:24.696218 18528 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.672487 18235 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.697505 18653 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.815150 13543 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:24.820104 13506 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:24.823055 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.761593 18657 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.824843 13690 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:24.825336 18643 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.826138 18528 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.826571 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:24.819455 18314 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:24.827561 18314 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.814395 18236 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.830202 18314 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.830838 18236 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:24.831274 18678 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.319s	user 0.000s	sys 0.002s
I20260709 12:58:24.806178 18502 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.833392 13691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a315f47e47184081bd93e74c72dba511"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:40872
I20260709 12:58:24.837230 13610 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:24.838702 13612 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:24.840245 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:24.767652 18573 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.745919 18661 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.850498 13691 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:24.853540 13691 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:24.680605 18670 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.823426 18576 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:58:24.884534 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.896862 13691 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.832468 18445 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:58:24.859023 18314 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:24.905685 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 98730.2us
I20260709 12:58:24.909538 18653 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.912984 18646 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.847700 18382 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.914472 18690 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:24.915015 18690 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:24.915411 18382 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:24.916468 18648 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.917223 13684 raft_consensus.cc:2368] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12.
I20260709 12:58:24.917155 18690 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.925638 18671 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.927366 18661 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.928153 18657 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:24.928273 18688 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:24.928846 18235 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.928900 18688 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:24.931329 18688 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:24.932269 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:24.860071 18525 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:58:24.932920 13619 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
W20260709 12:58:24.956948 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 72553us
I20260709 12:58:24.940833 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:24.957849 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.958240 13537 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
W20260709 12:58:24.959095 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 52936us
I20260709 12:58:24.939819 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:24.961410 13543 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:24.961987 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.938097 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.968395 13689 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:24.969283 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.970096 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:24.971283 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:24.956158 13438 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 8, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 8 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:24.931452 18502 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:24.969909 13689 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:24.951099 13584 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.975402 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:24.971659 18655 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:24.997076 18694 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:24.997732 18694 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:25.002249 18684 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.482s	user 0.000s	sys 0.002s
W20260709 12:58:24.976632 18683 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:58:25.007201 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:24.931643 18670 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:25.009464 13537 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 15: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:25.012953 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:25.013634 14244 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:25.009694 18694 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:25.028121 18687 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.511s	user 0.000s	sys 0.003s
I20260709 12:58:25.036643 13543 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:25.033164 18598 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:25.044979 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:24.975005 18653 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:25.054903 18653 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.057998 18653 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:25.060770 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.061638 13537 raft_consensus.cc:2393] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 16: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:25.063383 18501 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:25.063855 18501 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.064258 18501 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:25.071501 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 122069us
I20260709 12:58:24.974776 18680 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:25.072897 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:24.977777 13658 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:25.078619 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.078969 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.079489 13619 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:25.080886 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.081609 13610 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:58:25.083345 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.085794 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:25.086633 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 108986us
I20260709 12:58:25.089295 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.094066 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.094703 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:25.097427 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.098330 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:25.101336 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.104943 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:25.105234 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.074272 18671 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:25.099164 18657 consensus_queue.cc:1059] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:58:25.098124 18694 raft_consensus.cc:2764] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:58:25.110625 18657 consensus_queue.cc:1243] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:58:24.950628 18573 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:25.092023 18654 consensus_queue.cc:1048] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:58:25.077420 18680 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:25.114058 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.104285 18693 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:25.096262 18502 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:25.131537 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:25.136766 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:25.136513 18501 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.116416 18111 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:25.136998 18666 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:25.138710 18669 raft_consensus.cc:3055] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:25.140355 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.188402 13619 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:25.141307 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.189471 13612 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:25.146960 18671 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:25.148586 13534 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:58:25.151139 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.191628 13616 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:25.153198 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.155546 13582 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:25.156487 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.158286 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.166939 18435 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:25.168457 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.170492 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.116125 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:25.172963 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:25.206930 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 51476.5us
I20260709 12:58:25.211314 13692 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 13. Current term is 14. Ops: []
I20260709 12:58:25.212037 13658 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.212682 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.213176 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.214452 13689 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:25.098908 18653 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:25.174258 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:25.175761 13543 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:25.175976 13509 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
W20260709 12:58:25.178169 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 46730us
I20260709 12:58:25.117933 18561 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:25.172521 18686 log.cc:927] Time spent T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:58:25.175625 18680 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:25.221051 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46884.8us
I20260709 12:58:25.207223 18676 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:25.187937 18666 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:25.223356 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.173467 18673 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:58:25.192575 18671 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.221650 18661 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:25.225190 13506 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:25.193254 18111 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:25.228554 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.188158 18669 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:25.217072 18653 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.237558 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:25.222424 18586 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:25.224658 18649 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:25.183393 18693 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.256719 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:25.229475 18711 log.cc:927] Time spent T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:58:25.222500 18622 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:25.248329 13582 leader_election.cc:400] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12.
I20260709 12:58:25.249012 18614 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:25.249030 18653 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.232999 18502 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:25.292284 18502 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.294664 18502 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.220170 18435 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.226895 18676 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:25.230139 18694 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:25.299597 18435 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:25.275400 18709 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:58:25.300266 13612 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:25.304020 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.304652 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.305491 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:25.306944 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.245282 18528 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:58:25.277001 18693 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:25.258181 18382 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:25.321127 18382 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.323356 18382 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:25.277226 13543 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:25.247877 18673 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:25.225064 18561 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.278260 18501 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:25.278661 13537 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:25.286010 13684 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:58:25.287137 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:25.287657 18671 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.312304 14244 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 12: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:25.335196 18561 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.313170 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.336256 13536 raft_consensus.cc:2368] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13.
I20260709 12:58:25.273777 18614 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.316015 18111 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.307488 18640 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:25.306695 18519 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:58:25.326669 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 70048.2us
I20260709 12:58:25.381340 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.329303 18511 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:58:25.330140 18648 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:25.330595 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55906.4us
I20260709 12:58:25.337009 13541 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:25.340669 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.384783 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.340745 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.385360 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.385550 13610 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:25.348279 13619 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:58:25.299829 18676 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:25.349066 18614 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.382788 18640 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.380640 18519 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.396333 18519 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:25.397385 18671 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:25.325476 18622 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:25.405117 18622 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.406652 18519 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.408406 13534 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:25.409700 18694 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:25.411885 18713 log.cc:927] Time spent T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.134s	user 0.001s	sys 0.004s
I20260709 12:58:25.417407 13691 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:25.388960 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.420284 13536 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:25.391327 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.423326 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.392643 13537 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:58:25.430310 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.387408 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.311946 18631 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:58:25.386848 18382 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:58:25.443687 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56358.8us
I20260709 12:58:25.445588 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.446370 13615 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:25.445202 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.392866 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.435335 14237 raft_consensus.cc:1275] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:58:25.441481 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.449113 13690 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:25.450727 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.451370 13619 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:25.443219 17997 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:25.437830 18649 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:25.447690 18602 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:25.438285 18660 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:58:25.452661 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 62968.4us
I20260709 12:58:25.460351 13584 leader_election.cc:400] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13.
I20260709 12:58:25.460692 13584 leader_election.cc:403] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:25.452905 17997 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:25.461341 18654 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:25.461916 17997 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:25.455281 18602 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.454876 18586 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.455801 18519 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.478837 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.480851 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:25.481493 14237 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:25.475786 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.482822 13691 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:25.452049 18646 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:58:25.482003 14237 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:25.486456 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.473088 18660 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:25.490293 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.491753 13619 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:25.477207 18602 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:25.489198 18680 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.505048 18680 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.505666 14244 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:25.477977 18586 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.508302 18704 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:58:25.513585 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.488474 14237 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.516494 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.516889 14237 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:25.499221 18640 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.469069 18717 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:58:25.454071 18649 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.501628 18669 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.502110 18660 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.517131 13619 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:25.517592 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.521863 13690 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:25.486714 13615 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:25.522753 14209 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:25.517879 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.529052 13659 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.530460 13615 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:58:25.476693 18382 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:58:25.536080 18660 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:25.519449 18646 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:25.519207 18661 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:25.537886 18661 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.537771 17997 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.538501 18661 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:25.537933 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:25.538246 18511 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:25.543557 18511 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:25.544763 18511 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.538614 13617 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:25.538465 18676 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.549324 13610 raft_consensus.cc:1240] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:25.549921 18704 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13.
I20260709 12:58:25.552018 18640 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:25.552489 18640 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.552875 18640 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.552956 18676 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:25.556352 18661 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.557060 18631 consensus_queue.cc:1059] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:25.558286 18631 raft_consensus.cc:3055] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:25.556499 17997 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.547349 18614 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.558665 18631 raft_consensus.cc:740] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:25.590083 13610 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:58:25.591733 18382 consensus_queue.cc:1048] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:58:25.592151 18694 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:25.592387 18661 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.593082 18693 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:25.594040 13658 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.617455 18614 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.599403 18382 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:25.593526 18693 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:25.590098 18717 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:58:25.637190 14237 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:25.638628 18382 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:25.635525 18586 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.640244 18382 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.641669 18586 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.642393 18382 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:25.643610 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:25.644243 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:25.648392 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.649055 13610 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:25.650193 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.634076 18631 consensus_queue.cc:260] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.598708 18694 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:58:25.652776 18382 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:25.653792 18382 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.654161 18382 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:25.655603 18653 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.658062 18653 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:25.626672 18714 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.366s	user 0.001s	sys 0.001s
I20260709 12:58:25.659148 18602 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.623863 18640 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:25.660633 18712 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.386s	user 0.003s	sys 0.000s
I20260709 12:58:25.660919 14244 raft_consensus.cc:2368] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:25.661283 18640 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.661325 18602 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:25.663677 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.664333 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.664892 13615 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.665489 18648 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:25.665910 18648 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:25.666129 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.666728 13612 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.666849 18648 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:25.667776 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 224479us
I20260709 12:58:25.668741 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:25.670925 13582 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.670516 13610 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:58:25.676026 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.676573 13684 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:25.677049 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.679824 18666 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.681049 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.682094 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.682464 18528 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:25.682719 13692 raft_consensus.cc:2393] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:25.683024 18528 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.683495 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.684206 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.685210 13691 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:25.689632 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.690151 13686 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:25.690405 13585 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:25.693092 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:25.693710 13685 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.675220 18693 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.699150 18598 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:25.700753 18598 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:25.701689 18598 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.681083 18666 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:25.703059 18598 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.707445 18693 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:25.709513 18669 raft_consensus.cc:3060] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:25.711093 18631 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:25.716566 18528 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:25.717960 18648 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:25.719446 13689 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:25.720299 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:25.733945 18236 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:58:25.721655 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 250957us
I20260709 12:58:25.129254 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:25.744385 13438 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.746556 18690 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:25.746217 13438 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.747084 18690 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.722210 18688 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:25.725612 13612 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:25.754819 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.747839 13438 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 14, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.728094 13584 leader_election.cc:400] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:25.756855 13584 leader_election.cc:403] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:25.757066 18599 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:58:25.757506 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 78337.7us
I20260709 12:58:25.759053 18688 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:25.759511 18688 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:25.734175 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 289045us
W20260709 12:58:25.760813 13584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:25.761713 13690 raft_consensus.cc:2410] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:58:25.761759 13509 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.764902 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.765551 13542 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:25.766121 13539 raft_consensus.cc:1240] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:25.766479 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:25.766691 13685 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:25.767419 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.767804 13536 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:58:25.767020 13543 raft_consensus.cc:3060] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:25.773257 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.775272 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:25.776187 13534 raft_consensus.cc:3060] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.776649 13684 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:25.778467 13659 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.780689 18693 raft_consensus.cc:2804] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:25.774133 13438 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:25.781283 18693 raft_consensus.cc:697] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:25.782187 18693 consensus_queue.cc:237] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.771790 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:25.790189 14243 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:25.791641 13615 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:25.792227 18573 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
W20260709 12:58:25.763628 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 56344.4us
I20260709 12:58:25.793380 13687 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:25.795233 14209 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:25.802208 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.806850 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.809978 13534 raft_consensus.cc:2468] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:25.793329 18649 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:25.749666 18617 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:58:25.833958 13543 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:25.821240 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.844895 13615 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:25.822268 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:25.824946 13438 catalog_manager.cc:5697] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:25.847185 13610 raft_consensus.cc:2468] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:25.848589 13584 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:25.850069 13659 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.851148 18654 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:25.851744 18654 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:25.852106 18435 raft_consensus.cc:2764] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:58:25.852610 13438 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 10 to 11, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 11 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:25.852780 18654 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.854473 13438 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 10, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 10 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.818782 18586 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:25.818378 18646 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:25.856065 13438 catalog_manager.cc:5697] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 10 to 13, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 13 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.857208 18646 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:25.858153 13438 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 12 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.858616 18646 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.817597 18599 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:25.796921 13686 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:25.862649 13615 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:25.828976 18704 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:25.864581 13438 catalog_manager.cc:5697] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:25.875451 13438 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 12, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 12 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:25.880817 13438 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.860844 18627 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:58:25.884445 13438 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.869345 18602 raft_consensus.cc:2804] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:25.883200 18586 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:25.881584 18693 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:25.923470 18693 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.923807 18586 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.925889 18693 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:25.887424 18704 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:25.926982 18704 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:25.880439 18599 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:25.932766 18599 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.933658 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:25.935158 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:25.894443 13688 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:25.899223 18519 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:25.936679 18519 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:25.937534 13687 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:58:25.910028 18382 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:25.913421 13440 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49880 (request call id 329) took 34325 ms (client timeout 15000 ms). Trace:
I20260709 12:58:25.937574 18519 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:25.943928 18519 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:58:25.941071 13440 rpcz_store.cc:269] 0709 12:57:51.588538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:57:51.588824 (+   286us) service_pool.cc:224] Handling call
0709 12:57:52.858407 (+1269583us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540041e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 12:58:25.913316 (+33054909us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:19.769739 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:20.277397 (+507658us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:20.279883 (+  2486us) write_op.cc:435] Acquired schema lock
0709 12:58:20.279930 (+    47us) tablet.cc:662] Decoding operations
0709 12:58:20.281311 (+  1381us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:20.281427 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:20.281468 (+    41us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:58:20.282709 (+  1241us) tablet.cc:701] Row locks acquired
0709 12:58:20.282769 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:58:20.283074 (+   305us) write_op.cc:270] Start()
0709 12:58:20.283268 (+   194us) write_op.cc:276] Timestamp: P: 1783601900282990 usec, L: 0
0709 12:58:20.283316 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:58:20.424988 (+141672us) log.cc:844] Serialized 3599 byte log entry
0709 12:58:21.269013 (+844025us) op_driver.cc:464] REPLICATION: finished
0709 12:58:21.523792 (+254779us) write_op.cc:301] APPLY: starting
0709 12:58:21.527037 (+  3245us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:21.779101 (+252064us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:21.779257 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:21.885373 (+106116us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:21.923534 (+ 38161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 12:58:21.923640 (+   106us) write_op.cc:312] APPLY: finished
0709 12:58:21.924152 (+   512us) log.cc:844] Serialized 89 byte log entry
0709 12:58:22.310251 (+386099us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:22.416144 (+105893us) write_op.cc:454] Released schema lock
0709 12:58:22.416227 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:22.419296 (+  3069us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:22.419331 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":157789,"reported_tablets":49,"spinlock_wait_cycles":2859061504,"child_traces":[["op",{"apply.queue_time_us":242562,"apply.run_cpu_time_us":44579,"apply.run_wall_time_us":895453,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41711,"num_ops":9,"prepare.queue_time_us":456635,"prepare.run_cpu_time_us":15146,"prepare.run_wall_time_us":828363,"raft.queue_time_us":442651,"raft.run_cpu_time_us":13715,"raft.run_wall_time_us":322287,"replication_time_us":943271,"thread_start_us":851194,"threads_started":5,"wal-append.queue_time_us":697906,"wal-append.run_cpu_time_us":37772,"wal-append.run_wall_time_us":2255659}]]}
W20260709 12:58:25.948052 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70360.3us
W20260709 12:58:25.949255 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:25.950505 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.953698 18602 raft_consensus.cc:697] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:25.954561 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:25.955224 13534 raft_consensus.cc:2368] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:25.952370 13438 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 10, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.963075 13659 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:25.925889 18690 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:25.951658 17997 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:58:25.967056 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:25.977429 13543 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:58:25.983232 13584 leader_election.cc:400] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:25.985253 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:25.986230 13616 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 16: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:25.971725 13438 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 5 to 11, term changed from 9 to 11, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.989193 13438 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.990826 13438 catalog_manager.cc:5697] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:25.992584 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:25.997334 13688 raft_consensus.cc:2368] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 16. Current term is 17.
I20260709 12:58:25.998792 13438 catalog_manager.cc:5697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 7 to 13, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:26.007290 13688 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:58:26.007710 13688 pending_rounds.cc:85] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 6
I20260709 12:58:26.008122 13688 pending_rounds.cc:107] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:58:26.008392 13688 raft_consensus.cc:2889] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:26.005625 13438 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:26.017201 13438 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:25.985143 18669 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:25.988276 18708 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:26.059656 18708 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:26.061129 18708 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:26.070287 13615 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:58:26.070724 13615 pending_rounds.cc:85] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 6
I20260709 12:58:26.070992 13615 pending_rounds.cc:107] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:58:26.071259 13615 raft_consensus.cc:2889] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:26.004874 18648 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:58:26.078336 13506 leader_election.cc:400] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 16. Current term is 17.
I20260709 12:58:26.079351 13506 leader_election.cc:403] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:26.093478 13615 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:58:26.095508 18382 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:26.031559 18573 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:58:26.099467 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:26.103339 13541 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:58:26.104979 18688 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:26.107205 18688 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:26.113149 18725 log.cc:927] Time spent T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.175s	user 0.002s	sys 0.000s
I20260709 12:58:26.119840 13541 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
W20260709 12:58:26.120746 18724 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.226s	user 0.003s	sys 0.000s
I20260709 12:58:26.122268 13537 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:26.124092 18489 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:26.124495 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:26.125722 18561 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:26.046607 18602 consensus_queue.cc:237] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:26.132843 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54733.1us
I20260709 12:58:26.135352 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:26.136045 13612 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:26.136263 13616 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:58:26.136672 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:26.141134 17774 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 16. Current term is 17.
I20260709 12:58:26.153155 18511 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:26.157362 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:26.157958 13658 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:26.159570 13688 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:26.169818 13612 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:26.157195 18688 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 12:58:26.179009 18727 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.242s	user 0.001s	sys 0.001s
I20260709 12:58:26.082417 18669 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:58:26.188747 13506 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:26.191072 18546 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:26.186379 18653 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:58:26.272572 18735 log.cc:927] Time spent T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:58:26.235860 18653 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:26.303244 14237 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:58:26.315346 13534 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:58:26.313366 18649 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:58:26.347580 13693 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:58:26.350313 18653 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:26.359344 18573 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:58:26.389101 13686 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:26.390231 18382 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:26.394963 13615 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:58:26.402452 18330 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:58:26.372006 18730 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:58:26.430202 13534 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:58:26.434275 18733 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:58:26.443528 13612 raft_consensus.cc:1275] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:58:26.452673 14245 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:58:26.459617 13543 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: []
W20260709 12:58:26.456494 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 44368.4us
I20260709 12:58:26.455742 13686 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:58:26.466923 18528 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:26.473192 18382 consensus_queue.cc:1048] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:26.479251 13688 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:58:26.483191 18648 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:26.484910 18598 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:26.490500 13686 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:58:26.545643 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 66988.7us
W20260709 12:58:26.556095 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 74320.9us
I20260709 12:58:26.501319 18435 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 12:58:26.573161 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 74313.3us
I20260709 12:58:26.536597 18648 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 12:58:26.579998 18744 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:58:26.544407 18654 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:58:26.598904 13688 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:58:26.599444 13688 pending_rounds.cc:85] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 8
I20260709 12:58:26.599717 13688 pending_rounds.cc:107] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:58:26.600029 13688 raft_consensus.cc:2889] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:58:26.720209 18754 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 12:58:26.722960 18753 log.cc:927] Time spent T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 12:58:26.712723 18748 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:58:26.697055 18111 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:58:26.750314 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 190787us
I20260709 12:58:26.733532 18111 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:26.940824 18111 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:26.739739 18236 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 12:58:27.111727 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 285144us
W20260709 12:58:27.136392 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 397322us
W20260709 12:58:27.093294 17997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.015s	sys 0.020s
W20260709 12:58:27.152035 13509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.014s	sys 0.028s
W20260709 12:58:27.167469 13509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.014s	sys 0.028s
W20260709 12:58:27.128795 18758 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.336s	user 0.001s	sys 0.022s
W20260709 12:58:27.128922 18760 log.cc:927] Time spent T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.297s	user 0.004s	sys 0.023s
W20260709 12:58:27.185408 17997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.015s	sys 0.020s
W20260709 12:58:27.204488 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 374140us
I20260709 12:58:27.205194 18111 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:27.248679 13537 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:58:27.258186 18511 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:27.262425 18511 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.264798 13612 raft_consensus.cc:1275] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:58:27.294368 18602 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:27.298391 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:58:27.321271 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 51986.8us
I20260709 12:58:27.341403 18779 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:27.332875 18740 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:58:27.370193 18763 log.cc:927] Time spent T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 12:58:27.402803 18754 log.cc:927] Time spent T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:58:27.349189 18779 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:27.503700 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 222150us
I20260709 12:58:27.537184 18781 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:27.557499 18767 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.304s	user 0.001s	sys 0.001s
I20260709 12:58:27.542335 18511 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:27.578909 18779 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:27.575289 18781 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:27.551285 18501 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:27.590425 18781 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:27.591218 18794 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:27.593377 18801 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:27.598150 18801 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.607077 18800 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:27.591995 18794 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.607986 18800 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.610719 18794 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.611347 18800 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:27.620517 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:27.621487 13691 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:27.622797 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:27.634508 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:27.635185 14244 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:27.638705 17719 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:27.639256 17719 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.592543 18501 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:27.641070 14245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d73171142594388972ed22eae44f95f"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:27.642045 14245 raft_consensus.cc:606] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:27.642400 14245 raft_consensus.cc:614] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:27.642639 17719 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.631163 18794 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:27.644277 18794 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:27.644675 18794 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:27.647584 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:27.648059 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:27.650934 13508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.014s	sys 0.018s
W20260709 12:58:27.651845 13508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.014s	sys 0.019s
I20260709 12:58:27.654352 17719 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:27.655061 17719 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.632427 18782 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:27.666117 18801 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.667285 18793 consensus_queue.cc:1048] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:58:27.660557 18764 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:27.669593 18764 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:27.669955 18764 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:27.660102 18779 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:27.672545 18779 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.673220 18501 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.674814 18779 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:27.675966 18755 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.960s	user 0.008s	sys 0.000s
I20260709 12:58:27.677179 18782 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.679575 18782 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.680210 13610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56784 (request call id 2036) took 1201 ms. Trace:
I20260709 12:58:27.688481 13610 rpcz_store.cc:276] 0709 12:58:26.479227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:26.479347 (+   120us) service_pool.cc:224] Handling call
0709 12:58:26.479643 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:26.487622 (+  7979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:26.487651 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:26.488103 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:58:26.583486 (+ 95383us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b18003bfc80. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1
0709 12:58:26.583644 (+   158us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:58:26.583683 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:26.583756 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:27.675371 (+1091615us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 12:58:27.679989 (+  4618us) raft_consensus.cc:1685] finished
0709 12:58:27.680012 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:27.680153 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7657,"raft.queue_time_us":7973,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":28015,"spinlock_wait_cycles":41223680,"thread_start_us":45235,"threads_started":1,"wal-append.queue_time_us":104178}
I20260709 12:58:27.692775 14245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:27.693583 14245 raft_consensus.cc:606] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:27.695822 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:27.700759 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:27.701622 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:27.661311 18511 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:27.706056 17719 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.709466 18806 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:27.710122 18806 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:27.726784 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:27.730357 18794 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.732802 18794 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:27.739866 18751 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.428s	user 0.009s	sys 0.015s
I20260709 12:58:27.742009 14243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:27.742733 14243 raft_consensus.cc:606] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:27.743093 14243 raft_consensus.cc:614] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:27.727288 18511 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.749866 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:27.750495 13687 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:27.759368 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:27.760213 14237 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:27.752187 13690 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:27.764750 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:27.766805 13534 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:58:27.764950 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:27.770568 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:27.746372 18511 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:27.764497 18598 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:27.784572 18778 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:27.784997 18766 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:27.785300 18778 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:27.785687 18766 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.776221 18143 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:27.787530 18778 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.787305 18143 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:27.790831 18143 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.784605 18598 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.788580 18661 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:27.788880 18799 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:27.791607 18661 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:27.792224 18799 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:27.793900 18661 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.788064 18766 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:27.795152 18799 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.784667 18764 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.797652 18764 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:27.804315 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 196578us
I20260709 12:58:27.808743 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:27.809669 14243 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:27.772393 18781 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:27.818202 13691 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:27.832702 14243 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:27.840380 18806 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:27.821938 18781 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.813892 18382 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:27.937899 18746 log.cc:927] Time spent T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 1.234s	user 0.009s	sys 0.008s
W20260709 12:58:27.978583 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 212039us
W20260709 12:58:27.979436 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 208359us
W20260709 12:58:27.992398 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 183352us
I20260709 12:58:28.000794 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.002556 13614 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:28.012962 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.014300 13537 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:28.015762 17719 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.017194 17719 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.022436 17719 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.022617 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.023135 17719 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.027774 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:28.028228 13543 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 11.
I20260709 12:58:28.028852 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:28.029888 13689 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:28.030148 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.032682 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.033768 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.034461 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:28.034956 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.034020 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.035665 14243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:28.036541 14243 raft_consensus.cc:606] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:28.024217 17719 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.036755 13541 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:28.041711 14243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:38540
I20260709 12:58:28.051085 18111 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.046639 17774 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.052564 17774 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.051419 14243 raft_consensus.cc:606] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:28.055711 14243 raft_consensus.cc:614] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:28.049688 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.057962 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.058353 18111 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.057981 14237 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15.
I20260709 12:58:28.065778 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.066164 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:28.066774 13539 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:58:28.066807 13618 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:58:28.069396 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.069646 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.070484 13617 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:28.071023 13612 raft_consensus.cc:2410] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:58:28.074867 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.065932 17997 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:28.076395 17997 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.076737 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:28.077382 13537 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:58:28.077814 13541 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:28.079089 17997 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.081486 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.082257 18111 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.083284 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.084122 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.090400 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.091603 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.093291 13610 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:28.094636 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.095554 13619 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:28.096719 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.097743 13614 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:28.098876 18143 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.099695 18143 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.083097 13506 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:28.110344 17719 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.109424 14211 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:28.147759 18764 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.148772 18764 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.152057 18786 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:28.152740 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.159109 13687 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:28.185256 18511 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.185832 18799 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:28.186146 18511 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.188195 18649 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.189155 18649 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.211354 18143 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.196585 18528 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.212777 18528 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.194913 18786 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:28.224146 13618 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:28.215643 18528 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.260799 18779 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:28.260908 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 915343us
I20260709 12:58:28.262099 18771 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.261749 18779 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.267805 18766 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.268316 18779 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.268882 18766 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:28.271452 18766 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:58:28.272315 18693 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.273147 18693 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.275471 13619 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:58:28.275976 18653 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.262020 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.276762 18653 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.279037 17774 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.279686 18648 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.280501 18648 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.282500 18781 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.283015 18648 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.204582 18236 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:28.284384 18236 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:27.845463 18382 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.286903 18236 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.287055 18382 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.288076 18511 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:28.288448 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 205733us
I20260709 12:58:28.288808 18143 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:28.289295 18143 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.289769 18143 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:28.292573 18111 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:28.293431 18660 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.294091 18660 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.296062 18780 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.297544 18669 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.298408 18669 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.303754 18793 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:28.308883 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 93639.7us
W20260709 12:58:28.309264 18758 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:58:28.309856 18769 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.955s	user 0.007s	sys 0.010s
I20260709 12:58:28.310047 18783 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.313143 18598 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.314028 18598 raft_consensus.cc:993] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:28.314584 18598 raft_consensus.cc:1081] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:28.312166 14212 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:28.317068 18776 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:28.207868 18649 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.324800 18236 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.326440 18236 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.328228 18764 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.279234 18653 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:28.339452 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 77494.7us
I20260709 12:58:28.344866 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.310876 18783 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.304606 18793 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.351146 18793 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.356210 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.356547 13541 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:28.360133 18236 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.262883 18771 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.296855 18780 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.275720 18693 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.385979 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:28.384572 18805 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.643s	user 0.006s	sys 0.010s
I20260709 12:58:28.392915 13543 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:28.293453 18111 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.397652 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.405454 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.405483 13584 leader_election.cc:400] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15.
I20260709 12:58:28.406450 13584 leader_election.cc:403] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:28.406824 17719 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:28.418897 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 128749us
I20260709 12:58:28.421039 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.422730 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:28.424331 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 89490.7us
I20260709 12:58:28.425128 18648 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15.
I20260709 12:58:28.427750 13685 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:28.428416 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.417786 14243 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:28.413327 18783 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.438055 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.440825 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.444002 17719 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.445806 18669 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.446403 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.448956 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.450752 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.440279 18660 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.614372 18826 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.616859 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:28.617393 17719 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:28.585692 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 81296us
I20260709 12:58:28.458837 13659 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:28.435453 18794 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:28.457120 18774 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.588856 13541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:28.594537 18111 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.595048 18783 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.597206 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:28.600457 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 190454us
I20260709 12:58:28.600874 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:28.603761 13543 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:28.443296 18661 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:28.604880 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.605134 18780 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.645131 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.605445 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.605799 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.608858 18771 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.449445 18490 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.611470 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:28.611931 18793 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.613812 18782 raft_consensus.cc:993] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:28.616737 18826 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.620965 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.621682 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.621978 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.474421 18786 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.622779 13541 raft_consensus.cc:606] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:28.603566 18143 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.647778 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:28.651634 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 244862us
W20260709 12:58:28.654639 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 553936us
I20260709 12:58:28.655109 18661 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.651225 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.663465 18669 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:58:28.665444 18669 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.665177 18774 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.665922 18669 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:28.667565 18774 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.668084 13610 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:28.670990 18693 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:28.672250 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.679258 18681 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:58:28.660905 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 60277.7us
I20260709 12:58:28.714200 13539 raft_consensus.cc:2393] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 16: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:58:28.717752 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56001.8us
I20260709 12:58:28.661872 18793 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.712299 13619 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:58:28.753034 13618 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:28.763185 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.720681 18780 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.773252 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.755431 18794 raft_consensus.cc:1081] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:28.756053 18681 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:28.772888 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:28.776700 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 179688us
I20260709 12:58:28.778120 18528 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.778800 18666 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.756500 18490 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.779986 18653 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.785120 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.764740 18688 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:28.792057 18759 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:28.792207 18528 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.792994 18759 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.795279 18528 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.797499 14212 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:28.796572 18666 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.795611 18759 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.759757 18783 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.816149 18783 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.818087 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.818894 13619 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:28.820807 18688 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.821861 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.823618 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:28.824458 14245 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
W20260709 12:58:28.803546 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 91611.5us
I20260709 12:58:28.813073 18143 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:28.839579 13535 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
W20260709 12:58:28.841105 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 68056.4us
I20260709 12:58:28.850718 18653 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.854521 13658 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:28.856356 18693 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:28.856851 18693 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:28.857306 18693 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:28.850842 18825 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:28.814503 18780 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.800377 18681 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.893893 18669 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:28.870152 18825 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.896492 18669 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.896869 18825 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:28.900225 18759 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.901896 18653 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.901489 18759 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:28.922230 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.920377 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:28.951947 13617 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:28.959813 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:28.944904 18528 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:28.963056 18778 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.964125 18778 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:28.967170 18778 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.941358 18794 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:28.968170 18794 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.952800 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:28.914885 18688 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.968544 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.968564 18794 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:28.986472 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.988456 18793 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.989212 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.989806 18793 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:28.990226 18793 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:28.990577 18793 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:28.991824 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:28.940922 18826 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:28.993273 18666 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:28.990492 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.956478 18653 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:58:28.994797 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:28.993680 18780 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.013947 18111 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:58:29.015261 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 73729.3us
I20260709 12:58:29.015434 18111 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:29.021518 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.023715 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.027473 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.029865 13543 raft_consensus.cc:2410] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for term 18 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:58:29.028386 13542 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
I20260709 12:58:29.023307 18111 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:29.013726 18653 consensus_queue.cc:1243] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer d35f063cd3f942b0bf559b7c6e9b299f log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:58:29.034824 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:29.035499 13690 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:29.044932 13619 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:29.046736 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.018136 13615 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:39902
I20260709 12:58:29.048305 13615 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:29.048709 13615 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:29.030586 18778 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:29.057062 13611 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:39902
I20260709 12:58:29.057351 18778 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.059582 18778 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.063102 18490 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.064007 18759 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.057745 13611 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:29.070075 13611 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:29.047540 18653 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:29.077409 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:29.085322 18759 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.115346 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.074556 18490 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:58:29.118170 13611 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.117616 18669 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:29.131686 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.127419 18669 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.133173 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.134351 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.135133 18669 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.135468 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.141017 13690 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 16.
W20260709 12:58:29.141393 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 181641us
I20260709 12:58:29.098383 18653 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.145692 13611 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.145059 13659 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:29.147198 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:29.143965 13506 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:29.112789 18111 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.145480 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.156181 13685 raft_consensus.cc:2410] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:58:29.156553 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:29.157300 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.158043 13618 raft_consensus.cc:2393] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
W20260709 12:58:29.158571 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 304135us
I20260709 12:58:29.153903 18666 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:29.151042 18669 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:58:29.162061 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.163242 18759 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:58:29.167115 18807 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:29.167896 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:29.168476 14245 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:58:29.165251 18669 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:29.168699 14245 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:29.161810 18666 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:29.205940 13543 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:29.207573 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.208156 13618 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 17.
W20260709 12:58:29.209439 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 45346us
I20260709 12:58:29.203315 18669 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:29.212148 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:29.212944 18708 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:29.204047 18807 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:29.162917 18825 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.218068 18653 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.206169 18666 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:29.216012 18706 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:29.219822 18825 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.220254 18708 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.217437 18693 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.222648 18708 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.220451 18839 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:29.236967 13543 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 17.
I20260709 12:58:29.220182 18658 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:29.239214 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:29.239110 18658 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.240511 18660 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:29.241977 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.242537 13543 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:29.237306 13615 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.245028 18658 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.220314 18706 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.248587 18706 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.249303 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.253154 18826 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:29.253805 18826 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:29.254294 18826 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:29.225958 18681 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:29.265188 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.241093 18660 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:29.280525 18807 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.284732 13613 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:29.283155 18666 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.287853 18666 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.269188 18706 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:29.288091 18798 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:29.257710 18653 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:29.272884 13508 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:29.260852 18693 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.260143 18658 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:29.299041 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.258836 14245 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.309005 14245 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:29.310889 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:29.311575 14240 raft_consensus.cc:3060] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:58:29.312014 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:29.312623 14237 raft_consensus.cc:3055] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:29.312913 14237 raft_consensus.cc:740] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:29.313692 14237 consensus_queue.cc:260] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.314356 13615 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.314952 14237 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:29.318547 14245 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 17.
I20260709 12:58:29.322444 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.325515 13543 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:29.326839 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.327992 13584 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:29.316265 18111 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.329499 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:29.335073 13658 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.295966 18681 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.301803 18706 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.342619 18839 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:29.348220 18519 raft_consensus.cc:2764] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:58:29.343219 18677 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.346884 14240 raft_consensus.cc:2468] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:29.329576 18693 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:58:29.337958 18840 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:58:29.305325 18794 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.333653 18648 raft_consensus.cc:2804] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:29.348968 18839 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:29.302033 18798 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.355468 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.322955 18658 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.358721 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.361522 18786 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:29.361651 18706 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.361856 18661 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.364146 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.368044 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.353382 18693 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:58:29.368597 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 157064us
I20260709 12:58:29.374349 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.329769 18653 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.366143 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.374730 14245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.366261 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.378305 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.381031 14242 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:29.378381 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:29.381742 13688 raft_consensus.cc:2368] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15.
I20260709 12:58:29.382408 13542 raft_consensus.cc:3060] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:29.382740 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.386874 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.395731 14237 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:29.373066 18648 raft_consensus.cc:697] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:29.376611 18693 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:29.401943 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.402196 18786 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.407262 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.408056 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.408669 18786 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.409535 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.410041 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.411155 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.412127 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.412271 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.413269 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.414599 13686 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:29.416553 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:29.418376 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 643899us
I20260709 12:58:29.419019 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.419669 13693 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:29.407693 18681 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.423619 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.425747 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.427065 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.432749 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.434439 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.416364 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.419569 14211 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.440181 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.454706 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.455068 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.455665 13611 raft_consensus.cc:2410] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:58:29.461896 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.465725 13537 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:29.467139 13689 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:29.467367 13540 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:29.444918 18786 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:29.503176 13691 raft_consensus.cc:1240] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:29.377369 18660 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.459049 18681 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:29.470022 18839 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:29.516330 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 96830.4us
I20260709 12:58:29.508876 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.520356 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.525692 18658 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.534400 13542 raft_consensus.cc:2468] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:29.506986 18842 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:29.541301 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:29.513212 18681 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:58:29.546558 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 169681us
I20260709 12:58:29.548204 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.552485 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.544054 18681 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:29.551370 18793 consensus_meta.cc:287] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.001s
I20260709 12:58:29.564445 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.562636 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.562529 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.566461 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.566602 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.568045 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.532325 18839 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.588372 18798 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.548044 18143 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:29.589273 18143 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.570796 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.592836 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.571553 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.572553 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.573144 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.567831 18794 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.596614 13611 raft_consensus.cc:2410] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:58:29.566938 18658 raft_consensus.cc:2764] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:58:29.575189 13617 pending_rounds.cc:85] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 7
I20260709 12:58:29.597818 13617 pending_rounds.cc:107] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:58:29.598176 13617 raft_consensus.cc:2889] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:29.597941 18693 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.575208 18785 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:29.542099 18648 consensus_queue.cc:237] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.601558 13584 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:29.602465 13684 raft_consensus.cc:1240] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:58:29.603420 18661 raft_consensus.cc:2804] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:29.604192 18661 raft_consensus.cc:697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:29.604431 18669 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.605330 18661 consensus_queue.cc:237] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.607141 18669 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.608566 18693 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.576792 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.580490 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.584265 18842 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.584839 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.585202 18653 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.612694 18807 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:29.586565 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.587571 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.588132 13688 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:29.592991 13541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32778 (request call id 44) took 1004 ms. Trace:
W20260709 12:58:29.611294 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 142415us
W20260709 12:58:29.591802 18837 log.cc:927] Time spent T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.468s	user 0.001s	sys 0.003s
I20260709 12:58:29.545601 18786 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.624073 18786 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.627195 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.629171 13508 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:29.639469 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 62919.6us
I20260709 12:58:29.636261 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:29.645213 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 108697us
I20260709 12:58:29.646087 14211 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.648348 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.648458 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:29.650794 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:29.657280 14211 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.658871 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.616019 13541 rpcz_store.cc:276] 0709 12:58:28.588484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:28.588638 (+   154us) service_pool.cc:224] Handling call
0709 12:58:29.592567 (+1003929us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54000247a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e72f520 00007fa02e7ac652 00007fa03377a60a 00007fa02cd007d4 00007fa02cd00762 00007fa02cd00691 00007fa02cd0060c 00007fa02ccff8c2 00007fa02caed48c 00007fa02caec8c6
0709 12:58:29.592908 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63969536}
I20260709 12:58:29.665779 18786 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.671329 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.610832 18793 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.688982 18658 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:58:29.708786 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 60408.4us
I20260709 12:58:29.719326 18796 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:29.693372 18794 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:29.721671 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:29.765144 18528 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
W20260709 12:58:29.765513 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:29.729914 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 72717us
I20260709 12:58:29.701932 18586 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.693470 18786 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.760012 18851 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:29.763528 18793 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.763761 18842 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.764142 18839 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.767524 13535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49526 (request call id 1964) took 1162 ms. Trace:
I20260709 12:58:29.791949 13535 rpcz_store.cc:276] 0709 12:58:28.605065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:28.605180 (+   115us) service_pool.cc:224] Handling call
0709 12:58:29.767312 (+1162132us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540006e5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:29.767444 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427138304}
W20260709 12:58:29.774360 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 85359.6us
I20260709 12:58:29.793758 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.795433 18586 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:29.774874 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 59727.9us
I20260709 12:58:29.795562 18677 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:58:29.780525 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.798750 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.799609 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.800025 13658 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.800834 18793 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.801259 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.801347 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.790700 18851 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:29.802259 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:29.803033 18693 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:29.803591 18851 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:29.781208 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 99816.6us
I20260709 12:58:29.803800 18693 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:29.804279 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.758935 18796 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.803435 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.805820 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.806286 18681 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:29.786612 18826 consensus_meta.cc:287] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.006s
I20260709 12:58:29.807534 13658 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.807940 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.809156 18693 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:29.810424 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.811201 18830 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:29.813565 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:29.814471 18796 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.801491 18793 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:29.826987 18851 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:58:29.808344 18826 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.821821 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.798880 13537 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 16. Current term is 17. Ops: []
I20260709 12:58:29.820297 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.829686 14239 raft_consensus.cc:2393] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 11: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:29.831552 13690 raft_consensus.cc:1240] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:29.832542 13684 raft_consensus.cc:1240] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:29.765661 18528 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:29.843035 13692 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 7. Current term is 8. Ops: []
I20260709 12:58:29.844164 18777 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:58:29.848093 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.848594 13689 raft_consensus.cc:1240] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 9. Current term is 10. Ops: []
I20260709 12:58:29.845510 18586 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:29.857622 13685 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:29.862150 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.863713 18586 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:29.867451 18143 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.839538 18851 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:29.880234 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:29.880260 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 107589us
I20260709 12:58:29.889106 13509 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.894209 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.896940 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.898571 18708 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:29.899299 18708 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.898331 13538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.901270 13538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49540 (request call id 1830) took 1255 ms. Trace:
I20260709 12:58:29.901618 13538 rpcz_store.cc:276] 0709 12:58:28.645522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:28.677199 (+ 31677us) service_pool.cc:224] Handling call
0709 12:58:29.875224 (+1198025us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b5800002798. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 12:58:29.901211 (+ 25987us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82683264}
I20260709 12:58:29.902134 18528 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.903214 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.904644 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.902478 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.877199 18851 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:29.907094 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.912773 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.913554 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:29.910542 14212 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.909294 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.914433 13610 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:29.919796 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:29.921012 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:29.920773 14212 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.925691 13534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.929028 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.930980 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:29.932184 13692 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:29.934619 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.936853 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.937655 13614 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:29.937705 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.930110 13658 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.938145 13614 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:29.942085 13658 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.945968 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.944540 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:29.945569 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.950198 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:29.950416 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:29.948391 13508 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.952719 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.957515 14239 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:29.941212 18708 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:29.961007 18708 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.959136 18793 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.963264 18708 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:29.963335 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:58:29.966804 13614 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:29.967677 18864 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:29.941442 18528 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.969499 18586 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:29.970959 18586 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:29.973723 18681 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:29.975195 18826 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:29.975474 18111 consensus_queue.cc:1059] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:29.976310 18111 consensus_queue.cc:1243] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Peer ad2fdb1517c542439654bd3f7c72dad0 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:58:29.939695 18794 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:58:29.980268 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.981566 13536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49540 (request call id 1831) took 1329 ms. Trace:
I20260709 12:58:29.981995 13536 rpcz_store.cc:276] 0709 12:58:28.648347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:28.654290 (+  5943us) service_pool.cc:224] Handling call
0709 12:58:28.654799 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:58:28.655256 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:58:28.655292 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:58:29.633294 (+978002us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b2c000014c8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 12:58:29.641309 (+  8015us) log.cc:844] Serialized 26 byte log entry
0709 12:58:29.691627 (+ 50318us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:58:29.691666 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:58:29.691730 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:58:29.975290 (+283560us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b240014f498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b703fb0 00007fa02e37a488 00007fa02e37abe8 00007fa02e352398 00007fa02e34d76a 00007fa02e34cbb6
0709 12:58:29.977473 (+  2183us) raft_consensus.cc:1685] finished
0709 12:58:29.977495 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:58:29.977619 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39162,"raft.queue_time_us":1586,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":63816,"spinlock_wait_cycles":10341504,"thread_start_us":49890,"threads_started":1,"wal-append.queue_time_us":68996}
I20260709 12:58:29.984000 13535 raft_consensus.cc:1240] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:29.989828 18519 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:29.978477 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:29.949321 18848 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:29.980248 13582 leader_election.cc:400] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 14. Current term is 15.
W20260709 12:58:29.978502 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52167.8us
I20260709 12:58:29.994830 13658 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:29.980938 18653 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:58:30.000015 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.002386 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:30.003051 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:30.006197 13692 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:58:29.991724 18830 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:30.012342 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.015556 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.010166 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:29.998696 18826 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:30.016636 18780 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:30.028501 18653 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:30.026058 18841 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:30.014304 18830 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:30.031162 18830 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:30.033151 18842 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:30.045455 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.046998 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:30.048151 18842 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:58:30.049633 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:30.050427 13543 raft_consensus.cc:2393] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 14: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:30.045446 13614 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:30.056190 14239 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:30.059906 14241 raft_consensus.cc:1240] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:30.061898 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:30.062664 13689 raft_consensus.cc:2393] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:30.063808 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:30.066583 13582 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.067824 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.024979 18848 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:58:30.070803 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 123460us
I20260709 12:58:30.074811 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.078078 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.080240 14209 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.078120 14212 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.090996 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.091387 13689 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:30.095768 13506 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.102180 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.077023 18660 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:58:30.120077 13582 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:30.146257 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 52045.2us
W20260709 12:58:30.171192 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 91021.1us
I20260709 12:58:30.172395 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.172273 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.177532 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:30.177564 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.090291 18842 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.094489 18648 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:30.171867 13659 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.197645 13614 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:58:30.100425 18841 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:30.111069 18490 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:30.121151 18780 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:30.091054 18785 consensus_queue.cc:1059] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 12:58:30.183797 18693 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:30.059558 18830 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.108985 18681 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:30.090934 18677 consensus_queue.cc:1059] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 12:58:30.178130 18653 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:30.090804 18798 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:58:30.180045 18782 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.210608 18841 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.103233 18777 consensus_queue.cc:1059] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 12:58:30.201887 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 96375.3us
I20260709 12:58:30.225683 18881 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:58:30.262894 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 91127.1us
I20260709 12:58:30.221786 18842 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:58:30.290859 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 113410us
W20260709 12:58:30.301959 14210 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:30.302299 13614 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:58:30.302649 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 130326us
I20260709 12:58:30.303331 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.268878 18660 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:58:30.288605 18708 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:30.305729 13658 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.291661 18661 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:58:30.312947 18845 log.cc:927] Time spent T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:58:30.315019 18868 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:58:30.315845 18868 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.325150 18873 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:30.325836 18873 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.326452 18830 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:30.335379 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:30.286126 18681 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.306334 18708 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:30.334420 18777 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:58:30.385138 18777 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:30.380568 18708 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.330919 18873 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:30.389099 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 269097us
I20260709 12:58:30.305392 18693 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:58:30.346903 18793 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:58:30.330813 18851 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.393872 18688 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 12:58:30.410112 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 105834us
W20260709 12:58:30.413120 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 111209us
I20260709 12:58:30.389824 18796 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.394537 18143 raft_consensus.cc:3055] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:30.411950 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.416121 14212 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.414336 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.422367 13540 raft_consensus.cc:1240] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 17. Current term is 18. Ops: []
I20260709 12:58:30.421022 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.422056 18878 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.417634 18143 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:30.465291 18851 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:30.414390 18519 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:30.486238 18143 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.426118 18648 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:30.428707 18706 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:58:30.461022 18797 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:30.488602 14243 raft_consensus.cc:1240] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:30.473595 18681 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:30.477317 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 58494.1us
I20260709 12:58:30.492430 14241 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:30.479149 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:30.386246 18777 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:30.481633 18794 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:30.485131 18807 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:30.493685 18881 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:30.486420 18519 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.496580 18519 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:30.498414 18796 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
W20260709 12:58:30.489377 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 64891.1us
I20260709 12:58:30.498994 18796 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.501394 18796 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:30.503118 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:30.493815 18785 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:30.504979 18708 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:30.494283 18807 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.509341 18777 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:58:30.509375 18807 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:30.501408 13506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:30.504889 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:30.520543 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:30.413206 18830 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:30.527653 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:30.529765 13582 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.487685 18143 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:30.531241 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:30.540323 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:30.549017 13542 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:30.557701 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:30.564551 13539 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:58:30.556402 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.574767 13541 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:30.504980 18785 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:30.567795 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:30.523175 18774 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:30.587702 13582 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.536490 18648 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:30.589040 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:30.598048 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.598937 13684 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:30.562860 18884 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:58:30.605069 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 47640.6us
I20260709 12:58:30.605788 13508 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:30.608116 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:30.613090 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.614583 18850 raft_consensus.cc:993] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:30.615111 18850 raft_consensus.cc:1081] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:30.588166 18785 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:30.617039 18785 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.617444 18842 consensus_queue.cc:1059] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:30.618280 18785 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:30.619382 18841 consensus_queue.cc:1059] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:30.620182 18882 raft_consensus.cc:3055] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:58:30.620621 18882 raft_consensus.cc:740] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:30.572674 18865 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:30.629377 13617 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:30.549185 18780 consensus_queue.cc:1059] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:58:30.641649 18875 consensus_queue.cc:1059] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:58:30.642302 18865 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:30.640168 13438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41440 (request call id 210) took 11462 ms (client timeout 15000 ms). Trace:
I20260709 12:58:30.595364 18660 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:30.623077 18884 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:30.633245 18658 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.663532 14237 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:58:30.664043 18884 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:30.666887 18890 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:30.667995 18890 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:30.668401 18890 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:30.668864 18875 raft_consensus.cc:3055] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:30.669328 18875 raft_consensus.cc:740] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:30.670369 18875 consensus_queue.cc:260] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:30.658041 13438 rpcz_store.cc:269] 0709 12:58:19.231303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:19.232164 (+   861us) service_pool.cc:224] Handling call
0709 12:58:30.639906 (+11407742us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:26.377268 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:26.377927 (+   659us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:26.377965 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:58:26.377984 (+    19us) tablet.cc:662] Decoding operations
0709 12:58:26.379756 (+  1772us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:26.379814 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:26.379845 (+    31us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:58:26.482669 (+102824us) tablet.cc:701] Row locks acquired
0709 12:58:26.482692 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:58:26.482834 (+   142us) write_op.cc:270] Start()
0709 12:58:26.482955 (+   121us) write_op.cc:276] Timestamp: P: 1783601906482787 usec, L: 0
0709 12:58:26.482978 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:58:26.483558 (+   580us) log.cc:844] Serialized 8196 byte log entry
0709 12:58:26.831900 (+348342us) op_driver.cc:464] REPLICATION: finished
0709 12:58:27.592708 (+760808us) write_op.cc:301] APPLY: starting
0709 12:58:27.596563 (+  3855us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:27.744964 (+148401us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:27.745090 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:28.317877 (+572787us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:28.318440 (+   563us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 12:58:28.318506 (+    66us) write_op.cc:312] APPLY: finished
0709 12:58:28.319338 (+   832us) log.cc:844] Serialized 188 byte log entry
0709 12:58:28.433286 (+113948us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:28.612560 (+179274us) write_op.cc:454] Released schema lock
0709 12:58:28.612634 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:28.613357 (+   723us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:28.613391 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1356820,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":439762,"apply.run_cpu_time_us":39969,"apply.run_wall_time_us":1033686,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91823,"num_ops":21,"prepare.queue_time_us":258742,"prepare.run_cpu_time_us":12958,"prepare.run_wall_time_us":179691,"raft.queue_time_us":102519,"raft.run_cpu_time_us":23034,"raft.run_wall_time_us":426196,"replication_time_us":348696,"thread_start_us":727060,"threads_started":5,"wal-append.queue_time_us":211279,"wal-append.run_cpu_time_us":15016,"wal-append.run_wall_time_us":2720151}]]}
I20260709 12:58:30.621526 18882 consensus_queue.cc:260] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.685109 18771 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:58:30.687042 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 87442us
I20260709 12:58:30.688740 18660 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:30.689206 18660 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:30.689026 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.690196 18660 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.691622 18660 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:30.690755 18771 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:30.693356 18865 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:30.549422 18830 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.677090 18875 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:30.714018 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:30.715001 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 93132.7us
I20260709 12:58:30.731637 13582 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.728652 18851 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:30.737068 18851 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:30.737504 18851 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:30.738253 18890 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.741240 18830 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:30.717542 18882 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:30.748360 18890 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:30.765691 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 182491us
I20260709 12:58:30.765202 13582 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.773777 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:30.773931 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:30.778822 13659 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:30.786170 13659 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:30.766224 18771 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:30.767213 18890 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:30.787561 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:30.794651 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:30.726330 18648 consensus_queue.cc:1059] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:58:30.780965 18780 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:30.790408 18771 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:30.789224 18839 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:58:30.796938 18693 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:30.777331 18777 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:58:30.798237 18873 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:30.825609 18873 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.805701 18780 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:58:30.838022 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:30.841239 13617 raft_consensus.cc:1240] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:30.834544 18143 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:30.843384 18143 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.843658 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:30.808178 18771 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:30.847713 18693 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:30.848899 18693 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.852612 18490 raft_consensus.cc:3055] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:58:30.853080 18490 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:30.853955 18490 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.854483 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:30.855189 18490 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:30.856858 18885 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:30.860421 18885 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.862591 18885 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:30.869879 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:30.870561 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:30.832816 18780 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:30.873258 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:30.875591 13508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:30.809150 18884 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.885537 18884 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:30.893270 18793 raft_consensus.cc:3055] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:30.893776 18793 raft_consensus.cc:740] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:30.895026 18143 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:30.897177 18873 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:30.894654 18793 consensus_queue.cc:260] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.902696 18793 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:30.906213 18851 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:30.936116 18906 log.cc:927] Time spent T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.111s	user 0.001s	sys 0.003s
I20260709 12:58:30.938235 18774 raft_consensus.cc:3055] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:30.938678 18774 raft_consensus.cc:740] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:30.942482 18780 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:30.944996 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:30.947327 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:30.955688 13506 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:30.956187 13691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:30.960606 13691 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:30.960973 13691 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:30.957386 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:30.956182 13536 raft_consensus.cc:1240] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:30.957408 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:30.962817 14240 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 17.
I20260709 12:58:30.963246 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:30.964087 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 130121us
I20260709 12:58:30.965370 18111 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:30.965878 18111 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:30.939491 18774 consensus_queue.cc:260] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.002200 18111 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:31.003571 18851 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:31.018879 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:31.028107 18873 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:31.030722 18873 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:58:31.031306 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:30.968130 18865 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:31.049160 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:31.051816 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:31.044510 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:31.055059 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:31.058275 14240 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 15: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:31.032326 13508 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:31.062420 13582 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:31.063865 14241 raft_consensus.cc:1275] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:31.055884 13612 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:31.070497 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:31.004580 18774 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:58:31.077435 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 43995.4us
I20260709 12:58:31.070057 18872 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:31.090626 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:31.062489 18877 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:31.092943 18877 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.094694 18111 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:31.095566 13612 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:30.970065 18884 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:58:31.099836 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:31.107313 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:31.112636 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:31.115025 18111 consensus_queue.cc:1048] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:58:31.124891 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 92590.5us
I20260709 12:58:31.128331 18586 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:58:31.128808 18586 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:31.129155 18586 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:31.129248 13541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:31.129997 13541 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:31.132189 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:31.134912 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:31.136526 13536 raft_consensus.cc:1275] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:31.140432 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:31.141383 18661 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:31.141963 18661 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:31.142406 18661 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:31.132285 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:31.113619 18874 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:58:31.148118 13508 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:31.150550 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:31.152081 18877 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:31.152195 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:31.162803 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 114937us
I20260709 12:58:31.121308 18648 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:58:31.174214 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:31.168866 13691 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.177188 13692 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:31.198905 13509 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:31.200309 18708 consensus_queue.cc:1059] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:31.201396 18807 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:31.202132 18807 raft_consensus.cc:3055] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:31.202522 18807 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:31.202375 18894 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:58:31.203595 18894 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:31.203974 18894 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:31.204869 18894 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.207679 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:31.196445 18885 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:31.207885 18688 consensus_queue.cc:1059] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:31.170183 18865 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:31.182464 18872 consensus_queue.cc:1048] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:58:31.214514 13536 raft_consensus.cc:1240] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:31.214757 18780 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:31.217072 13509 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:31.217905 18899 log.cc:927] Time spent T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.389s	user 0.000s	sys 0.005s
I20260709 12:58:31.203466 18807 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.219265 18807 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:31.222543 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:31.224174 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:31.224776 13543 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:31.209925 18894 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:31.225804 13659 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:31.175998 18907 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:31.247434 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:31.251426 13691 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:31.262471 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:31.263257 14238 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:31.246675 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:31.232914 18708 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:58:31.272311 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 76885.5us
I20260709 12:58:31.276321 18850 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:58:31.277699 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:31.246179 18786 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:31.292167 13619 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:31.222076 18586 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.322259 14238 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
W20260709 12:58:31.326444 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 50706us
I20260709 12:58:31.241516 18848 consensus_queue.cc:1059] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:58:31.342487 13658 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:31.344192 13535 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:58:31.296242 18850 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:31.360613 13543 raft_consensus.cc:2468] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:31.352521 18793 raft_consensus.cc:2804] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:31.365242 18793 raft_consensus.cc:697] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:31.366690 18793 consensus_queue.cc:237] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.362730 13584 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:31.370380 18890 raft_consensus.cc:2804] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:31.371155 18890 raft_consensus.cc:697] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:31.372933 18890 consensus_queue.cc:237] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.409936 18586 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:31.413905 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:31.421510 18528 raft_consensus.cc:3055] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:31.422130 18528 raft_consensus.cc:740] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:31.428640 18528 consensus_queue.cc:260] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.332373 18661 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:31.398166 18771 consensus_meta.cc:287] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.000s
I20260709 12:58:31.449389 18528 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:31.450532 18839 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:31.462634 13509 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:31.462540 18839 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.470158 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:31.455526 18774 consensus_queue.cc:1059] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:58:31.473706 18839 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:31.479499 18661 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:31.482182 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:31.483518 14239 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:31.488198 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:31.485478 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:31.489934 14239 raft_consensus.cc:2393] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 15: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:31.490239 13619 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:31.491722 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:31.471159 18830 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:31.493693 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:31.494311 18830 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:31.498147 18830 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:31.468024 18771 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.501928 18771 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:31.502775 13615 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:31.503872 18839 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:31.508625 18839 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:31.509109 18839 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:31.515695 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:31.524247 13536 raft_consensus.cc:3055] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:31.524724 13536 raft_consensus.cc:740] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:31.525491 13536 consensus_queue.cc:260] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.526676 13536 raft_consensus.cc:3060] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:31.536152 13542 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:31.535643 14239 raft_consensus.cc:1240] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13. Ops: []
I20260709 12:58:31.546995 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:31.554894 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:31.563138 18786 consensus_queue.cc:1059] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:31.534307 13437 catalog_manager.cc:5697] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 7 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 12 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.565099 18786 raft_consensus.cc:3055] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:31.564015 18807 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:58:31.565578 18786 raft_consensus.cc:740] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:31.566448 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:31.566473 18786 consensus_queue.cc:260] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.577116 18786 raft_consensus.cc:3060] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:31.583387 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:31.585213 13537 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:31.586019 18798 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:31.587402 18586 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:31.596304 18798 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:31.597169 18798 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:31.572078 13437 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 11 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.602717 13437 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 9 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.611812 18830 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.628558 18888 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:31.644544 13437 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 9 to 13, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 13 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.645262 18888 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.646343 13437 catalog_manager.cc:5697] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 9 to 13, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.618178 18519 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:31.648336 18519 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:31.648401 18888 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:31.649377 18519 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.650177 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:31.650851 13693 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:31.651062 18519 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:31.652132 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:31.636631 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:31.653647 18807 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:31.654091 18807 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:31.654289 18885 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:58:31.654475 18807 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:31.654924 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:31.656221 18885 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:31.656749 18885 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:31.657676 18885 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.639269 18839 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.640415 13616 raft_consensus.cc:1240] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:31.668247 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:31.679150 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:31.684115 13535 raft_consensus.cc:1240] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 11. Current term is 12. Ops: []
I20260709 12:58:31.678864 13536 raft_consensus.cc:2468] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
W20260709 12:58:31.701632 18490 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:58:31.647706 13437 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.715674 18894 consensus_queue.cc:1059] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:31.715977 18780 raft_consensus.cc:3055] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:58:31.716481 18780 raft_consensus.cc:740] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:31.717362 18782 raft_consensus.cc:3055] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:31.717906 18782 raft_consensus.cc:740] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:31.717720 18780 consensus_queue.cc:260] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.718991 18782 consensus_queue.cc:260] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.721390 18782 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:31.723531 13437 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:31.725347 13437 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.719401 18839 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:31.733867 18930 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:58:31.737809 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:31.742909 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:31.743722 14239 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:31.742952 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:31.733363 18798 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.760370 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:31.761219 13616 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:31.766047 18780 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:31.727414 13437 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.765488 14239 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:31.775730 13536 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:31.778426 13535 raft_consensus.cc:1240] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:58:31.737695 18830 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:31.785386 18890 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:31.780220 13437 catalog_manager.cc:5697] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 6 to 9, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 9 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.790066 13437 catalog_manager.cc:5697] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 7 to 10, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:31.791745 13437 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 6 to 8, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 8 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.798831 13616 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:31.800082 13659 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:31.802186 18839 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:31.803953 18839 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:31.805176 18839 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.793473 13437 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.809614 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:31.809053 18798 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:31.811106 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:31.821666 13693 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:31.830816 13615 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:31.832472 13509 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:31.833880 18519 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:31.820521 13437 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 10 to 13, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.840443 18794 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:31.856279 18794 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.823683 18807 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.859395 18794 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:31.862262 13437 catalog_manager.cc:5697] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 6 to 10, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.871687 14241 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 12.
I20260709 12:58:31.872169 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:31.872673 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:31.872318 13437 catalog_manager.cc:5697] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 9 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.873291 13536 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:31.874687 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:31.896312 13437 catalog_manager.cc:5697] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 11 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.870039 18519 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:31.899657 13508 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:31.894507 18894 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:31.902326 13437 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.913417 13437 catalog_manager.cc:5697] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 8 to 10, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.909469 18894 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:31.909158 18798 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:31.927524 13535 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:58:31.929344 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:31.915285 18528 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:31.931294 13437 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:31.912535 18786 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:31.927282 18894 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:31.935959 13437 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 8 to 10, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.933646 18528 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.941632 18798 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:31.949927 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:31.950659 13437 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 11 to 13, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 13 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.951867 13693 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:31.936717 18882 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:58:31.956445 13437 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:31.942690 18786 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.950129 18807 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:31.944778 18519 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:31.970059 18786 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:31.974462 18798 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:31.975337 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:31.976051 13543 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:31.976696 18528 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:31.959936 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:31.979825 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:31.984031 18528 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:31.984442 18528 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:31.984789 18528 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.023165 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.026757 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.027782 13615 raft_consensus.cc:2410] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:58:32.034757 18797 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.035302 18797 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.036007 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.037448 18797 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.039880 18793 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:32.046978 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.009639 18894 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.057603 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.032397 18886 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.056424 14241 raft_consensus.cc:1275] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:58:32.031675 18850 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:32.092319 18850 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.096928 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.097425 18894 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.098709 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:32.099288 13543 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.105502 18807 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.105715 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:32.106076 18807 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:32.067291 18886 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.068176 18786 consensus_queue.cc:1048] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:58:32.121026 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.121708 13688 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 10.
I20260709 12:58:32.121955 13535 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:32.123036 13506 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:32.111322 18528 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.126541 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:32.130697 13693 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:32.134581 14240 raft_consensus.cc:2368] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:32.135691 13543 raft_consensus.cc:2468] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:58:32.136226 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:32.136993 13692 raft_consensus.cc:1275] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:58:32.137387 13692 pending_rounds.cc:85] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Aborting all ops after (but not including) 8
I20260709 12:58:32.137704 13692 pending_rounds.cc:107] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:58:32.136961 14239 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:32.137882 13692 raft_consensus.cc:2889] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:32.140465 13506 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.066414 18780 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:32.138737 13508 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:32.153137 14211 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.121562 18850 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:32.126703 18886 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:32.158957 13508 leader_election.cc:400] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:32.160212 13508 leader_election.cc:403] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:32.160022 18798 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:32.166147 18798 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:32.166508 18798 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.152679 18793 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.154578 18888 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:32.154688 18875 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:32.163658 18782 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:32.182546 14211 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.161110 18807 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.155098 18780 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.161993 18771 consensus_queue.cc:1048] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:58:32.190271 18894 raft_consensus.cc:2804] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:32.178365 18888 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:32.207718 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.211115 13615 raft_consensus.cc:2368] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:32.181025 18782 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:32.177770 18850 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:58:32.191995 18875 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:32.173069 18798 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.178617 18793 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.240751 18798 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:32.240978 18793 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.220635 18528 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.216398 18782 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.232596 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:32.248028 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.248273 13543 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.210863 18888 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:32.250315 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:32.251130 18850 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527) with higher term. Message: Invalid argument: T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:32.228231 18875 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.235198 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:32.228396 13537 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:32.230386 18939 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:32.237362 18780 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.216578 18894 raft_consensus.cc:697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:32.258286 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.257318 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.266357 18660 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.266079 18885 consensus_queue.cc:1059] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:32.267760 18660 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.269126 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.269750 13612 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 9.
I20260709 12:58:32.270977 18894 consensus_queue.cc:237] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.272701 13537 raft_consensus.cc:2393] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 18: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:32.274606 13543 raft_consensus.cc:2468] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:32.278821 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:32.266351 18939 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.289291 18939 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:32.291960 18780 raft_consensus.cc:3055] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:32.292488 18780 raft_consensus.cc:740] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:32.293406 18780 consensus_queue.cc:260] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.294695 18780 raft_consensus.cc:3060] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.298712 18782 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.300083 18939 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.300734 18939 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.301167 14239 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 16: Already voted for candidate 0f34cda059124f839711dbef61bb2c28 in this term.
I20260709 12:58:32.304406 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.305725 13612 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 12: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:32.304782 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.272071 18660 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.303900 18939 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.306381 13616 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:32.336372 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:32.340409 18660 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.341325 18660 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.342264 18888 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:32.343384 18873 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.343982 18873 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.345006 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.346052 18873 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.345975 13537 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:58:32.345005 18660 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.350014 18939 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:58:32.350732 18939 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:32.351115 18939 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:58:32.352259 18660 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.353829 18660 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:32.323442 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 49778.7us
I20260709 12:58:32.358058 13509 leader_election.cc:400] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 for earlier term 12. Current term is 13.
I20260709 12:58:32.356181 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:32.359824 18890 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:32.360253 18890 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.360620 18890 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.366513 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.367409 18886 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:58:32.368278 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.371531 18782 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.373281 18890 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.373543 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:32.374362 13612 raft_consensus.cc:3060] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.377179 18888 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.377465 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:32.378259 13543 raft_consensus.cc:2393] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 16: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:32.348239 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.388873 13537 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
I20260709 12:58:32.389726 14241 raft_consensus.cc:1240] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16. Ops: []
I20260709 12:58:32.392737 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:32.393420 13692 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:32.398175 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:32.401116 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:32.401822 13616 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:58:32.403376 13692 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:32.405321 13506 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:32.407358 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:32.408592 18850 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:32.409147 18850 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:58:32.409657 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 178084us
I20260709 12:58:32.410714 18850 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:32.424340 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.424996 13536 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 18.
I20260709 12:58:32.411116 18660 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.426013 13658 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.411762 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.418596 18873 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:58:32.427811 18873 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.428310 18873 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:58:32.420282 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.421838 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:32.431017 13535 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:58:32.431103 14241 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:32.420044 18890 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.422927 13612 raft_consensus.cc:2468] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:58:32.434475 13693 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:32.435452 13537 raft_consensus.cc:2368] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16.
I20260709 12:58:32.436218 14212 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:32.437462 18894 raft_consensus.cc:2804] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:32.438021 18894 raft_consensus.cc:697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:32.438949 18894 consensus_queue.cc:237] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.442699 18886 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:32.443152 13584 leader_election.cc:400] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16.
I20260709 12:58:32.443522 13584 leader_election.cc:403] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:32.444728 18660 raft_consensus.cc:3060] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.446595 13616 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 11.
I20260709 12:58:32.447399 18942 raft_consensus.cc:2804] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:32.454981 18942 raft_consensus.cc:697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:32.455668 18786 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.456332 18786 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.456301 18942 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.457688 18528 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.458227 18528 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.459991 14241 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:32.460417 18528 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.461889 18660 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 15. Current term is 16.
I20260709 12:58:32.470794 18786 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.461970 18886 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:58:32.450191 18873 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.453737 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.484011 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.462121 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.464615 13543 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:58:32.488456 13616 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 16: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:32.486573 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.497319 13685 raft_consensus.cc:2368] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:32.497300 13584 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:32.503857 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.504926 13612 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 15: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:32.506765 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.509147 13684 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 16: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:32.512295 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.520313 13612 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:32.521083 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.521901 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.553187 13612 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:32.555764 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.557230 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:32.557941 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.495054 18850 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.505473 18528 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:58:32.562736 13689 raft_consensus.cc:2393] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 16: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:32.556561 13537 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:32.520115 18660 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:32.565680 13535 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:32.520402 18885 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:32.497745 18939 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.559183 18955 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:58:32.563711 18528 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:32.591991 13692 raft_consensus.cc:2368] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 18. Current term is 19.
W20260709 12:58:32.620342 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 57935.3us
I20260709 12:58:32.564345 18850 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.622175 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.598018 18528 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:32.596146 18955 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:32.585978 18885 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.645780 18885 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.640166 18955 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:32.649863 13543 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:32.652830 18939 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.653678 18842 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.654265 18842 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.663688 18873 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.682108 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:32.682145 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.687114 13537 raft_consensus.cc:2393] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:32.687788 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:32.688439 13616 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:58:32.687107 18873 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.692715 18873 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.694165 18842 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.695511 18873 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.710708 13616 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 19.
I20260709 12:58:32.700387 18850 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:32.708257 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.725080 13692 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:32.706241 13691 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:32.695597 18842 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:32.733693 18842 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.734153 18842 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.740427 13506 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:32.744069 18807 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:32.744642 18807 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:32.744817 18490 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:32.744911 13659 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:32.746582 18939 raft_consensus.cc:2804] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:58:32.747149 18939 raft_consensus.cc:697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:32.748112 18939 consensus_queue.cc:237] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.745106 18807 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:32.757483 18955 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.771310 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.804534 13612 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:32.773471 13535 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
W20260709 12:58:32.776248 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 228808us
W20260709 12:58:32.777287 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 228466us
I20260709 12:58:32.781893 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.759285 18807 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.814535 13584 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:32.806093 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:32.763188 18528 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:32.832629 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 210523us
I20260709 12:58:32.845700 13582 leader_election.cc:400] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:32.846153 13582 leader_election.cc:403] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:32.851033 13543 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 16: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:32.859392 13693 raft_consensus.cc:2368] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9.
I20260709 12:58:32.865170 18660 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:32.849949 18956 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:32.870759 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:32.878324 13616 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 15.
I20260709 12:58:32.861392 18889 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:58:32.854919 18958 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:58:32.877290 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:32.874323 13612 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:32.889037 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:32.889741 13535 raft_consensus.cc:3060] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:58:32.890105 18782 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:32.890739 18782 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.892797 18660 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.895412 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:32.906332 13537 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:32.913059 13689 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:58:32.878077 18956 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:32.915468 18956 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:32.896483 18942 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:32.916458 18942 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:32.916944 18942 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:32.890019 18962 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
W20260709 12:58:32.923432 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 69224.6us
I20260709 12:58:32.904508 13616 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:58:32.922657 18528 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:32.929171 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:32.933468 13691 raft_consensus.cc:2368] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9.
I20260709 12:58:32.933662 13582 leader_election.cc:400] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 18. Current term is 19.
I20260709 12:58:32.934036 13582 leader_election.cc:403] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:32.931299 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.936972 18958 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:32.937271 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:32.938064 18958 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.942405 18842 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:32.901731 18786 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:32.946822 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 89759.2us
I20260709 12:58:32.868170 18886 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:58:32.901126 18793 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:32.950412 13582 leader_election.cc:400] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9.
I20260709 12:58:32.964672 14239 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:32.966040 13615 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:58:32.966492 13615 pending_rounds.cc:85] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 11
I20260709 12:58:32.966735 13615 pending_rounds.cc:107] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:58:32.967005 13615 raft_consensus.cc:2889] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:32.971899 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:32.972563 13537 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:32.974189 13535 raft_consensus.cc:2468] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 19.
I20260709 12:58:32.946269 18778 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 18. Current term is 19.
I20260709 12:58:32.964792 18894 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:33.012665 13537 raft_consensus.cc:2468] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
I20260709 12:58:32.956542 18786 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:32.969388 18793 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.044648 13658 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:33.047030 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:33.060662 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 82193us
I20260709 12:58:33.023192 18957 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:58:33.026486 18894 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.088424 18894 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:33.074607 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:33.090372 13692 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:33.078940 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:33.093811 14238 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:33.094830 13535 raft_consensus.cc:1275] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:58:33.095136 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:33.105283 18957 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:33.106021 18957 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:33.106535 18957 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:33.107996 18793 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:33.037933 18956 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:33.074172 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 111223us
W20260709 12:58:33.112794 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 46478.8us
I20260709 12:58:33.116353 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:33.074992 18968 log.cc:927] Time spent T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:58:33.123232 13610 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:58:33.125723 13689 raft_consensus.cc:2368] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:33.078771 13537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:33.078948 18977 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:33.080205 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:33.082581 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:33.043985 18782 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:33.061657 18955 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:33.046213 18807 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:33.076937 18786 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:33.142642 18586 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:33.144354 13540 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:58:33.126847 13537 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:33.070008 14241 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:33.133432 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:33.185598 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52255.2us
I20260709 12:58:33.186112 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:33.064509 18963 raft_consensus.cc:2804] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:58:33.067042 18842 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:33.135773 18977 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.145711 18889 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:33.111423 18956 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:33.147078 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:33.148170 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:33.198639 13536 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:33.149088 18942 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.150588 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:33.151134 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:33.206458 13616 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 16: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:33.152211 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:33.207490 13612 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 12: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:33.154923 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:33.208211 13543 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:33.157284 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:33.161815 18980 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:33.164013 13542 raft_consensus.cc:1275] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:58:33.125028 18974 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:58:33.159806 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:33.171010 18962 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:33.171962 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:33.172389 13582 leader_election.cc:400] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9.
I20260709 12:58:33.195384 18794 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:33.201061 13658 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:33.196511 18528 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:33.211290 13619 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:33.236742 13535 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 18: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:33.238449 13687 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:33.197822 18963 raft_consensus.cc:697] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:33.223255 13539 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
W20260709 12:58:33.228757 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 56453.3us
I20260709 12:58:33.229808 13611 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:33.230388 13615 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 18.
W20260709 12:58:33.188988 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54441.9us
I20260709 12:58:33.209930 13613 raft_consensus.cc:3060] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:58:33.235414 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 109175us
I20260709 12:58:33.257434 13582 leader_election.cc:400] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:33.257827 13582 leader_election.cc:403] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:58:33.249188 18798 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:33.259305 18798 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.260697 13658 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:33.261756 18798 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:33.263065 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:33.266469 18842 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:33.267217 18842 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:33.238094 18964 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:58:33.238718 18956 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:33.270233 18956 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:33.270784 18942 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:33.271313 18956 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:33.227234 18894 consensus_queue.cc:1048] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:58:33.278347 13543 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:33.236140 18794 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:33.232712 18782 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:58:33.293781 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:33.246529 18528 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:33.305394 13542 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 18.
I20260709 12:58:33.305255 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:33.310367 13539 raft_consensus.cc:2368] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16.
I20260709 12:58:33.316689 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:33.329177 13611 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:33.329016 13610 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:58:33.318797 18972 consensus_queue.cc:1048] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:58:33.290349 18794 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:33.336877 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:33.339252 18963 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.343159 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:33.349591 13613 raft_consensus.cc:2468] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:58:33.368983 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:33.369706 14238 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 17.
I20260709 12:58:33.372239 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:33.375460 13611 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 16.
I20260709 12:58:33.381333 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:33.353866 18977 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:33.353645 18890 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755) with higher term. Message: Invalid argument: T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f for earlier term 13. Current term is 14.
I20260709 12:58:33.363080 18660 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:58:33.406373 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 179345us
W20260709 12:58:33.405694 18986 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:58:33.421000 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:33.421777 13611 raft_consensus.cc:2368] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16.
I20260709 12:58:33.388691 18967 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:58:33.386574 18519 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:58:33.450644 13659 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:33.362022 18842 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.459213 13509 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:33.432374 18980 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:58:33.460314 13659 leader_election.cc:400] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16.
I20260709 12:58:33.463169 13659 leader_election.cc:403] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:58:33.441757 18967 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:33.417081 18660 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.464404 18955 raft_consensus.cc:2804] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:33.466185 18980 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:33.473839 18839 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:33.474519 18967 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:58:33.442688 18975 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:58:33.482010 18974 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:33.461033 18882 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 12:58:33.521639 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 62468.1us
I20260709 12:58:33.449743 18528 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.492239 18889 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:33.498966 18980 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:33.528331 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dd04e83486049ae859032792f652869"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:33.529991 13539 raft_consensus.cc:606] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:33.545310 18660 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:33.531332 18889 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:33.571400 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 99005.2us
W20260709 12:58:33.574576 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 86878.8us
I20260709 12:58:33.526034 18974 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:33.582962 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:33.590132 13539 raft_consensus.cc:606] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:33.590574 13539 raft_consensus.cc:614] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:58:33.594691 13582 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:33.631856 18958 raft_consensus.cc:2764] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:58:33.632196 13613 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:58:33.633235 18992 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:58:33.615123 18956 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.635507 18956 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:33.635396 18974 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:33.633191 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54853.2us
I20260709 12:58:33.646023 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:33.649089 18962 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:33.697598 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:33.710618 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 57649.7us
I20260709 12:58:33.755388 18955 raft_consensus.cc:697] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:33.757416 13690 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:33.759286 18955 consensus_queue.cc:237] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.766999 13539 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:58:33.774241 13658 leader_election.cc:400] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Vote denied by peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209) with higher term. Message: Invalid argument: T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16.
I20260709 12:58:33.794421 18889 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:33.798030 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:33.798656 13539 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:58:33.798825 18957 consensus_meta.cc:287] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: Time spent flushing consensus metadata: real 0.692s	user 0.005s	sys 0.006s
I20260709 12:58:33.800468 18961 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:33.806893 18794 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.810570 18958 consensus_queue.cc:1059] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:58:33.811571 18958 consensus_queue.cc:1243] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:58:33.812170 18958 raft_consensus.cc:3055] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:58:33.812541 18958 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:33.813344 18958 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.814503 18958 raft_consensus.cc:3060] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:33.822727 18839 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16.
W20260709 12:58:33.810460 19011 log.cc:927] Time spent T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:58:33.835579 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:33.836107 13691 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 16. Current term is 17. Ops: []
I20260709 12:58:33.862057 18961 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:33.834837 13539 raft_consensus.cc:2468] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:33.835489 13543 raft_consensus.cc:1240] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 8. Current term is 9. Ops: []
I20260709 12:58:33.838182 14242 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:33.843816 18794 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:33.848608 18999 log.cc:927] Time spent T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:58:33.871747 13543 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:58:33.870422 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:33.875945 13658 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:33.837076 18886 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:33.837996 18957 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.866155 18778 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:58:33.892519 18962 raft_consensus.cc:2804] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:33.861792 18967 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.862571 18961 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:33.866110 18980 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.940603 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:33.941313 14237 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 12.
I20260709 12:58:33.941936 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:33.942713 13542 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:33.949306 13692 raft_consensus.cc:2393] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 14: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:33.915570 18886 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:33.960402 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:33.946705 18962 raft_consensus.cc:697] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:33.966639 13536 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:58:33.971645 18961 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:33.973846 18957 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:33.975828 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:33.985877 13542 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:58:33.977852 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:33.980129 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:33.987326 13540 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:33.999364 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:33.951225 18977 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:58:34.008175 18961 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:58:34.069314 19019 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:58:34.070580 19016 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:58:34.077098 13612 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:58:34.077723 13612 pending_rounds.cc:85] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 5
I20260709 12:58:34.078074 13612 pending_rounds.cc:107] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:58:34.078415 13612 raft_consensus.cc:2889] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:34.008145 18957 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:58:34.072273 18961 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:34.093312 13540 raft_consensus.cc:2468] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
W20260709 12:58:34.092111 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 143395us
I20260709 12:58:34.095217 13658 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:34.105522 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:34.103489 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:34.105026 19014 raft_consensus.cc:2804] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:34.117168 19014 raft_consensus.cc:697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:34.118731 19014 consensus_queue.cc:237] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:34.120167 18981 log.cc:927] Time spent T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:58:34.094827 18961 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:34.130779 18980 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:34.132148 18972 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:58:34.137744 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:34.131616 18962 consensus_queue.cc:237] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:34.138684 13616 raft_consensus.cc:2368] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 18. Current term is 19.
I20260709 12:58:34.134308 18886 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:34.153573 14212 leader_election.cc:400] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 election: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 18. Current term is 19.
I20260709 12:58:34.154117 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:34.156494 14212 leader_election.cc:403] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:58:34.157757 13543 raft_consensus.cc:2393] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 18: Already voted for candidate ad2fdb1517c542439654bd3f7c72dad0 in this term.
I20260709 12:58:34.163714 19025 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:58:34.172231 13543 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:58:34.175079 18957 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:34.176752 18967 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:34.197378 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 151584us
I20260709 12:58:34.198477 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:34.214443 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:34.209278 19005 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:34.217391 13438 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:34.219528 19005 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:34.235968 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:34.238373 13536 raft_consensus.cc:3060] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:58:34.244408 13615 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:34.239696 19005 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:34.255744 19027 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:34.257776 18962 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:58:34.270455 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 271332us
I20260709 12:58:34.274035 18942 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:58:34.290729 13536 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 19.
I20260709 12:58:34.313917 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:34.315559 13584 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:34.342792 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:34.343632 13613 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:58:34.256264 18961 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:34.342640 18890 raft_consensus.cc:2804] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:58:34.363569 19034 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:58:34.418928 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 103449us
I20260709 12:58:34.421834 13690 raft_consensus.cc:1240] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:58:34.382895 19025 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367) with higher term. Message: Invalid argument: T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 for earlier term 18. Current term is 19.
I20260709 12:58:34.366217 19005 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:34.478279 14242 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59164 (request call id 1489) took 1152 ms. Trace:
I20260709 12:58:34.478729 14242 rpcz_store.cc:276] 0709 12:58:33.325879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:33.330119 (+  4240us) service_pool.cc:224] Handling call
0709 12:58:34.478070 (+1147951us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54005f0fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 12:58:34.478179 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135147904}
I20260709 12:58:34.479357 13616 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 12. Current term is 13. Ops: []
W20260709 12:58:34.482894 19029 log.cc:927] Time spent T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:58:34.486976 14240 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:58:34.488742 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:34.490648 18967 consensus_queue.cc:1059] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:34.491452 18967 consensus_queue.cc:1243] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:58:34.497488 18997 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:34.498083 18997 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:34.453913 18890 raft_consensus.cc:697] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:34.500881 18890 consensus_queue.cc:237] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:34.506933 19036 log.cc:927] Time spent T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:58:34.518813 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:34.526484 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 156416us
W20260709 12:58:34.535116 19038 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 12:58:34.529491 19008 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:58:34.530133 19033 log.cc:927] Time spent T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.201s	user 0.000s	sys 0.003s
I20260709 12:58:34.557257 14242 raft_consensus.cc:2468] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 19.
I20260709 12:58:34.533636 19005 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:34.584867 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 174415us
I20260709 12:58:34.602141 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:34.602996 13614 raft_consensus.cc:1240] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 16. Current term is 17. Ops: []
I20260709 12:58:34.607024 13508 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:58:34.608099 18938 raft_consensus.cc:2764] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:58:34.613867 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:34.656176 14238 raft_consensus.cc:3060] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:34.662034 13542 raft_consensus.cc:1275] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:58:34.667992 18961 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:34.650758 18959 raft_consensus.cc:3055] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:58:34.668988 18959 raft_consensus.cc:740] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:34.669981 18959 consensus_queue.cc:260] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:34.671413 18959 raft_consensus.cc:3060] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:34.676050 18850 raft_consensus.cc:993] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:34.676856 18850 raft_consensus.cc:1081] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:34.679945 18997 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:34.663998 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:34.685691 14238 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:58:34.688700 13438 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 8 to 15, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 15 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.701557 13536 raft_consensus.cc:1275] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:58:34.703125 13543 raft_consensus.cc:1275] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:58:34.704640 13535 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:58:34.708662 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:58:34.709944 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 53271.8us
I20260709 12:58:34.727206 13438 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 12, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.728973 13438 catalog_manager.cc:5697] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.729651 18938 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:34.730522 13438 catalog_manager.cc:5697] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.742504 19008 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:34.743126 18942 consensus_queue.cc:1048] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:34.741735 18962 consensus_queue.cc:1048] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 12:58:34.729807 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 65854us
I20260709 12:58:34.747994 18963 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:34.748996 18998 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:34.763448 13438 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 13 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.765754 13438 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 17, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 17 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.769223 13615 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dd04e83486049ae859032792f652869"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:58:34.775611 13615 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:34.776049 13615 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:34.775940 13438 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 12 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.780222 13438 catalog_manager.cc:5697] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 14, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 14 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:34.784098 13438 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.787649 13438 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.768541 13584 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:34.797837 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:34.798619 13539 raft_consensus.cc:606] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:34.791535 13438 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 13 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.801425 13438 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 10 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.802992 13438 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 9 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.804841 13438 catalog_manager.cc:5697] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:34.806738 13438 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:34.816489 13438 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.817756 13615 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:34.821776 13438 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:34.836863 13615 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:34.837941 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:34.838572 13691 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:34.816012 19010 consensus_queue.cc:1048] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:58:34.821486 13584 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:34.810370 18889 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:58:34.881564 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:34.885799 13539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:34.886627 13539 raft_consensus.cc:606] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:34.889870 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:34.892004 13690 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:58:34.866106 18997 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:58:34.898170 13437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49880 (request call id 330) took 21909 ms (client timeout 15000 ms). Trace:
I20260709 12:58:34.907488 13691 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:34.917836 13582 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:34.963195 18939 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:34.964560 13614 raft_consensus.cc:1275] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:58:34.964722 18939 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:34.964974 13614 pending_rounds.cc:85] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Aborting all ops after (but not including) 9
I20260709 12:58:34.965107 18939 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:34.965309 13614 pending_rounds.cc:107] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:58:34.965574 13614 raft_consensus.cc:2889] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:58:34.966003 18939 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:34.898581 13437 rpcz_store.cc:269] 0709 12:58:12.975672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:13.014015 (+ 38343us) service_pool.cc:224] Handling call
0709 12:58:34.884951 (+21870936us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:32.219523 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:32.220194 (+   671us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:32.220229 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:58:32.220242 (+    13us) tablet.cc:662] Decoding operations
0709 12:58:32.221905 (+  1663us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:32.221995 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:32.222012 (+    17us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:58:32.229716 (+  7704us) tablet.cc:701] Row locks acquired
0709 12:58:32.229770 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:58:32.229954 (+   184us) write_op.cc:270] Start()
0709 12:58:32.230102 (+   148us) write_op.cc:276] Timestamp: P: 1783601912229888 usec, L: 0
0709 12:58:32.230125 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:58:32.275559 (+ 45434us) log.cc:844] Serialized 8981 byte log entry
0709 12:58:32.470361 (+194802us) op_driver.cc:464] REPLICATION: finished
0709 12:58:32.613761 (+143400us) write_op.cc:301] APPLY: starting
0709 12:58:32.613892 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:32.673397 (+ 59505us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b540041e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7af32a 00007fa02e734f14 00007fa02e734ec3 00007fa02e734e38 00007fa02e734df3 00007fa02e7340dc 00007fa02e75de16 00007fa02e745cbb 00007fa02e73039d 00007fa02e733d75
0709 12:58:32.925159 (+251762us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:32.937700 (+ 12541us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:33.175899 (+238199us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:33.176551 (+   652us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 12:58:33.176601 (+    50us) write_op.cc:312] APPLY: finished
0709 12:58:33.177221 (+   620us) log.cc:844] Serialized 204 byte log entry
0709 12:58:33.178295 (+  1074us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:33.178561 (+   266us) write_op.cc:454] Released schema lock
0709 12:58:33.178615 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:33.204898 (+ 26283us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:33.210945 (+  6047us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3397,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":143063,"apply.run_cpu_time_us":33185,"apply.run_wall_time_us":580463,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100175,"num_ops":23,"prepare.queue_time_us":30687,"prepare.run_cpu_time_us":8221,"prepare.run_wall_time_us":180596,"raft.queue_time_us":35973,"raft.run_cpu_time_us":50988,"raft.run_wall_time_us":220003,"replication_time_us":240016,"spinlock_wait_cycles":21136768,"thread_start_us":264254,"threads_started":4,"wal-append.queue_time_us":137685,"wal-append.run_cpu_time_us":4464,"wal-append.run_wall_time_us":1763714}]]}
I20260709 12:58:34.992712 13535 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:34.993419 13535 raft_consensus.cc:606] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:35.006238 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:35.007089 13539 raft_consensus.cc:3055] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:58:35.007514 13539 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:34.983186 18889 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:58:34.986263 18939 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:35.025454 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:58:35.029989 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:34.916072 18962 consensus_queue.cc:1059] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 12:58:35.056936 19010 raft_consensus.cc:3055] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:58:35.057397 19010 raft_consensus.cc:740] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:35.058218 19010 consensus_queue.cc:260] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:35.034955 19050 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:58:35.079118 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 79328.5us
W20260709 12:58:35.065138 19056 log.cc:927] Time spent T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:58:35.100019 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 114806us
I20260709 12:58:35.059881 19010 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:35.109885 18938 raft_consensus.cc:993] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:35.126216 18889 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:35.146442 18889 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:35.163144 18938 raft_consensus.cc:1081] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:35.172009 13539 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:35.173231 13539 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:35.200492 13539 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
W20260709 12:58:35.201491 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 196535us
W20260709 12:58:35.209707 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 209322us
I20260709 12:58:35.224707 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:35.233844 19046 raft_consensus.cc:993] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:35.234431 18976 raft_consensus.cc:1081] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:35.236130 18959 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:35.251472 13684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:35.253191 14242 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:58:35.262483 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 180978us
I20260709 12:58:35.252177 13684 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:35.266438 13684 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:35.273550 13614 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:58:35.274253 13614 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:35.274649 13614 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:35.317831 13684 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:35.260942 18957 consensus_queue.cc:1048] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 12:58:35.395224 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 59796.3us
W20260709 12:58:35.406000 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 67170.6us
I20260709 12:58:35.416934 13612 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 10. Current term is 11. Ops: []
I20260709 12:58:35.459827 13684 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:35.462069 13614 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:35.472846 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 55003.1us
I20260709 12:58:35.479439 13614 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:35.493192 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:35.493273 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:35.493788 13684 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:35.494077 14244 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:35.495172 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:35.528257 13684 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
W20260709 12:58:35.555112 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 165230us
I20260709 12:58:35.559103 13543 raft_consensus.cc:1275] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:58:35.561595 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:35.562214 13536 raft_consensus.cc:3055] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:35.562490 13536 raft_consensus.cc:740] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:35.556999 13582 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:35.575145 13536 consensus_queue.cc:260] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:35.578809 14244 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:35.580404 13658 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:35.599198 19065 log.cc:927] Time spent T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.137s	user 0.002s	sys 0.000s
I20260709 12:58:35.574256 18889 raft_consensus.cc:2804] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:35.619031 18964 raft_consensus.cc:2804] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:35.636929 18964 raft_consensus.cc:697] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:35.638239 18964 consensus_queue.cc:237] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:35.662431 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 87636.6us
W20260709 12:58:35.663973 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:35.627277 18889 raft_consensus.cc:697] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:35.698853 13684 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:58:35.701630 19043 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:58:35.706979 18889 consensus_queue.cc:237] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:35.815213 19076 log.cc:927] Time spent T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:58:35.845005 18961 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:35.865216 13539 raft_consensus.cc:3055] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:35.865857 13539 raft_consensus.cc:740] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:35.866899 13539 consensus_queue.cc:260] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:35.874648 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 166324us
I20260709 12:58:35.879796 18961 consensus_queue.cc:1048] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:58:35.880177 18797 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:35.888240 13539 raft_consensus.cc:3060] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:35.891260 13616 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:35.869503 19077 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:35.908493 13539 raft_consensus.cc:2468] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:35.941766 13536 raft_consensus.cc:3060] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:35.950196 13536 raft_consensus.cc:2468] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
W20260709 12:58:35.950723 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 86799.2us
I20260709 12:58:35.888602 18797 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:35.904588 19077 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:35.982934 19077 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:35.990314 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:35.998271 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:35.968066 13686 raft_consensus.cc:1240] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Rejecting Update request from peer ad2fdb1517c542439654bd3f7c72dad0 for earlier term 14. Current term is 15. Ops: []
I20260709 12:58:35.979312 18797 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:36.030889 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:36.034399 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
I20260709 12:58:36.035874 14211 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:58:36.057699 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 106133us
I20260709 12:58:36.059767 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:36.062676 14211 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:36.065325 18658 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:36.057945 18797 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:36.128131 18960 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:36.156417 18960 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:36.166983 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0" is_pre_election: true
W20260709 12:58:36.143831 19070 log.cc:927] Time spent T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.468s	user 0.000s	sys 0.004s
I20260709 12:58:36.159036 18960 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:36.180296 13536 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:58:36.204694 19094 log.cc:927] Time spent T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.005s
I20260709 12:58:36.196466 19047 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:58:36.231819 13684 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:36.240408 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:36.258997 18960 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:36.259665 18960 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:36.261902 18960 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:36.263451 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:36.264261 13615 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:36.267131 13542 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:58:36.284121 14209 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:36.291877 18961 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:58:36.307106 19026 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:36.319792 13536 raft_consensus.cc:1275] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:36.324116 19047 consensus_queue.cc:1048] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:36.384353 19062 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:58:36.385638 13438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41440 (request call id 211) took 5106 ms. Trace:
I20260709 12:58:36.397302 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:36.407094 13689 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:36.386062 13438 rpcz_store.cc:276] 0709 12:58:31.279226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:31.281493 (+  2267us) service_pool.cc:224] Handling call
0709 12:58:36.385541 (+5104048us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:35.240453 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:35.241251 (+   798us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:35.241299 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:58:35.241326 (+    27us) tablet.cc:662] Decoding operations
0709 12:58:35.242804 (+  1478us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:35.242885 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:35.242914 (+    29us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:58:35.376398 (+133484us) tablet.cc:701] Row locks acquired
0709 12:58:35.376418 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:58:35.376549 (+   131us) write_op.cc:270] Start()
0709 12:58:35.376667 (+   118us) write_op.cc:276] Timestamp: P: 1783601915376512 usec, L: 0
0709 12:58:35.376693 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:58:35.377538 (+   845us) log.cc:844] Serialized 6994 byte log entry
0709 12:58:35.845828 (+468290us) op_driver.cc:464] REPLICATION: finished
0709 12:58:35.973526 (+127698us) write_op.cc:301] APPLY: starting
0709 12:58:35.973612 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:36.150119 (+176507us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:36.150282 (+   163us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:36.227808 (+ 77526us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:36.228121 (+   313us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:58:36.228166 (+    45us) write_op.cc:312] APPLY: finished
0709 12:58:36.228564 (+   398us) log.cc:844] Serialized 164 byte log entry
0709 12:58:36.281501 (+ 52937us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:36.326874 (+ 45373us) write_op.cc:454] Released schema lock
0709 12:58:36.326946 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:36.327382 (+   436us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:36.327413 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1367706,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":127408,"apply.run_cpu_time_us":24565,"apply.run_wall_time_us":353819,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79295,"num_ops":18,"prepare.queue_time_us":361842,"prepare.run_cpu_time_us":25217,"prepare.run_wall_time_us":291783,"raft.queue_time_us":172852,"raft.run_cpu_time_us":34065,"raft.run_wall_time_us":98728,"replication_time_us":468897,"thread_start_us":572681,"threads_started":4,"wal-append.queue_time_us":250541}]]}
I20260709 12:58:36.410493 13439 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 13 to 14, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 14 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:36.415261 19088 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:36.415940 19088 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:36.418507 13439 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 9 to 11, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:36.419178 19088 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:36.422505 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:36.433588 13689 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 17.
I20260709 12:58:36.420372 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:36.435038 13619 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 17: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:36.436800 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:36.440846 19087 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:36.441260 19087 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:36.428390 13439 catalog_manager.cc:5697] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 8 to 12, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 12 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:36.452596 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:36.441658 19087 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:36.458309 18979 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:36.458810 18979 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:36.459197 18979 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:36.472401 18979 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:36.500721 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:36.501524 13619 raft_consensus.cc:2393] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:36.504165 18979 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:36.505358 13439 catalog_manager.cc:5697] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 1 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 13 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:36.506127 14242 raft_consensus.cc:1275] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:58:36.500497 19087 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:36.514859 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:36.516878 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:36.520058 19087 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:36.524425 13619 raft_consensus.cc:2393] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ad2fdb1517c542439654bd3f7c72dad0 in current term 17: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:36.542497 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:36.544324 13689 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 17.
I20260709 12:58:36.553442 13684 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:36.548489 13506 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:36.563221 14209 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:36.563763 19088 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:36.573269 19026 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:36.577127 19088 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:36.572130 19014 consensus_queue.cc:1048] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:58:36.588318 19026 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:36.627038 19088 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:36.645546 19026 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:36.791334 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 60691.5us
I20260709 12:58:36.861277 13436 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:36.865226 13436 catalog_manager.cc:5697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:36.867201 13436 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 11 to 14, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:36.869580 13436 catalog_manager.cc:5697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 8 to 15, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 15 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:36.871464 13436 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:36.888487 13436 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 13 to 14, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 14 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:36.896144 13436 catalog_manager.cc:5697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
W20260709 12:58:36.901069 13439 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49892 (request call id 320) took 22329 ms (client timeout 15000 ms). Trace:
W20260709 12:58:36.901618 13439 rpcz_store.cc:269] 0709 12:58:14.571883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:14.620176 (+ 48293us) service_pool.cc:224] Handling call
0709 12:58:36.900982 (+22280806us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:36.630322 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:36.661190 (+ 30868us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:36.661228 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:58:36.661247 (+    19us) tablet.cc:662] Decoding operations
0709 12:58:36.661829 (+   582us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:36.661906 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:36.661932 (+    26us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:58:36.662317 (+   385us) tablet.cc:701] Row locks acquired
0709 12:58:36.662334 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:58:36.662473 (+   139us) write_op.cc:270] Start()
0709 12:58:36.662591 (+   118us) write_op.cc:276] Timestamp: P: 1783601916662432 usec, L: 0
0709 12:58:36.662614 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:58:36.686798 (+ 24184us) log.cc:844] Serialized 1694 byte log entry
0709 12:58:36.751360 (+ 64562us) op_driver.cc:464] REPLICATION: finished
0709 12:58:36.751854 (+   494us) write_op.cc:301] APPLY: starting
0709 12:58:36.751974 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:36.754542 (+  2568us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:36.754661 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:36.757890 (+  3229us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:36.758074 (+   184us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:58:36.758121 (+    47us) write_op.cc:312] APPLY: finished
0709 12:58:36.758384 (+   263us) log.cc:844] Serialized 49 byte log entry
0709 12:58:36.758553 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:36.758651 (+    98us) write_op.cc:454] Released schema lock
0709 12:58:36.758689 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:36.759115 (+   426us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:36.805184 (+ 46069us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3719,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":142,"apply.run_cpu_time_us":6907,"apply.run_wall_time_us":7280,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16704,"num_ops":4,"prepare.queue_time_us":86441,"prepare.run_cpu_time_us":4075,"prepare.run_wall_time_us":67190,"replication_time_us":88584,"spinlock_wait_cycles":36096,"thread_start_us":53174,"threads_started":1}]]}
I20260709 12:58:36.907662 13436 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 13 to 16, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 16 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:36.917197 13436 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:36.922348 13436 catalog_manager.cc:5697] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:37.069188 13440 catalog_manager.cc:5697] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:37.088558 13440 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 14 to 16, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 16 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.091194 13440 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 14 to 15, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:37.096174 13615 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:37.097892 18972 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:37.109494 13440 catalog_manager.cc:5697] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 10 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.113240 13440 catalog_manager.cc:5697] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.114394 13440 catalog_manager.cc:5697] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 12 to 14, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.117870 13440 catalog_manager.cc:5697] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 10 to 11, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:37.142275 13440 catalog_manager.cc:5697] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.143751 13440 catalog_manager.cc:5697] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 10 to 14, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 14 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.145198 13440 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.146341 13440 catalog_manager.cc:5697] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 12 to 15, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:37.147336 13440 catalog_manager.cc:5697] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.150120 13440 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 14 to 15, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 15 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:37.153999 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:58:37.164400 13436 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49896 (request call id 306) took 11422 ms (client timeout 15000 ms). Trace:
I20260709 12:58:37.177752 13619 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:58:37.257414 19110 log.cc:927] Time spent T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:58:37.276610 13692 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:37.250269 19116 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:58:37.294324 19026 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:58:37.339780 13689 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:58:37.173049 13436 rpcz_store.cc:269] 0709 12:58:25.739786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:25.741215 (+  1429us) service_pool.cc:224] Handling call
0709 12:58:37.162160 (+11420945us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:36.941351 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:36.941920 (+   569us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:36.941959 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:58:36.941974 (+    15us) tablet.cc:662] Decoding operations
0709 12:58:36.942999 (+  1025us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:36.943055 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:36.943084 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:58:36.943961 (+   877us) tablet.cc:701] Row locks acquired
0709 12:58:36.943985 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:58:36.944167 (+   182us) write_op.cc:270] Start()
0709 12:58:36.944279 (+   112us) write_op.cc:276] Timestamp: P: 1783601916944113 usec, L: 0
0709 12:58:36.944300 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:58:36.944961 (+   661us) log.cc:844] Serialized 4000 byte log entry
0709 12:58:36.947250 (+  2289us) op_driver.cc:464] REPLICATION: finished
0709 12:58:36.948740 (+  1490us) write_op.cc:301] APPLY: starting
0709 12:58:36.948824 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:36.974879 (+ 26055us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:36.974976 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:37.005797 (+ 30821us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:37.010891 (+  5094us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 12:58:37.010939 (+    48us) write_op.cc:312] APPLY: finished
0709 12:58:37.011258 (+   319us) log.cc:844] Serialized 97 byte log entry
0709 12:58:37.011449 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:37.015482 (+  4033us) write_op.cc:454] Released schema lock
0709 12:58:37.015533 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:37.020734 (+  5201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:37.020771 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":49576,"reported_tablets":47,"spinlock_wait_cycles":1280,"child_traces":[["op",{"apply.queue_time_us":1243,"apply.run_cpu_time_us":13396,"apply.run_wall_time_us":67121,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41760,"num_ops":10,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":3906,"prepare.run_wall_time_us":5557,"replication_time_us":2812}]]}
I20260709 12:58:37.362771 18807 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:58:37.407419 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 70148.7us
I20260709 12:58:37.453198 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:37.453801 13540 raft_consensus.cc:606] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:37.496325 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:37.496975 13536 raft_consensus.cc:606] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
W20260709 12:58:37.500501 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 72435.9us
W20260709 12:58:37.501902 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 159126us
I20260709 12:58:37.542826 13536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:37.543428 13536 raft_consensus.cc:606] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
W20260709 12:58:37.603189 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 55153.8us
W20260709 12:58:37.745558 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 56709.1us
W20260709 12:58:37.746510 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 242906us
I20260709 12:58:37.761559 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85eb0ac09bf7466899637997aae42111"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:37.762203 13540 raft_consensus.cc:606] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:37.871425 19147 raft_consensus.cc:993] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:37.872071 19147 raft_consensus.cc:1081] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:37.874692 13692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:37.875223 13692 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:37.875556 13692 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:37.878717 19122 raft_consensus.cc:993] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:37.885811 19142 raft_consensus.cc:1081] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:37.887823 13690 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:37.887565 13692 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:37.888437 13690 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:37.888864 13690 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:37.904227 14212 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.017s	sys 0.000s
W20260709 12:58:37.905886 14212 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.017s	sys 0.000s
W20260709 12:58:37.906208 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 623047us
I20260709 12:58:37.903879 13690 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:37.924181 13690 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:37.925000 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:37.932281 13619 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:37.959487 13692 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:37.972178 13438 catalog_manager.cc:5697] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 14 to 16, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:37.973922 13438 catalog_manager.cc:5697] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:37.975359 13438 catalog_manager.cc:5697] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 16, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:37.988112 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:37.988958 13542 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:58:37.989446 13542 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:37.990301 13542 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:37.991325 13542 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:37.995193 13539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:37.995815 13539 raft_consensus.cc:3055] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:37.996181 13539 raft_consensus.cc:740] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:37.997130 13539 consensus_queue.cc:260] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:37.998801 13539 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:37.999203 13438 catalog_manager.cc:5697] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:38.000355 13438 catalog_manager.cc:5697] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:38.001428 13438 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 12 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:38.002382 13438 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 13, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:38.003533 13438 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 12, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:38.004914 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:38.005502 13615 raft_consensus.cc:3060] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:38.007603 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 243686us
I20260709 12:58:38.009249 13619 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:38.010083 13440 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49892 (request call id 321) took 6106 ms. Trace:
I20260709 12:58:38.035571 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:38.043684 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 131022us
I20260709 12:58:38.047961 13539 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:38.014226 13440 rpcz_store.cc:276] 0709 12:58:31.904378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:31.919548 (+ 15170us) service_pool.cc:224] Handling call
0709 12:58:38.010017 (+6090469us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:37.254950 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:37.257267 (+  2317us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:37.257302 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:58:37.257320 (+    18us) tablet.cc:662] Decoding operations
0709 12:58:37.258498 (+  1178us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:37.258562 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:37.258590 (+    28us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:58:37.260129 (+  1539us) tablet.cc:701] Row locks acquired
0709 12:58:37.260148 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:58:37.260317 (+   169us) write_op.cc:270] Start()
0709 12:58:37.260436 (+   119us) write_op.cc:276] Timestamp: P: 1783601917260278 usec, L: 0
0709 12:58:37.260459 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:58:37.261345 (+   886us) log.cc:844] Serialized 5149 byte log entry
0709 12:58:37.265905 (+  4560us) op_driver.cc:464] REPLICATION: finished
0709 12:58:37.266569 (+   664us) write_op.cc:301] APPLY: starting
0709 12:58:37.266661 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:37.273669 (+  7008us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:37.273757 (+    88us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:37.701463 (+427706us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:37.701700 (+   237us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:58:37.701755 (+    55us) write_op.cc:312] APPLY: finished
0709 12:58:37.702082 (+   327us) log.cc:844] Serialized 121 byte log entry
0709 12:58:37.742328 (+ 40246us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:37.746694 (+  4366us) write_op.cc:454] Released schema lock
0709 12:58:37.755127 (+  8433us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:37.808253 (+ 53126us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:37.810025 (+  1772us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":69063,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":399,"apply.run_cpu_time_us":19815,"apply.run_wall_time_us":509488,"cfile_cache_hit":52,"cfile_cache_hit_bytes":54288,"num_ops":13,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":4634,"prepare.run_wall_time_us":9302,"replication_time_us":5276,"spinlock_wait_cycles":16640}]]}
I20260709 12:58:38.058013 13658 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:38.060513 19115 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:38.071437 18963 raft_consensus.cc:2804] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:38.072023 18963 raft_consensus.cc:697] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:38.072886 18963 consensus_queue.cc:237] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:38.076145 19115 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:38.077061 19115 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:38.080042 13540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:38.080691 13540 raft_consensus.cc:606] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:38.088943 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:58:38.090899 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:38.101734 13615 raft_consensus.cc:2468] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:38.105460 19142 raft_consensus.cc:993] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:38.105921 19142 raft_consensus.cc:1081] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:38.106448 13542 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:38.108698 13692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:38.109354 13692 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:38.109757 13692 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:38.121917 13692 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:38.124017 13692 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:38.144210 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:38.144840 13542 raft_consensus.cc:3055] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:38.145574 13542 raft_consensus.cc:740] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:38.146405 13542 consensus_queue.cc:260] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:38.148690 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:38.149294 14238 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:38.164117 14238 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:38.171464 13542 raft_consensus.cc:3060] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:38.180964 18974 raft_consensus.cc:993] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:38.181543 19142 raft_consensus.cc:1081] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:38.188127 13690 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85eb0ac09bf7466899637997aae42111"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:38.195818 13690 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:38.201051 13690 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:58:38.237499 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 78706.6us
I20260709 12:58:38.312224 13542 raft_consensus.cc:2468] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:38.282778 13690 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:38.315347 13658 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:38.326390 13690 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:38.333876 19115 raft_consensus.cc:2804] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:38.334558 19115 raft_consensus.cc:697] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:38.335521 19115 consensus_queue.cc:237] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:38.340212 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:38.340878 14238 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:38.350842 14238 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:38.406621 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:38.408360 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 56510.1us
I20260709 12:58:38.427095 13540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:38.428645 13540 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:38.429220 13540 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:38.430519 18963 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:38.431305 18963 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:38.433027 18963 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:38.387575 13438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:41440 (request call id 212) took 1951 ms. Trace:
I20260709 12:58:38.486544 13435 catalog_manager.cc:5697] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:38.496199 13540 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:38.499141 13540 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:38.516136 13435 catalog_manager.cc:5697] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 13 to 15, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 15 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:38.545022 13540 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:38.443497 13438 rpcz_store.cc:276] 0709 12:58:36.435960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:36.445853 (+  9893us) service_pool.cc:224] Handling call
0709 12:58:38.387204 (+1941351us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:38.188803 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:38.189354 (+   551us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:38.189391 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:58:38.189410 (+    19us) tablet.cc:662] Decoding operations
0709 12:58:38.190359 (+   949us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:38.190564 (+   205us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:38.190598 (+    34us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:58:38.191265 (+   667us) tablet.cc:701] Row locks acquired
0709 12:58:38.191286 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:58:38.191428 (+   142us) write_op.cc:270] Start()
0709 12:58:38.191539 (+   111us) write_op.cc:276] Timestamp: P: 1783601918191386 usec, L: 0
0709 12:58:38.191565 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:58:38.205251 (+ 13686us) log.cc:844] Serialized 3256 byte log entry
0709 12:58:38.287004 (+ 81753us) op_driver.cc:464] REPLICATION: finished
0709 12:58:38.287375 (+   371us) write_op.cc:301] APPLY: starting
0709 12:58:38.287467 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:38.293159 (+  5692us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:38.293405 (+   246us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:38.298510 (+  5105us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:38.298745 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:58:38.298801 (+    56us) write_op.cc:312] APPLY: finished
0709 12:58:38.299094 (+   293us) log.cc:844] Serialized 81 byte log entry
0709 12:58:38.300002 (+   908us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:38.300138 (+   136us) write_op.cc:454] Released schema lock
0709 12:58:38.300187 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:38.301973 (+  1786us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:38.302004 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":103853,"reported_tablets":11,"spinlock_wait_cycles":3456,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":11082,"apply.run_wall_time_us":14537,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33408,"num_ops":8,"prepare.queue_time_us":148218,"prepare.run_cpu_time_us":4302,"prepare.run_wall_time_us":41693,"replication_time_us":95269,"thread_start_us":92983,"threads_started":1}]]}
I20260709 12:58:38.565598 13543 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:58:38.608727 19122 raft_consensus.cc:993] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:38.629030 19116 raft_consensus.cc:1081] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:38.653577 13692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:38.654426 13692 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:38.654815 13692 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:38.666330 19112 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:38.694550 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59318
I20260709 12:58:38.695353 13690 raft_consensus.cc:606] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Received request to transfer leadership
I20260709 12:58:38.701360 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59318
I20260709 12:58:38.701974 13690 raft_consensus.cc:606] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Received request to transfer leadership
W20260709 12:58:38.703120 13425 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 048528e413ac4c67a1cfd3dfe70673c7 already in progress [suppressed 1 similar messages]
I20260709 12:58:38.703727 13438 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:38.705715 13438 catalog_manager.cc:5697] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 15 to 16, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 16 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:38.717837 13689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59318
I20260709 12:58:38.718626 13689 raft_consensus.cc:606] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Received request to transfer leadership
I20260709 12:58:38.724365 13540 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:58:38.738186 13692 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:38.742179 19064 raft_consensus.cc:993] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: : Instructing follower ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:58:38.742947 19064 raft_consensus.cc:1081] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Signalling peer ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:58:38.749205 18963 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:38.759277 13692 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:38.762156 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:38.762856 13616 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:38.766317 13542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:49526
I20260709 12:58:38.767024 13542 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:38.767417 13542 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:38.791061 13535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:38.791816 13535 raft_consensus.cc:3055] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:58:38.796551 13535 raft_consensus.cc:740] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:38.797470 13535 consensus_queue.cc:260] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:38.798648 13535 raft_consensus.cc:3060] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:38.812520 13615 raft_consensus.cc:1275] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:58:38.817029 19133 consensus_queue.cc:1048] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:38.820771 13612 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:58:38.823462 13435 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49880 (request call id 331) took 6432 ms. Trace:
I20260709 12:58:38.824390 19115 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:38.852550 13616 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:38.870465 13542 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:38.893797 13489 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:58:38.894358 13489 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 048528e413ac4c67a1cfd3dfe70673c7 already in progress
W20260709 12:58:38.894675 13489 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 048528e413ac4c67a1cfd3dfe70673c7 already in progress
I20260709 12:58:38.895378 13659 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:38.906453 19064 raft_consensus.cc:2804] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:38.908604 14240 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:38.911158 13535 raft_consensus.cc:2468] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:38.913308 19064 raft_consensus.cc:697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:38.920131 19064 consensus_queue.cc:237] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:38.944000 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:38.944700 13616 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:39.028723 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:39.029438 13692 raft_consensus.cc:3055] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:39.029728 13692 raft_consensus.cc:740] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:39.030627 13692 consensus_queue.cc:260] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:38.824015 13435 rpcz_store.cc:276] 0709 12:58:32.390654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:32.390945 (+   291us) service_pool.cc:224] Handling call
0709 12:58:38.823379 (+6432434us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:38.650959 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:38.651768 (+   809us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:38.651820 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:58:38.651836 (+    16us) tablet.cc:662] Decoding operations
0709 12:58:38.652410 (+   574us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:38.652522 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:38.652551 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:58:38.653135 (+   584us) tablet.cc:701] Row locks acquired
0709 12:58:38.653194 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:58:38.653356 (+   162us) write_op.cc:270] Start()
0709 12:58:38.653554 (+   198us) write_op.cc:276] Timestamp: P: 1783601918653286 usec, L: 0
0709 12:58:38.653588 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:58:38.656311 (+  2723us) log.cc:844] Serialized 923 byte log entry
0709 12:58:38.659737 (+  3426us) op_driver.cc:464] REPLICATION: finished
0709 12:58:38.661806 (+  2069us) write_op.cc:301] APPLY: starting
0709 12:58:38.661919 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:38.664163 (+  2244us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:38.664294 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:38.665916 (+  1622us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:38.675228 (+  9312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 12:58:38.675301 (+    73us) write_op.cc:312] APPLY: finished
0709 12:58:38.675627 (+   326us) log.cc:844] Serialized 33 byte log entry
0709 12:58:38.675833 (+   206us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:38.675964 (+   131us) write_op.cc:454] Released schema lock
0709 12:58:38.676028 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:38.676516 (+   488us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:38.676545 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":1695,"apply.run_cpu_time_us":5682,"apply.run_wall_time_us":14693,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8352,"num_ops":2,"prepare.queue_time_us":270,"prepare.run_cpu_time_us":4154,"prepare.run_wall_time_us":8506,"replication_time_us":5979,"spinlock_wait_cycles":23424}]]}
I20260709 12:58:39.032604 13692 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:39.048996 13692 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:39.046028 14240 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:58:39.060293 13542 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:39.064662 13616 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:39.083580 19112 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:39.087322 18963 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:39.107503 13536 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:58:39.110386 13543 raft_consensus.cc:1275] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:58:39.132025 13656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:39.133594 13506 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:39.134665 19150 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:39.135411 19150 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:39.135270 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:39.137621 14238 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:39.136674 19150 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:39.110035 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:39.192456 14238 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 14.
I20260709 12:58:39.195353 13435 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:39.197397 13436 catalog_manager.cc:5697] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 15 to 17, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
W20260709 12:58:39.201685 13441 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49896 (request call id 305) took 32391 ms (client timeout 15000 ms). Trace:
I20260709 12:58:39.201382 13436 catalog_manager.cc:5697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
W20260709 12:58:39.202297 13441 rpcz_store.cc:269] 0709 12:58:06.810278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:06.810669 (+   391us) service_pool.cc:224] Handling call
0709 12:58:07.206140 (+395471us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b540041e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 12:58:39.201472 (+31995332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":290511,"reported_tablets":47,"spinlock_wait_cycles":18071424}
I20260709 12:58:39.202975 13436 catalog_manager.cc:5697] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 14 to 15, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
W20260709 12:58:39.205042 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 63498us
I20260709 12:58:39.204957 13436 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:39.216274 13436 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 14 to 16, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 16 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:39.220538 18963 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:39.225363 19064 consensus_queue.cc:1048] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:58:39.236904 13436 catalog_manager.cc:5697] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:39.251441 13436 catalog_manager.cc:5697] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:39.261797 13436 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 7 to 9, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:39.267282 13435 catalog_manager.cc:5697] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:39.326174 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 47646.5us
W20260709 12:58:39.328510 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 49858.5us
I20260709 12:58:39.335675 13436 catalog_manager.cc:5697] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:39.339008 13436 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 10 to 11, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:39.528530 13441 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 13 to 14, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:39.540658 13436 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49896 (request call id 307) took 1292 ms. Trace:
I20260709 12:58:39.532285 13441 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 16 to 17, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } }
I20260709 12:58:39.536139 13536 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:58:39.585909 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47047.5us
I20260709 12:58:39.593731 13441 catalog_manager.cc:5697] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 9 to 12, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } }
I20260709 12:58:39.598971 19112 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:39.606153 13611 raft_consensus.cc:1275] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:39.588145 13436 rpcz_store.cc:276] 0709 12:58:38.248552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:38.248781 (+   229us) service_pool.cc:224] Handling call
0709 12:58:39.540573 (+1291792us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:58:39.387447 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:58:39.388108 (+   661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:58:39.388145 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:58:39.388166 (+    21us) tablet.cc:662] Decoding operations
0709 12:58:39.389228 (+  1062us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:58:39.389286 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:58:39.389317 (+    31us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:58:39.391003 (+  1686us) tablet.cc:701] Row locks acquired
0709 12:58:39.391023 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:58:39.391159 (+   136us) write_op.cc:270] Start()
0709 12:58:39.391286 (+   127us) write_op.cc:276] Timestamp: P: 1783601919391112 usec, L: 0
0709 12:58:39.391314 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:58:39.391961 (+   647us) log.cc:844] Serialized 4004 byte log entry
0709 12:58:39.394308 (+  2347us) op_driver.cc:464] REPLICATION: finished
0709 12:58:39.394704 (+   396us) write_op.cc:301] APPLY: starting
0709 12:58:39.394791 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:58:39.400550 (+  5759us) tablet.cc:1370] finished BulkCheckPresence
0709 12:58:39.400654 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:58:39.442997 (+ 42343us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:58:39.443239 (+   242us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 12:58:39.443289 (+    50us) write_op.cc:312] APPLY: finished
0709 12:58:39.443614 (+   325us) log.cc:844] Serialized 97 byte log entry
0709 12:58:39.457667 (+ 14053us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:39.457815 (+   148us) write_op.cc:454] Released schema lock
0709 12:58:39.457864 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:58:39.458271 (+   407us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:58:39.458309 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":54341,"reported_tablets":34,"spinlock_wait_cycles":88320,"child_traces":[["op",{"apply.queue_time_us":220,"apply.run_cpu_time_us":13964,"apply.run_wall_time_us":63484,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41760,"num_ops":10,"prepare.queue_time_us":624,"prepare.run_cpu_time_us":4240,"prepare.run_wall_time_us":4814,"replication_time_us":2890}]]}
I20260709 12:58:39.608376 14238 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:39.610060 19150 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:58:39.616585 19176 consensus_queue.cc:1048] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:39.645262 13690 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:39.646920 19150 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:58:39.684959 13436 catalog_manager.cc:5697] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 14 to 15, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 15 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:39.752380 13616 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:58:39.757364 18974 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:58:39.805258 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 68581us
I20260709 12:58:39.812443 13440 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49892 (request call id 322) took 1568 ms. Trace:
I20260709 12:58:39.812868 13440 rpcz_store.cc:276] 0709 12:58:38.244109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:38.244968 (+   859us) service_pool.cc:224] Handling call
0709 12:58:39.812245 (+1567277us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":23,"spinlock_wait_cycles":39680}
I20260709 12:58:40.022033 13543 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:40.029668 14244 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:58:40.033330 13612 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:58:40.034955 19142 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:58:40.038478 18974 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:58:40.054529 19150 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Committing config change with OpId 14.14: config changed from index 11 to 14, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:40.060628 14244 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Committing config change with OpId 14.14: config changed from index 11 to 14, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:40.079680 13612 raft_consensus.cc:2955] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Committing config change with OpId 14.14: config changed from index 11 to 14, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:40.121949 13427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 048528e413ac4c67a1cfd3dfe70673c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:58:40.126765 13440 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 11 to 14, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 14 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:40.177846 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 048528e413ac4c67a1cfd3dfe70673c7 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:59318
I20260709 12:58:40.182878 13686 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:40.212731 19196 tablet_replica.cc:333] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:58:40.214224 19196 raft_consensus.cc:2243] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:40.215090 19196 raft_consensus.cc:2272] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:40.219734 13612 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:58:40.221756 13542 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:58:40.223327 19179 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:58:40.227456 19112 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:58:40.227272 13658 consensus_peers.cc:597] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f97a6462cd4341ddbd7058ce9537c1eb. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:40.408407 19198 log.cc:927] Time spent T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:58:40.556691 19196 ts_tablet_manager.cc:1916] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:58:40.555493 19211 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Committing config change with OpId 9.7: config changed from index -1 to 7, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:40.595137 13612 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index -1 to 7, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:40.607965 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:40.621340 13690 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:40.637748 13440 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 7, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:40.682886 19196 ts_tablet_manager.cc:1929] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.13
I20260709 12:58:40.683593 19196 log.cc:1199] T 048528e413ac4c67a1cfd3dfe70673c7 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/048528e413ac4c67a1cfd3dfe70673c7
I20260709 12:58:40.685875 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 048528e413ac4c67a1cfd3dfe70673c7 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:58:40.698892 13540 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index -1 to 7, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:40.720161 19226 ts_tablet_manager.cc:933] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:40.794075 13615 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:58:40.804303 19226 tablet_copy_client.cc:323] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:58:40.820928 13702 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet f97a6462cd4341ddbd7058ce9537c1eb from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:58:40.821619 13702 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet f97a6462cd4341ddbd7058ce9537c1eb from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-f97a6462cd4341ddbd7058ce9537c1eb
I20260709 12:58:40.826375 13540 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:58:40.831161 13658 consensus_peers.cc:597] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 921257a2480244e88f3ff036fb3be8d2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:40.832593 18882 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:40.834883 18882 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:40.878139 13702 tablet_copy_source_session.cc:215] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:40.883150 19226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97a6462cd4341ddbd7058ce9537c1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:40.908113 19226 tablet_copy_client.cc:806] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:58:40.908833 19226 tablet_copy_client.cc:670] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:58:40.913511 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59336
I20260709 12:58:40.914130 13690 raft_consensus.cc:606] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:40.918514 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:59336
I20260709 12:58:40.919071 13690 raft_consensus.cc:606] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Received request to transfer leadership to TS ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:40.918998 19231 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:40.930076 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85eb0ac09bf7466899637997aae42111"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:59336
I20260709 12:58:40.930668 13690 raft_consensus.cc:606] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Received request to transfer leadership to TS ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:40.914194 19226 tablet_copy_client.cc:538] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:58:40.934058 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:58:40.937129 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:40.929422 13615 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:40.961803 19226 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:58:40.983604 13540 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.001446 13436 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 16 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:41.088696 19179 raft_consensus.cc:993] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: : Instructing follower ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:58:41.089316 19214 raft_consensus.cc:1081] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Signalling peer ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:58:41.093351 13542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:49526
I20260709 12:58:41.093974 13542 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:41.094368 13542 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:41.104729 13542 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:41.107959 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:41.108567 13616 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:41.110422 13615 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:41.121853 13616 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 18.
I20260709 12:58:41.124014 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:41.123792 13542 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:41.124709 13689 raft_consensus.cc:3055] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:58:41.125093 13689 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:41.125860 13509 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:41.126791 19200 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:58:41.127269 19200 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 18 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:41.128180 19200 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:41.125905 13689 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:41.128079 19214 raft_consensus.cc:993] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:58:41.140683 19231 raft_consensus.cc:1081] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:58:41.142894 13689 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:41.151131 14244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59176
I20260709 12:58:41.151810 14244 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:41.152340 14244 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:41.163343 13543 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:58:41.165333 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f6f9228cbd74e95b5192c80f7f4a2b6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:41.166306 19203 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:41.169327 14244 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:41.173561 13690 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:58:41.177937 19202 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:41.249523 13689 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 18.
I20260709 12:58:41.251184 14244 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:41.253999 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:41.260106 13542 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:41.270433 19199 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.287779 13684 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.308487 13440 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 10, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.321863 13543 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.336611 13542 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:41.342634 14211 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:41.347538 19222 raft_consensus.cc:2804] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:41.350636 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:41.351291 13684 raft_consensus.cc:3055] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:41.351558 13684 raft_consensus.cc:740] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:41.352434 13684 consensus_queue.cc:260] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:41.354784 13684 raft_consensus.cc:3060] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:41.348287 19222 raft_consensus.cc:697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:41.358876 19222 consensus_queue.cc:237] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:41.366302 19242 ts_tablet_manager.cc:933] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:41.383451 19226 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=1 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:58:41.371701 13616 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:41.394407 13436 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 17 to 18, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 18 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:41.400352 13684 raft_consensus.cc:2468] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:41.424255 19212 raft_consensus.cc:993] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: : Instructing follower ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:58:41.424847 19212 raft_consensus.cc:1081] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Signalling peer ad2fdb1517c542439654bd3f7c72dad0 to start an election
I20260709 12:58:41.425256 13542 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:58:41.426688 19201 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:41.429270 13542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85eb0ac09bf7466899637997aae42111"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
 from {username='slave'} at 127.0.0.1:49526
I20260709 12:58:41.429811 13542 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:41.430119 13542 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:41.432497 19226 tablet_bootstrap.cc:492] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:58:41.433328 19226 ts_tablet_manager.cc:1403] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.472s	user 0.034s	sys 0.017s
I20260709 12:58:41.437108 13684 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:58:41.442329 13542 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:41.445202 19201 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:41.449388 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:41.435369 19242 tablet_copy_client.cc:323] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:58:41.450014 13684 raft_consensus.cc:3055] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:58:41.452360 13584 consensus_peers.cc:597] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c25c2050d4547c39924ceb739727ae8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:41.450448 13684 raft_consensus.cc:740] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:41.453397 13542 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:41.454402 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:41.456578 14240 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:41.457739 13684 consensus_queue.cc:260] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:41.458986 13684 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:41.466030 13702 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet 921257a2480244e88f3ff036fb3be8d2 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:58:41.466943 13702 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet 921257a2480244e88f3ff036fb3be8d2 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-921257a2480244e88f3ff036fb3be8d2
I20260709 12:58:41.471768 14240 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:41.481983 13508 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:41.483305 19200 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:41.483722 13684 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2fdb1517c542439654bd3f7c72dad0 in term 13.
I20260709 12:58:41.483889 19200 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Becoming Leader. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:41.485324 19200 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:41.476812 13436 catalog_manager.cc:5697] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:41.498663 19226 raft_consensus.cc:359] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:41.520404 19226 raft_consensus.cc:740] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:58:41.522051 19226 consensus_queue.cc:260] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:41.525102 19226 ts_tablet_manager.cc:1434] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.001s
I20260709 12:58:41.519594 13702 tablet_copy_source_session.cc:215] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:41.531098 19242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 921257a2480244e88f3ff036fb3be8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:41.544997 13702 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-f97a6462cd4341ddbd7058ce9537c1eb received from {username='slave'} at 127.0.0.1:40872
I20260709 12:58:41.546079 13702 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-f97a6462cd4341ddbd7058ce9537c1eb on tablet f97a6462cd4341ddbd7058ce9537c1eb with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:41.543277 13543 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.523712 19201 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.575485 13689 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.606483 13436 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:41.628786 13440 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: term changed from 12 to 13, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65). New cstate: current_term: 13 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:41.639197 13616 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:58:41.641914 19150 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:41.725363 14240 raft_consensus.cc:1217] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 LEARNER]: Deduplicated request from leader. Original: 9.6->[9.7-9.7]   Dedup: 9.7->[]
I20260709 12:58:41.729276 19226 ts_tablet_manager.cc:933] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:41.751228 13689 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:41.761328 13540 raft_consensus.cc:1275] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:41.768491 19222 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:58:41.808241 19223 consensus_queue.cc:1048] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:58:41.807953 19226 tablet_copy_client.cc:323] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:58:41.876212 13690 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:58:41.886566 19142 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:41.888175 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:58:41.888852 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-2f6f9228cbd74e95b5192c80f7f4a2b6
I20260709 12:58:41.907565 19242 tablet_copy_client.cc:806] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:58:41.923870 13629 tablet_copy_source_session.cc:215] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:41.928607 19226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:41.928503 19242 tablet_copy_client.cc:670] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:58:41.937466 13689 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:58:41.939282 19142 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:58:41.967206 19266 ts_tablet_manager.cc:933] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:41.976366 19242 tablet_copy_client.cc:538] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:58:42.007169 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:42.053416 19266 tablet_copy_client.cc:323] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
I20260709 12:58:42.112226 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 5c25c2050d4547c39924ceb739727ae8 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:58:42.112982 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 5c25c2050d4547c39924ceb739727ae8 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-5c25c2050d4547c39924ceb739727ae8
I20260709 12:58:42.116184 19226 tablet_copy_client.cc:806] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:58:42.116807 19226 tablet_copy_client.cc:670] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:58:42.108415 14238 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 12:58:42.148909 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 99918.2us
I20260709 12:58:42.140044 18974 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
W20260709 12:58:42.175052 19271 log.cc:927] Time spent T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.101s	user 0.001s	sys 0.002s
I20260709 12:58:42.203975 19226 tablet_copy_client.cc:538] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:58:42.211047 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46215.8us
I20260709 12:58:42.226361 13629 tablet_copy_source_session.cc:215] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:42.328140 19266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c25c2050d4547c39924ceb739727ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:42.341410 19242 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:58:42.354156 19226 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:58:42.369505 18882 raft_consensus.cc:1064] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:42.372416 18882 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:42.414286 19266 tablet_copy_client.cc:806] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:58:42.414915 19266 tablet_copy_client.cc:670] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:58:42.426265 19266 tablet_copy_client.cc:538] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:58:42.501725 19266 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:58:42.565003 13616 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:58:42.569039 19261 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 12:58:42.694190 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 146576us
I20260709 12:58:42.699430 13542 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:58:42.702927 14238 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:58:42.758584 19242 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=2 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:58:42.759810 19242 tablet_bootstrap.cc:492] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:58:42.768179 19242 ts_tablet_manager.cc:1403] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.427s	user 0.047s	sys 0.025s
I20260709 12:58:42.763854 19179 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:58:42.775207 19281 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:58:42.829485 19242 raft_consensus.cc:359] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:42.830569 19242 raft_consensus.cc:740] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:58:42.831370 19242 consensus_queue.cc:260] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:42.845847 14238 raft_consensus.cc:1217] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.8->[16.9-16.9]   Dedup: 16.9->[]
I20260709 12:58:42.856040 19226 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:58:42.890831 18882 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index 7 to 8, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:42.896674 19226 tablet_bootstrap.cc:492] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:58:42.900476 19226 ts_tablet_manager.cc:1403] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.547s	user 0.048s	sys 0.000s
I20260709 12:58:42.903568 13543 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:42.941700 19242 ts_tablet_manager.cc:1434] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.170s	user 0.007s	sys 0.000s
I20260709 12:58:42.961719 13612 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:42.971052 19266 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:58:42.981752 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 52679.5us
I20260709 12:58:42.981491 13440 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 7 to 8, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:42.993541 14240 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:42.985409 19226 raft_consensus.cc:359] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:42.995886 19226 raft_consensus.cc:740] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:58:42.996652 19226 consensus_queue.cc:260] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:42.999459 19226 ts_tablet_manager.cc:1434] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.098s	user 0.002s	sys 0.003s
I20260709 12:58:43.001195 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-2f6f9228cbd74e95b5192c80f7f4a2b6 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:58:43.001685 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-2f6f9228cbd74e95b5192c80f7f4a2b6 on tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:43.014151 19266 tablet_bootstrap.cc:492] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:58:43.028059 19266 ts_tablet_manager.cc:1403] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.527s	user 0.033s	sys 0.015s
I20260709 12:58:43.033838 13702 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-921257a2480244e88f3ff036fb3be8d2 received from {username='slave'} at 127.0.0.1:40872
I20260709 12:58:43.034379 13702 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-921257a2480244e88f3ff036fb3be8d2 on tablet 921257a2480244e88f3ff036fb3be8d2 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:43.056136 19266 raft_consensus.cc:359] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:43.064679 19266 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:58:43.065416 19266 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:43.067416 19266 ts_tablet_manager.cc:1434] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.000s
I20260709 12:58:43.109176 14244 raft_consensus.cc:1217] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 12:58:43.112752 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-5c25c2050d4547c39924ceb739727ae8 received from {username='slave'} at 127.0.0.1:39902
I20260709 12:58:43.113296 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-5c25c2050d4547c39924ceb739727ae8 on tablet 5c25c2050d4547c39924ceb739727ae8 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:43.150851 13690 consensus_queue.cc:237] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:43.159942 14244 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:58:43.164480 13612 raft_consensus.cc:1275] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:58:43.164933 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:43.166715 19257 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:43.172960 19179 consensus_queue.cc:1048] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:43.187052 18882 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.192329 13612 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.197505 14240 raft_consensus.cc:2955] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.224830 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f97a6462cd4341ddbd7058ce9537c1eb with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:58:43.227331 13435 catalog_manager.cc:5697] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 8 to 9, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.235891 19280 raft_consensus.cc:1064] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:43.256902 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet f97a6462cd4341ddbd7058ce9537c1eb with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41508
I20260709 12:58:43.256574 19280 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 7, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:43.314811 14240 raft_consensus.cc:1217] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.8->[11.9-11.9]   Dedup: 11.9->[]
I20260709 12:58:43.329005 19292 tablet_replica.cc:333] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:43.330360 19292 raft_consensus.cc:2243] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:43.331351 19292 raft_consensus.cc:2272] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:43.372996 13616 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:58:43.374607 19179 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:43.376713 13542 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:58:43.378396 19292 ts_tablet_manager.cc:1916] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:58:43.378500 14240 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:58:43.380532 19179 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:43.381559 19280 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:43.485446 19257 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.492871 13542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:43.493706 13542 raft_consensus.cc:606] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Received request to transfer leadership to TS 058c47afa94546dba814cf345f414f4a
I20260709 12:58:43.505463 14240 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.530848 13540 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.537016 13542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47"
dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
mode: GRACEFUL
new_leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:32778
I20260709 12:58:43.537782 13542 raft_consensus.cc:606] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Received request to transfer leadership to TS d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:43.547734 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:58:43.524222 19292 ts_tablet_manager.cc:1929] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 9.8
I20260709 12:58:43.553328 19292 log.cc:1199] T f97a6462cd4341ddbd7058ce9537c1eb P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/f97a6462cd4341ddbd7058ce9537c1eb
I20260709 12:58:43.557565 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:43.559439 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet f97a6462cd4341ddbd7058ce9537c1eb (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:58:43.565831 13616 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.577417 13440 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.644115 13690 consensus_queue.cc:237] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:43.671795 13616 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:58:43.674916 18882 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:43.681313 14240 raft_consensus.cc:1275] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:58:43.692108 19262 consensus_queue.cc:1048] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:43.694078 13616 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.695715 18882 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.721542 19256 raft_consensus.cc:1064] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:43.724470 19256 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:43.729470 14240 raft_consensus.cc:2955] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.743161 13425 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 921257a2480244e88f3ff036fb3be8d2 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:58:43.763360 14244 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:43.765066 19258 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:43.759331 13440 catalog_manager.cc:5697] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 16 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.783229 19250 raft_consensus.cc:993] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:43.783731 19250 raft_consensus.cc:1081] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Signalling peer d35f063cd3f942b0bf559b7c6e9b299f to start an election
I20260709 12:58:43.786089 13617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
 from {username='slave'} at 127.0.0.1:56786
I20260709 12:58:43.786702 13617 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:43.787093 13617 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:43.792531 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet 921257a2480244e88f3ff036fb3be8d2 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:41508
I20260709 12:58:43.793640 19292 tablet_replica.cc:333] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:43.795300 19292 raft_consensus.cc:2243] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:43.796154 19292 raft_consensus.cc:2272] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:43.803238 13542 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:43.807200 19292 ts_tablet_manager.cc:1916] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:58:43.814867 19255 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:43.794950 13617 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:43.817565 13617 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:43.820420 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:43.821003 13543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:43.821604 13543 raft_consensus.cc:3055] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:58:43.821913 13543 raft_consensus.cc:740] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
I20260709 12:58:43.821030 14238 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:43.822942 13543 consensus_queue.cc:260] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:43.829468 13543 raft_consensus.cc:3060] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:58:43.841387 14238 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:58:43.845507 13584 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:43.850960 13543 raft_consensus.cc:2468] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 9.
I20260709 12:58:43.855105 19258 raft_consensus.cc:2804] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:58:43.855990 19258 raft_consensus.cc:697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:43.857821 19258 consensus_queue.cc:237] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:43.877386 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 82345.7us
I20260709 12:58:43.879330 13690 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:58:43.889288 19301 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.913370 14244 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.915299 19300 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:43.924804 19292 ts_tablet_manager.cc:1929] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.10
I20260709 12:58:43.925344 19292 log.cc:1199] T 921257a2480244e88f3ff036fb3be8d2 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/921257a2480244e88f3ff036fb3be8d2
I20260709 12:58:43.927044 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet 921257a2480244e88f3ff036fb3be8d2 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:58:43.941943 19256 raft_consensus.cc:1064] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:43.944691 19256 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:43.951725 13542 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.964599 13440 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: config changed from index 10 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:43.969007 13435 catalog_manager.cc:5697] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 8 to 9, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 9 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:43.973763 18974 raft_consensus.cc:993] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:43.974340 18974 raft_consensus.cc:1081] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:43.978458 13684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45312
I20260709 12:58:43.979094 13684 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:43.979483 13684 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:44.020907 13684 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:44.033545 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:44.034071 13684 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:44.034319 14244 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:44.040845 13542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:44.041972 13542 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:58:44.043777 13542 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ad2fdb1517c542439654bd3f7c72dad0, State: Running, Role: LEADER
W20260709 12:58:44.042502 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 89671.8us
I20260709 12:58:44.054025 14244 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:44.055824 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:44.060631 13542 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:44.062465 13542 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:44.063738 19257 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:44.064358 19257 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:44.065351 19257 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:44.076099 13542 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:44.101588 13690 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } attrs { replace: true } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.111328 14240 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:44.118772 19304 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:44.125793 13542 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:44.127619 19304 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:44.141378 13686 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:44.176082 13435 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:44.173370 19258 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:58:44.283634 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 109498us
I20260709 12:58:44.307125 13616 consensus_queue.cc:237] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:44.314034 14244 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:58:44.328161 19304 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:44.355466 14238 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:58:44.357339 19304 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.360067 19203 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.364692 14238 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.372138 19301 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:44.379549 13690 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.386909 13542 raft_consensus.cc:1275] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:58:44.399446 13543 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.406461 13435 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:44.416642 14240 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.418061 19258 consensus_queue.cc:1048] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:44.429638 13536 raft_consensus.cc:1275] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:58:44.442951 13612 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:44.444439 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:58:44.453990 19255 consensus_queue.cc:1048] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:44.467445 13440 catalog_manager.cc:5697] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 11 to 12, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:44.470775 13692 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:58:44.474114 13543 raft_consensus.cc:2955] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.485404 19256 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:44.492792 14242 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:44.493909 13683 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 with delete_type TABLET_DATA_TOMBSTONED (TS 058c47afa94546dba814cf345f414f4a not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:59318
I20260709 12:58:44.497335 14238 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:58:44.497014 19262 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:58:44.520903 19301 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:44.524267 19203 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Committing config change with OpId 11.11: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.556567 13690 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.560204 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:58:44.564154 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 47490.2us
I20260709 12:58:44.592536 13440 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:44.597996 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5c25c2050d4547c39924ceb739727ae8 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:58:44.647354 19319 tablet_replica.cc:333] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:58:44.700294 13525 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c25c2050d4547c39924ceb739727ae8 with delete_type TABLET_DATA_TOMBSTONED (TS ad2fdb1517c542439654bd3f7c72dad0 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:41508
I20260709 12:58:44.731756 13540 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:44.758805 18882 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:58:44.829309 19319 raft_consensus.cc:2243] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 12:58:44.867820 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 110854us
I20260709 12:58:44.878597 14240 raft_consensus.cc:2955] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:44.880379 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 108690us
I20260709 12:58:44.880254 19319 raft_consensus.cc:2272] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:44.883782 19330 tablet_replica.cc:333] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:44.937292 19319 ts_tablet_manager.cc:1916] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:58:44.942313 19330 raft_consensus.cc:2243] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:44.943302 19330 raft_consensus.cc:2272] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:44.948921 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 51841.7us
I20260709 12:58:44.948009 19330 ts_tablet_manager.cc:1916] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:58:45.084688 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 95319.9us
W20260709 12:58:45.089673 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 272350us
W20260709 12:58:45.093732 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 103762us
W20260709 12:58:45.192729 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 85276.8us
W20260709 12:58:45.315490 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 213771us
W20260709 12:58:45.387053 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 98245.6us
W20260709 12:58:45.521981 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 62570.1us
I20260709 12:58:45.538491 19330 ts_tablet_manager.cc:1929] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.10
I20260709 12:58:45.539023 19330 log.cc:1199] T 5c25c2050d4547c39924ceb739727ae8 P ad2fdb1517c542439654bd3f7c72dad0: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/wals/5c25c2050d4547c39924ceb739727ae8
W20260709 12:58:45.548058 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 62680.7us
I20260709 12:58:45.552482 19319 ts_tablet_manager.cc:1929] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 12:58:45.553018 19319 log.cc:1199] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 058c47afa94546dba814cf345f414f4a: Deleting WAL directory at /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/wals/2f6f9228cbd74e95b5192c80f7f4a2b6
I20260709 12:58:45.554111 19319 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59318 (request call id 6) took 1061 ms. Trace:
I20260709 12:58:45.554497 19319 rpcz_store.cc:276] 0709 12:58:44.493083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:44.493744 (+   661us) service_pool.cc:224] Handling call
0709 12:58:44.628200 (+134456us) ts_tablet_manager.cc:1151] Deleting tablet 2f6f9228cbd74e95b5192c80f7f4a2b6
0709 12:58:44.629910 (+  1710us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:58:45.231652 (+601742us) tablet_metadata.cc:665] Metadata flushed
0709 12:58:45.552336 (+320684us) tablet_metadata.cc:665] Metadata flushed
0709 12:58:45.554029 (+  1693us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49866,"tablet-delete.queue_time_us":113512,"thread_start_us":2542,"threads_started":1}
I20260709 12:58:45.556131 13425 catalog_manager.cc:5028] TS 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): tablet 2f6f9228cbd74e95b5192c80f7f4a2b6 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
I20260709 12:58:45.564400 13427 catalog_manager.cc:5028] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): tablet 5c25c2050d4547c39924ceb739727ae8 (table test-workload [id=a24a8d200f1a452bad6c157f8a71cf12]) successfully deleted
W20260709 12:58:45.612202 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 113349us
I20260709 12:58:45.746093 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:45.922968 13690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59336
I20260709 12:58:45.923731 13690 raft_consensus.cc:606] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:46.038261 13536 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:46.102151 13616 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
new_leader_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:55276
I20260709 12:58:46.102847 13616 raft_consensus.cc:606] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Received request to transfer leadership to TS 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:46.104820 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:58:46.107846 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:58:46.115666 13690 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:58:46.129037 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 63318.1us
I20260709 12:58:46.127764 19353 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:58:46.131618 13616 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:58:46.137465 19350 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:58:46.142588 19324 raft_consensus.cc:993] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:58:46.143218 19324 raft_consensus.cc:1081] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:58:46.145016 14238 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59162
I20260709 12:58:46.145599 14238 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:46.145969 14238 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:58:46.149130 13508 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c5bef52669b468c8765408698f5884d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:46.158183 14238 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:46.163345 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:46.164476 13616 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:58:46.164809 13616 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:46.165585 13616 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:46.167033 13616 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:46.173702 14238 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:46.174839 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:46.175566 13686 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:46.178244 13616 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:58:46.179592 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:46.180383 19331 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:58:46.181048 19331 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:46.181958 19331 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:46.181440 19353 raft_consensus.cc:2955] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.213958 19367 raft_consensus.cc:993] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: : Instructing follower 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:58:46.214550 19367 raft_consensus.cc:1081] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Signalling peer 0f34cda059124f839711dbef61bb2c28 to start an election
I20260709 12:58:46.215022 13616 raft_consensus.cc:2955] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.216337 14244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5"
dest_uuid: "0f34cda059124f839711dbef61bb2c28"
 from {username='slave'} at 127.0.0.1:59176
I20260709 12:58:46.217257 14244 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:46.217702 14244 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:46.218575 13440 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 15 to 16, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:46.207741 13686 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:58:46.230993 14244 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:46.239259 13536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "ad2fdb1517c542439654bd3f7c72dad0"
I20260709 12:58:46.239861 13536 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:46.240183 14244 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:46.246636 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:46.251204 13686 raft_consensus.cc:3055] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:58:46.229430 13690 raft_consensus.cc:2955] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.261278 13536 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:46.264530 13686 raft_consensus.cc:740] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:46.265592 14211 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0; no voters: 
I20260709 12:58:46.266562 19360 raft_consensus.cc:2804] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:46.267087 19360 raft_consensus.cc:697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:46.268101 19360 consensus_queue.cc:237] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:46.265555 13686 consensus_queue.cc:260] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:46.273087 13686 raft_consensus.cc:3060] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:46.286929 13436 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index -1 to 11, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 16 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.301592 13616 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:46.311301 13690 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:58:46.311800 13584 consensus_peers.cc:597] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6dd04e83486049ae859032792f652869. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:46.312163 13536 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:58:46.313776 19362 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:46.316327 19324 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:58:46.317763 13686 raft_consensus.cc:2468] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:46.351961 19362 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.360910 13536 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.374255 13692 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.488811 13435 catalog_manager.cc:5697] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 14, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 14 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:46.506493 13440 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 8, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.510439 13536 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:46.576897 19383 ts_tablet_manager.cc:933] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:58:46.601379 13688 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:58:46.608296 19331 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:58:46.640776 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 79154.9us
I20260709 12:58:46.692435 19383 tablet_copy_client.cc:323] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
I20260709 12:58:46.693817 13693 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:58:46.694410 13553 tablet_copy_service.cc:140] P ad2fdb1517c542439654bd3f7c72dad0: Received BeginTabletCopySession request for tablet 2c5bef52669b468c8765408698f5884d from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:45566)
I20260709 12:58:46.695004 13553 tablet_copy_service.cc:161] P ad2fdb1517c542439654bd3f7c72dad0: Beginning new tablet copy session on tablet 2c5bef52669b468c8765408698f5884d from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:45566: session id = 0f34cda059124f839711dbef61bb2c28-2c5bef52669b468c8765408698f5884d
I20260709 12:58:46.697654 19377 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:58:46.705183 13508 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56f36011554a4cd78d7b17b92a823c8f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:46.705855 13553 tablet_copy_source_session.cc:215] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:46.713167 13616 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:58:46.715582 13616 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:58:46.715775 19377 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:46.717190 19360 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:46.731061 19353 raft_consensus.cc:2955] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.741335 13687 raft_consensus.cc:2955] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.747859 19383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c5bef52669b468c8765408698f5884d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:46.759609 13616 raft_consensus.cc:2955] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:46.771622 13536 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:58:46.778667 19360 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:46.788635 13537 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:46.802204 13441 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 reported cstate change: config changed from index -1 to 12, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 18 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:46.808661 19383 tablet_copy_client.cc:806] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
W20260709 12:58:46.858989 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 90038.1us
I20260709 12:58:46.849066 19383 tablet_copy_client.cc:670] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
W20260709 12:58:46.919425 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 49474.8us
I20260709 12:58:47.018093 13688 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:58:47.034294 19345 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:58:47.037331 19383 tablet_copy_client.cc:538] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:58:47.042580 13611 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:58:47.058450 19377 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:58:47.071445 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 204751us
I20260709 12:58:47.101109 13687 raft_consensus.cc:1275] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:58:47.109045 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:58:47.122853 19383 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
W20260709 12:58:47.168632 13508 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c88adb75300c48e98d20928bce53551d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:47.171430 19402 consensus_queue.cc:1048] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:58:47.200651 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
I20260709 12:58:47.206012 19369 raft_consensus.cc:2955] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Committing config change with OpId 10.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:47.207904 13688 raft_consensus.cc:2955] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:47.214141 19413 ts_tablet_manager.cc:933] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:47.233367 13658 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:58:47.284601 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 117361us
I20260709 12:58:47.287639 13440 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New cstate: current_term: 10 leader_uuid: "ad2fdb1517c542439654bd3f7c72dad0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
W20260709 12:58:47.298064 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 49468.6us
I20260709 12:58:47.316104 19411 ts_tablet_manager.cc:933] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
W20260709 12:58:47.368661 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 71600.4us
W20260709 12:58:47.394663 13508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 63855.5us
I20260709 12:58:47.426608 19413 tablet_copy_client.cc:323] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.66:39367
W20260709 12:58:47.432111 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 47438.5us
I20260709 12:58:47.451287 13616 raft_consensus.cc:2955] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index -1 to 9, NON_VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } }
I20260709 12:58:47.458326 13629 tablet_copy_service.cc:140] P d35f063cd3f942b0bf559b7c6e9b299f: Received BeginTabletCopySession request for tablet 6dd04e83486049ae859032792f652869 from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:39902)
I20260709 12:58:47.458981 13629 tablet_copy_service.cc:161] P d35f063cd3f942b0bf559b7c6e9b299f: Beginning new tablet copy session on tablet 6dd04e83486049ae859032792f652869 from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:39902: session id = 0f34cda059124f839711dbef61bb2c28-6dd04e83486049ae859032792f652869
I20260709 12:58:47.491701 13629 tablet_copy_source_session.cc:215] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:47.506186 19413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd04e83486049ae859032792f652869. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:58:47.515000 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.515946 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.516844 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:47.510285 19411 tablet_copy_client.cc:323] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
W20260709 12:58:47.524351 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.535022 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.535732 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.537267 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.541891 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.545612 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.543727 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.547068 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.547511 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.552110 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.565443 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.575585 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.577273 13584 consensus_peers.cc:597] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.579144 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.579735 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.580322 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.582010 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.588454 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:47.593739 12409 tablet_copy_service.cc:362] P ad2fdb1517c542439654bd3f7c72dad0: Destroying tablet copy session 0f34cda059124f839711dbef61bb2c28-2c5bef52669b468c8765408698f5884d due to service shutdown
W20260709 12:58:47.594039 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:47.594219 12409 tablet_copy_service.cc:434] P ad2fdb1517c542439654bd3f7c72dad0: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-2c5bef52669b468c8765408698f5884d on tablet 2c5bef52669b468c8765408698f5884d with peer 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:47.594633 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.595196 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.610210 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.624239 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.623135 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.657339 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.721724 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.732851 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:47.736275 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:47.745642 19427 ts_tablet_manager.cc:933] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:58:47.831482 19427 tablet_copy_client.cc:323] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.65:39209
W20260709 12:58:47.834188 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 70406.7us
W20260709 12:58:47.843977 19383 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.112s	user 0.004s	sys 0.000s
I20260709 12:58:47.843993 19413 tablet_copy_client.cc:806] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:58:47.852661 19413 tablet_copy_client.cc:670] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:58:47.922832 19383 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:58:47.924211 19383 tablet_bootstrap.cc:492] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:58:47.925087 19383 ts_tablet_manager.cc:1403] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.802s	user 0.057s	sys 0.002s
I20260709 12:58:47.985934 19383 raft_consensus.cc:359] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:47.986953 19383 raft_consensus.cc:740] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
W20260709 12:58:48.075605 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 106763us
I20260709 12:58:47.987699 19383 consensus_queue.cc:260] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:48.099261 19413 tablet_copy_client.cc:538] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:58:48.118944 19377 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.119565 19377 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.124504 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.128798 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.132050 19377 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.160147 19424 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.170384 13506 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.177618 19422 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:48.206200 19369 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.207494 19369 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.174583 19424 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.218997 19369 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.225440 19424 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.220420 19413 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:58:48.230547 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.239655 19353 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.240435 19377 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.240386 19353 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.241353 19377 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.243259 19353 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.244390 19377 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.245739 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:48.214013 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 127663us
W20260709 12:58:48.275848 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 174166us
I20260709 12:58:48.282121 19377 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.284039 19377 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.286365 19377 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.286957 19377 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.287453 19377 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.306644 19383 ts_tablet_manager.cc:1434] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.375s	user 0.013s	sys 0.019s
I20260709 12:58:48.326813 14242 raft_consensus.cc:1217] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.10->[16.11-16.11]   Dedup: 16.11->[]
I20260709 12:58:48.327534 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.331609 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.335825 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.339504 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.343173 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.339715 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.362318 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.338879 19377 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.365047 13508 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.379191 19345 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:48.379442 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.379468 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:58:48.368376 19438 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.372906 19377 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.390781 12409 tablet_replica.cc:333] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:48.397459 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.377712 19422 raft_consensus.cc:493] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.410362 19422 raft_consensus.cc:515] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.416282 12409 raft_consensus.cc:2243] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:48.420532 12409 raft_consensus.cc:2272] T 3f7e1f6a8119482c9f8056bb615edfa8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:48.423504 12409 tablet_replica.cc:333] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:48.425688 19369 raft_consensus.cc:493] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.428805 19353 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.429353 19353 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:58:48.432531 12409 raft_consensus.cc:2243] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:58:48.432890 19422 leader_election.cc:290] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.433597 19353 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:48.433893 12409 raft_consensus.cc:2272] T c665edf3d7834e18a67df3f75b30856a P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:48.411803 19377 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.442842 19377 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.444967 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.451752 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.452965 12409 tablet_replica.cc:333] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:48.453949 12409 raft_consensus.cc:2243] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:48.426322 19369 raft_consensus.cc:515] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.454543 12409 raft_consensus.cc:2272] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:48.455727 19422 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.456629 19422 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.398408 19438 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.490520 19422 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.491731 13506 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:48.490808 12409 tablet_replica.cc:333] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:48.456652 19369 leader_election.cc:290] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.494445 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.500140 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.501163 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.501865 12409 raft_consensus.cc:2243] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:58:48.511734 19369 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:48.513216 13506 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.514437 13506 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:48.515071 13506 raft_consensus.cc:2686] T 1c2d2db0b5674796bbd0426953d99538 P ad2fdb1517c542439654bd3f7c72dad0: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:58:48.515944 13506 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:48.523157 19424 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:58:48.523368 19353 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:48.525445 19377 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.525993 19377 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:48.528191 19377 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.530201 19438 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.530522 13506 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.533039 19422 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:48.535038 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.549455 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.553087 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.553709 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:48.554028 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 157865us
I20260709 12:58:48.556510 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f54dde33fa845098e64673b576d780c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.557741 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.557253 19424 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.559444 19424 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.559870 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.562806 19424 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.564035 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.565299 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f9e2a62b8a498d93662b589c6a6870" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.566721 13509 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.558743 19369 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.569438 19438 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.569684 19369 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.570017 19438 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.572120 19345 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.572711 19345 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.574888 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.574951 19345 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.575728 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.575858 19369 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.576807 19353 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.577308 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.577376 19353 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.581281 19437 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:48.582064 19437 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.582557 19437 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.584942 19437 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.586396 19353 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.586601 13506 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:48.587553 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.588210 19377 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:58:48.639881 13509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 73161.4us
W20260709 12:58:48.698850 13506 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 110615us
I20260709 12:58:48.700448 19424 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.702281 19424 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.590651 19422 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:48.704486 19422 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.685158 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.702782 13506 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:48.708839 19438 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.722091 19424 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.724871 19437 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.719521 19422 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.728623 19424 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:48.733327 19439 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.743166 19439 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.744930 19437 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.746142 19439 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.737171 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.737584 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.747666 19437 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.763978 19377 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.764861 19377 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.770321 19377 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.773942 19424 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.774672 19369 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:58:48.793040 19424 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:48.793190 19369 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.794474 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.795463 19424 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:48.795504 19369 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.778533 12409 raft_consensus.cc:2272] T 1d73171142594388972ed22eae44f95f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:48.780236 13509 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.797133 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.797868 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:48.786772 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.798748 13509 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.800822 13509 leader_election.cc:304] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.803755 19422 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:48.804780 19422 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:48.805842 19442 raft_consensus.cc:2749] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:58:48.808964 13509 leader_election.cc:304] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:48.810118 19424 raft_consensus.cc:2749] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:58:48.815446 12409 tablet_replica.cc:333] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:48.824330 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.824729 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.827031 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.828653 19377 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.830189 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.830271 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.834069 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.829726 19377 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.838380 12409 raft_consensus.cc:2243] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:48.846457 12409 raft_consensus.cc:2272] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:48.846549 19377 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:48.848788 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 75541.3us
W20260709 12:58:48.853909 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 101350us
I20260709 12:58:48.882243 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.884518 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.886755 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.900962 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.905180 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.907480 19442 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.908442 19442 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.911801 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.912643 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.914598 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.916926 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.917450 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.926350 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.911062 19442 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.936968 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:48.939815 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:48.940344 12409 tablet_replica.cc:333] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:48.940268 13508 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:48.941465 19422 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:48.942442 19422 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:48.942907 19369 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:48.945894 19422 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:48.947228 13508 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:48.948055 13508 raft_consensus.cc:2686] T 75988f1a2dda4bd6962cd5f3ad1c304a P ad2fdb1517c542439654bd3f7c72dad0: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:58:48.949148 13508 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:48.950939 19442 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:48.973542 12409 raft_consensus.cc:2243] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:58:48.985046 19413 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:58:48.992478 12409 raft_consensus.cc:2272] T 56f36011554a4cd78d7b17b92a823c8f P ad2fdb1517c542439654bd3f7c72dad0 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:48.992767 13508 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:48.993757 19424 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:48.996584 12409 tablet_replica.cc:333] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:48.998155 12409 raft_consensus.cc:2243] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:48.999277 13509 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.000389 19377 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:49.001492 13509 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.002452 19377 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:58:49.003506 13509 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.004554 19377 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:48.999322 12409 raft_consensus.cc:2272] T 835fe463242d40e5b82ddb5898323fc5 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.009810 13509 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.016300 19377 raft_consensus.cc:1064] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:49.019369 19413 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:58:49.020625 19413 tablet_bootstrap.cc:492] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:58:49.021811 19413 ts_tablet_manager.cc:1403] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.802s	user 0.047s	sys 0.012s
I20260709 12:58:49.022220 13509 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.029047 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:49.029899 19442 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:49.031204 13509 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.019650 19377 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:49.032919 19424 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:49.034834 13509 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.036343 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:49.028741 19369 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:49.039285 19369 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:49.041941 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "ad2fdb1517c542439654bd3f7c72dad0" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:58:49.043192 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:49.052127 12409 tablet_replica.cc:333] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.053607 13509 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.053926 12409 raft_consensus.cc:2243] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.055699 13509 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.056722 12409 raft_consensus.cc:2272] T d9eb8a2c5d7846099d30c0839aefc5a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.057206 19369 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:49.058516 19424 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:58:49.065822 12409 tablet_replica.cc:333] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.067569 12409 raft_consensus.cc:2243] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.065162 13509 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.069360 19369 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:58:49.074077 12409 raft_consensus.cc:2272] T 3f54dde33fa845098e64673b576d780c P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.081740 12409 tablet_replica.cc:333] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.077522 13509 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.081979 19413 raft_consensus.cc:359] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:49.082834 19369 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:58:49.083065 19413 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:58:49.084519 13509 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad2fdb1517c542439654bd3f7c72dad0; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.092730 19369 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:49.086169 12409 raft_consensus.cc:2243] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.084069 19413 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:49.100730 19413 ts_tablet_manager.cc:1434] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.079s	user 0.010s	sys 0.002s
I20260709 12:58:49.101642 12409 raft_consensus.cc:2272] T 9aa892a8dfec433d8aa859d442e3e309 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.105278 12409 tablet_replica.cc:333] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.106978 12409 raft_consensus.cc:2243] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.109995 13629 tablet_copy_service.cc:342] P d35f063cd3f942b0bf559b7c6e9b299f: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-6dd04e83486049ae859032792f652869 received from {username='slave'} at 127.0.0.1:39902
W20260709 12:58:49.110450 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 102786us
I20260709 12:58:49.110735 13629 tablet_copy_service.cc:434] P d35f063cd3f942b0bf559b7c6e9b299f: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-6dd04e83486049ae859032792f652869 on tablet 6dd04e83486049ae859032792f652869 with peer 0f34cda059124f839711dbef61bb2c28
I20260709 12:58:49.107878 12409 raft_consensus.cc:2272] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.122776 12409 tablet_replica.cc:333] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.123883 12409 raft_consensus.cc:2243] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.128638 12409 raft_consensus.cc:2272] T 9d739d3dda4e4370b33be67fca1c66d5 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.136600 12409 tablet_replica.cc:333] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.138047 12409 raft_consensus.cc:2243] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.139444 12409 raft_consensus.cc:2272] T 86d7b5c530f14daaade417c2e3903072 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.148537 12409 tablet_replica.cc:333] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.151495 13688 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:58:49.152019 13612 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:58:49.151543 14242 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 LEARNER]: Refusing update from remote peer ad2fdb1517c542439654bd3f7c72dad0: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:58:49.153204 19424 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:49.155325 19424 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:49.161233 19377 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:49.149696 12409 raft_consensus.cc:2243] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.162989 12409 pending_rounds.cc:70] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Trying to abort 1 pending ops.
I20260709 12:58:49.163388 12409 pending_rounds.cc:77] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0: Aborting op as it isn't in flight: id { term: 16 index: 12 } timestamp: 7305633501256400896 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "2c5bef52669b468c8765408698f5884d" old_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } } } new_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } } }
I20260709 12:58:49.164790 12409 raft_consensus.cc:2917] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: Aborting config change with OpId 16.12: Aborted: Op aborted
I20260709 12:58:49.165267 12409 raft_consensus.cc:2889] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:58:49.165692 12409 raft_consensus.cc:2272] T 2c5bef52669b468c8765408698f5884d P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.173995 14244 raft_consensus.cc:1217] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 12:58:49.208803 12409 tablet_replica.cc:333] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.266495 12409 raft_consensus.cc:2243] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.311679 12409 raft_consensus.cc:2272] T c827d49183b8406b9adff9666f6d83a9 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:49.335209 19447 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.102s	user 0.003s	sys 0.000s
I20260709 12:58:49.403234 12409 tablet_replica.cc:333] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.404456 12409 raft_consensus.cc:2243] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.405830 12409 raft_consensus.cc:2272] T 019c73868e744f4eba4ac7d97d34a137 P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.409058 12409 tablet_replica.cc:333] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.410061 12409 raft_consensus.cc:2243] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.411545 12409 raft_consensus.cc:2272] T 85eb0ac09bf7466899637997aae42111 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.414819 12409 tablet_replica.cc:333] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.416118 12409 raft_consensus.cc:2243] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.417172 12409 raft_consensus.cc:2272] T f125a6a76f974409807014b9ddaa1b8c P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.421164 12409 tablet_replica.cc:333] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.422266 12409 raft_consensus.cc:2243] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.423153 12409 raft_consensus.cc:2272] T 1c9f20cfde5343c9b91c4e2adc5320ce P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.427156 12409 tablet_replica.cc:333] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.428268 12409 raft_consensus.cc:2243] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.429642 12409 raft_consensus.cc:2272] T be389b2236484464bc3e9f558d7b4230 P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:49.428102 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 106845us
W20260709 12:58:49.433950 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 169426us
I20260709 12:58:49.437831 12409 tablet_replica.cc:333] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.439096 12409 raft_consensus.cc:2243] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.444561 12409 raft_consensus.cc:2272] T f9f9e2a62b8a498d93662b589c6a6870 P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.460947 12409 tablet_replica.cc:333] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.462597 12409 raft_consensus.cc:2243] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.463896 12409 raft_consensus.cc:2272] T 6af1705dc4aa4e738a8416263f5af754 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.467197 12409 tablet_replica.cc:333] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.468293 12409 raft_consensus.cc:2243] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.474309 12409 raft_consensus.cc:2272] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.488250 12409 tablet_replica.cc:333] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.489267 12409 raft_consensus.cc:2243] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.495095 12409 raft_consensus.cc:2272] T 6b48fb5973b84b06a9ad858a9b3967f8 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.501221 12409 tablet_replica.cc:333] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.502300 12409 raft_consensus.cc:2243] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.502959 12409 raft_consensus.cc:2272] T 0863151795784affb539e2b7a55fc2b8 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.506176 12409 tablet_replica.cc:333] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.507222 12409 raft_consensus.cc:2243] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.508100 12409 raft_consensus.cc:2272] T 32b8156356cf4b709a2903d6861a117c P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.511435 12409 tablet_replica.cc:333] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.512555 12409 raft_consensus.cc:2243] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.513305 12409 raft_consensus.cc:2272] T 87a8c869a6b840e591423340bc32af00 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.517002 12409 tablet_replica.cc:333] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.517928 12409 raft_consensus.cc:2243] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.519241 12409 raft_consensus.cc:2272] T f2ce7a788b8c40629fc1921a4286d084 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.522699 12409 tablet_replica.cc:333] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.523674 12409 raft_consensus.cc:2243] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.524816 12409 raft_consensus.cc:2272] T 1a689534693d4469befa622d6b4917e3 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.528755 12409 tablet_replica.cc:333] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.530109 12409 raft_consensus.cc:2243] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.532015 12409 raft_consensus.cc:2272] T c88adb75300c48e98d20928bce53551d P ad2fdb1517c542439654bd3f7c72dad0 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.535889 12409 tablet_replica.cc:333] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.536824 12409 raft_consensus.cc:2243] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.540694 12409 raft_consensus.cc:2272] T 048528e413ac4c67a1cfd3dfe70673c7 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.545357 12409 tablet_replica.cc:333] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.546305 12409 raft_consensus.cc:2243] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.547590 12409 raft_consensus.cc:2272] T 82077fd60c1346d7b057a65a778ee2f5 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.551680 12409 tablet_replica.cc:333] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.552616 12409 raft_consensus.cc:2243] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.553923 12409 raft_consensus.cc:2272] T 5406eb61fd534169b3c1645f80144779 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.557379 12409 tablet_replica.cc:333] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.558328 12409 raft_consensus.cc:2243] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.559479 12409 raft_consensus.cc:2272] T 973866cfc22e48ea931cb1a6fb16872f P ad2fdb1517c542439654bd3f7c72dad0 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.562697 12409 tablet_replica.cc:333] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.563598 12409 raft_consensus.cc:2243] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.564879 12409 raft_consensus.cc:2272] T 72d95a7092b0445eb00a28a1fdb8d239 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.567595 19400 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:49.568491 19400 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:49.569520 12409 tablet_replica.cc:333] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.570524 12409 raft_consensus.cc:2243] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.571455 12409 raft_consensus.cc:2272] T 6dd04e83486049ae859032792f652869 P ad2fdb1517c542439654bd3f7c72dad0 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.574846 12409 tablet_replica.cc:333] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.575989 12409 raft_consensus.cc:2243] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.577016 12409 raft_consensus.cc:2272] T 2ee08c4df55f41c7a18951168c44fb5a P ad2fdb1517c542439654bd3f7c72dad0 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.580225 12409 tablet_replica.cc:333] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.581117 12409 raft_consensus.cc:2243] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:58:49.582382 12409 raft_consensus.cc:2272] T be11851ed11044eea9f0fc3dbdbe0599 P ad2fdb1517c542439654bd3f7c72dad0 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.588207 12409 tablet_replica.cc:333] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.589345 12409 raft_consensus.cc:2243] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.591409 12409 raft_consensus.cc:2272] T 86a0c010e8cc4b828fec8e8547e37902 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:49.599579 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:49.600024 19400 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:49.606410 19444 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:49.607059 19444 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:49.611399 13584 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:49.612154 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:49.615859 13690 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:58:49.618240 12409 tablet_replica.cc:333] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.627440 19444 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:49.630265 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:49.631002 13619 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:49.632285 12409 raft_consensus.cc:2243] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.633152 12409 raft_consensus.cc:2272] T 12c636bac0584e2e92ff66a089d880ee P ad2fdb1517c542439654bd3f7c72dad0 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.633641 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:49.634505 19444 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:49.634920 19444 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:49.635418 19444 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:49.636349 12409 tablet_replica.cc:333] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.637158 12409 raft_consensus.cc:2243] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.637876 12409 raft_consensus.cc:2272] T 91a3f49b0e164d27994cfe440af6f514 P ad2fdb1517c542439654bd3f7c72dad0 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.657141 12409 tablet_replica.cc:333] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.658020 12409 raft_consensus.cc:2243] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.658740 12409 raft_consensus.cc:2272] T 35f2068be68d418fb0b123e59c3db757 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:49.661545 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:49.668645 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:49.672508 19407 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:49.672979 19407 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:49.673334 19407 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:49.690502 19407 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:49.692827 19444 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:49.695957 12409 tablet_replica.cc:333] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.696815 12409 raft_consensus.cc:2243] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.696897 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:49.697723 13619 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 17: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:49.699105 19407 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:49.703658 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:49.704568 13688 raft_consensus.cc:2393] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 17: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:58:49.705381 13658 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:58:49.712936 13584 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:49.697723 12409 raft_consensus.cc:2272] T 68566bdf08224971a44a28240a8bf9e4 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.695652 19444 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:49.732827 12409 tablet_replica.cc:333] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.733946 12409 raft_consensus.cc:2243] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.735299 12409 raft_consensus.cc:2272] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P ad2fdb1517c542439654bd3f7c72dad0 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.735845 19443 raft_consensus.cc:1064] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:49.747689 19400 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:49.748538 19400 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:49.755529 13584 leader_election.cc:336] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:49.739348 19443 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:49.756176 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:49.759490 19407 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:58:49.763185 19400 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:49.765031 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:49.767105 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:49.770328 19400 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:49.778403 12409 tablet_replica.cc:333] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.779366 12409 raft_consensus.cc:2243] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.787487 12409 raft_consensus.cc:2272] T e7d1ab18f48a4689857964b7200b72c9 P ad2fdb1517c542439654bd3f7c72dad0 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:58:49.791813 14242 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 LEARNER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:58:49.798442 19384 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:49.800961 12409 tablet_replica.cc:333] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0: stopping tablet replica
I20260709 12:58:49.808573 12409 raft_consensus.cc:2243] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:58:49.809408 12409 raft_consensus.cc:2272] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P ad2fdb1517c542439654bd3f7c72dad0 [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 12:58:49.810693 13658 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:49.811276 13658 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:49.812137 19444 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:58:49.818589 13688 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:58:49.821663 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:49.821513 19384 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:58:49.824924 13584 consensus_peers.cc:597] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:49.828297 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:49.843477 19331 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:49.844175 19331 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:49.843526 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:49.848425 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:49.849007 19331 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:49.849347 13612 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:49.851272 14212 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:49.852342 19331 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:49.852767 19331 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:49.856328 19331 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:49.855772 14211 leader_election.cc:336] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:58:49.871953 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:49.921838 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:49.928807 19331 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:49.934754 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:49.933938 19443 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:49.952548 19331 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:49.953258 14211 leader_election.cc:336] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:49.953917 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:49.954633 13619 raft_consensus.cc:3060] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:49.961971 14242 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:49.970202 13619 raft_consensus.cc:2468] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:49.979351 13688 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:49.990152 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:49.990836 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:49.992470 14212 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:49.990598 13438 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: config changed from index 8 to 9, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:49.993307 19331 raft_consensus.cc:2804] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:49.998091 19331 raft_consensus.cc:697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:49.999070 19331 consensus_queue.cc:237] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:50.022017 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.039062 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.039844 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.048731 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.065788 19407 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.067317 19407 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:58:50.106091 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.107589 19407 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:50.111280 13584 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:50.115187 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:50.117575 13582 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.119282 19400 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:58:50.120114 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.122774 13507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:58:50.165242 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.177989 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.197572 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.218768 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.241432 13619 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:50.263229 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.276016 13441 catalog_manager.cc:5697] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:58:50.309047 14244 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:50.312692 19443 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:50.313171 13688 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:58:50.314472 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.316205 19443 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:50.326794 13692 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:50.331877 19400 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:50.350732 13441 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 9 to 10, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:50.355584 14244 raft_consensus.cc:2955] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:50.358129 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.373508 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.389875 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.400827 13428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6dd04e83486049ae859032792f652869 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:58:50.414903 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.432685 19384 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.433888 19458 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.434451 19458 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.436738 19458 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:50.438282 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:58:50.441066 13658 leader_election.cc:336] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:58:50.449447 13509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:58:50.433409 19384 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.450673 13619 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:50.451398 19458 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.452009 19458 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:50.454317 19458 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:50.456555 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.456984 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:50.457628 13619 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 18.
W20260709 12:58:50.458630 13658 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:50.462766 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.463636 19356 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:50.464072 19356 raft_consensus.cc:493] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.464450 19356 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:58:50.464267 13584 leader_election.cc:336] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:50.467648 19458 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:58:50.469029 19458 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.470199 19458 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:58:50.472663 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:50.474953 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.475514 19384 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:50.485548 13427 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59819.5us
I20260709 12:58:50.496572 13611 raft_consensus.cc:1275] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:58:50.498893 13427 catalog_manager.cc:4755] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6dd04e83486049ae859032792f652869: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 12:58:50.499461 19331 consensus_queue.cc:1048] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:58:50.501104 19451 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.501680 19451 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:50.510424 14211 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:50.512283 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:50.513845 14212 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:50.511658 19458 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:50.515321 19331 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:50.518734 19458 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:50.518890 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:50.523345 13612 raft_consensus.cc:3060] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:58:50.533331 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:50.533442 19451 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.536221 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:58:50.540071 13658 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:58:50.550835 19458 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.551453 19458 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:50.556238 19458 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:50.557015 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:50.536299 13612 raft_consensus.cc:2468] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 19.
I20260709 12:58:50.561446 13659 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.565923 19444 raft_consensus.cc:2804] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:58:50.566552 19444 raft_consensus.cc:697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 19 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:50.477288 19356 raft_consensus.cc:515] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.577455 13692 raft_consensus.cc:2393] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 17: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:50.581393 13582 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.584615 19384 raft_consensus.cc:3060] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:58:50.586402 13658 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.588074 13658 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:50.597471 19400 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.598148 19400 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.617425 19458 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:58:50.567510 19444 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:50.630896 19443 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.647023 19443 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.648139 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86d7b5c530f14daaade417c2e3903072" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:50.651490 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:50.623960 19356 leader_election.cc:290] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:50.669755 19473 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.630004 19400 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:50.686648 13658 leader_election.cc:336] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.687372 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:50.649823 19443 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:50.674113 19476 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.679243 19451 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.693312 19476 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.708393 19458 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.696980 19473 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:50.708259 19451 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.722108 19458 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.723471 19384 raft_consensus.cc:2749] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:50.724164 19473 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.726449 19476 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:50.743177 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 46372.6us
I20260709 12:58:50.746637 19451 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.749512 19479 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.755507 19458 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:50.767679 13619 raft_consensus.cc:2468] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:50.768585 13441 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 18 to 19, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 19 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:50.752060 19479 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:50.784091 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 75803us
W20260709 12:58:50.796976 13584 leader_election.cc:336] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.800398 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:50.803134 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.803391 14242 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
W20260709 12:58:50.804529 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.805305 13584 leader_election.cc:336] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.805816 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.810461 13659 leader_election.cc:304] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.813691 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:50.817070 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:50.817800 14244 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:58:50.822365 19480 raft_consensus.cc:2804] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:50.823302 19480 raft_consensus.cc:697] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:50.808297 19384 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:50.812922 19475 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:50.830184 19478 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.830811 19478 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.833180 19478 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.832747 19480 consensus_queue.cc:237] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.795089 19479 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.852803 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:50.853633 13619 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:50.856544 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:50.864830 19411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59164 (request call id 1678) took 3701 ms. Trace:
W20260709 12:58:50.865372 13584 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.865368 19411 rpcz_store.cc:276] 0709 12:58:47.163151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:47.163251 (+   100us) service_pool.cc:224] Handling call
0709 12:58:47.323685 (+160434us) ts_tablet_manager.cc:934] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
0709 12:58:47.398782 (+ 75097us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02b6a80ac 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c1caa 00007fa02b6c1bd2 00007fa033726131
0709 12:58:50.864566 (+3465784us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":19855,"spinlock_wait_cycles":4280576,"tablet-copy.queue_time_us":72647,"thread_start_us":213873,"threads_started":2}
W20260709 12:58:50.861491 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.869349 14209 connection.cc:466] server connection from 127.0.0.1:59164 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59164 (request call id 1678) could send its response
I20260709 12:58:50.872453 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:50.873240 13692 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 10.
I20260709 12:58:50.873843 19475 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.874389 19475 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.873869 13659 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:50.882077 19444 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:50.882531 19444 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.882985 19444 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:50.884630 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:50.890208 19475 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.891290 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:50.894276 19444 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.895406 19458 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.896245 13692 raft_consensus.cc:2393] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 17: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:50.898012 19356 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.898571 19356 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.898741 19458 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.900591 19470 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.900753 19356 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:50.900727 19458 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:50.901242 19470 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.901782 19450 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.902662 19450 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.903613 19470 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:50.905483 19356 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.908358 19356 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:50.908995 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:50.906235 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:50.912398 13688 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:50.910389 19356 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:50.905076 19450 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:50.917021 19383 tablet_copy_client.cc:1131] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.917938 19427 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59164 (request call id 1685) took 3396 ms. Trace:
I20260709 12:58:50.918401 19427 rpcz_store.cc:276] 0709 12:58:47.521521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:58:47.521673 (+   152us) service_pool.cc:224] Handling call
0709 12:58:47.746076 (+224403us) ts_tablet_manager.cc:934] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
0709 12:58:50.917720 (+3171644us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":281,"tablet-copy.queue_time_us":164335,"thread_start_us":202209,"threads_started":2}
W20260709 12:58:50.923581 14209 connection.cc:466] server connection from 127.0.0.1:59164 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:59164 (request call id 1685) could send its response
I20260709 12:58:50.928488 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.931515 19444 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:50.932322 19478 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.933197 19478 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.934314 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:50.934983 13617 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:50.935534 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:50.936033 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:50.936192 13611 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:50.936475 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:50.936705 13612 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:50.937075 13619 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
W20260709 12:58:50.937374 13584 leader_election.cc:336] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:50.937673 14211 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.938158 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:50.938380 13584 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.944983 19482 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:58:50.945715 19482 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.945922 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:50.945112 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:50.946774 14211 leader_election.cc:336] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.947220 19443 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:58:50.947206 14211 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.946084 19482 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:50.951081 19481 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:50.948770 13582 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.951884 19481 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:58:50.954419 19478 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.952539 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:50.955294 19444 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:58:50.955941 19444 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.956358 19444 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:50.961099 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:50.961552 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:50.962980 19478 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.963436 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:50.963708 19478 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:50.964331 19450 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:50.964743 19450 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.965113 19450 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:50.966347 19478 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:50.967157 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:50.967804 14244 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:50.970929 19480 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:50.971390 19480 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.972110 19480 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:50.972903 13441 catalog_manager.cc:5697] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 16 to 17, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
W20260709 12:58:50.976692 13584 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:50.982600 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:50.983379 19481 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.984443 19479 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.984555 19481 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.983963 19478 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:58:50.985059 19479 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:50.985512 19478 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:50.986092 19478 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:58:50.987937 19479 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:50.993309 14211 leader_election.cc:336] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:50.993878 14211 leader_election.cc:336] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.001577 19450 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.004680 19481 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:51.005505 19356 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.005985 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:51.006078 19356 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.006822 13619 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:58:51.008199 19356 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:51.008255 14212 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.009609 19481 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:51.009393 19444 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.010123 19481 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.010583 19481 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:51.010720 13611 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:50.989864 19482 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:51.015761 19384 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:58:51.016099 19482 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:51.025147 19444 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:51.029574 19479 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.030241 19479 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.032444 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:51.033346 19444 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:51.033738 19444 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.034123 19444 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:51.037181 14211 leader_election.cc:336] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.037920 19481 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.039202 19479 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:51.025444 13584 leader_election.cc:336] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.040781 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:51.041430 13611 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
W20260709 12:58:51.045437 14211 leader_election.cc:336] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.042215 19450 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:51.049480 14212 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.050526 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:51.050874 19433 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:51.051179 13688 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:51.051304 19433 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.051766 19433 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:51.010125 19480 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.056030 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:51.056241 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:51.056630 13612 raft_consensus.cc:3060] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:51.057008 19480 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:51.059749 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:51.060534 13688 raft_consensus.cc:2393] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 15: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:51.061405 19433 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.065959 13612 raft_consensus.cc:2468] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 18.
W20260709 12:58:51.068540 14211 leader_election.cc:336] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.063966 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:58:51.068737 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:51.069710 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:51.075798 13616 raft_consensus.cc:3060] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:51.075882 13692 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:58:51.077831 13688 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:58:51.070495 19433 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:51.071050 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:51.077467 19444 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.084514 14244 raft_consensus.cc:3060] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:51.084697 13616 raft_consensus.cc:2468] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 15.
I20260709 12:58:51.085861 19444 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:58:51.088300 13584 leader_election.cc:336] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.069710 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:51.078724 19481 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:51.104393 13611 raft_consensus.cc:3060] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:51.079805 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:58:51.080766 13658 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.107120 13617 raft_consensus.cc:3060] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:51.109685 13659 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:51.110507 19444 raft_consensus.cc:2804] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:51.110914 13659 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:51.111897 19480 raft_consensus.cc:2804] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:58:51.112398 19480 raft_consensus.cc:697] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 18 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:51.113229 19480 consensus_queue.cc:237] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.115566 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:58:51.116271 14238 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:51.110991 19444 raft_consensus.cc:697] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:51.119648 19444 consensus_queue.cc:237] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.120976 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.122083 13612 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:58:51.122987 13617 raft_consensus.cc:2468] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:58:51.126230 14212 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.126585 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.127367 19433 raft_consensus.cc:2804] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:51.127563 19356 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:58:51.127978 19433 raft_consensus.cc:697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:51.127588 19384 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:58:51.129236 19433 consensus_queue.cc:237] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:51.130091 13584 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.092559 14244 raft_consensus.cc:2468] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:51.094723 13692 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 11.
I20260709 12:58:51.122125 19481 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:58:51.136709 19481 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.137148 13611 raft_consensus.cc:2468] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:58:51.137285 19481 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:51.139050 14212 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.071198 19478 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.141024 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.150784 19384 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:58:51.151643 19384 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.152554 19384 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:51.144057 19451 raft_consensus.cc:2804] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:51.156281 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:58:51.159286 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:51.157063 14244 raft_consensus.cc:3060] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:51.163246 13658 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:51.164083 13658 leader_election.cc:336] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.164505 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:51.165526 13658 leader_election.cc:336] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:51.166323 13658 leader_election.cc:336] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:51.167033 13658 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:51.167802 13658 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.171365 13441 catalog_manager.cc:5697] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 14 to 15, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 15 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:51.173458 14244 raft_consensus.cc:2468] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:51.178516 13619 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 058c47afa94546dba814cf345f414f4a in current term 16: Already voted for candidate d35f063cd3f942b0bf559b7c6e9b299f in this term.
I20260709 12:58:51.168943 19444 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:58:51.185412 13582 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.170081 19451 raft_consensus.cc:697] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:51.201705 19474 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:58:51.236243 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46752.1us
W20260709 12:58:51.243074 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 53089us
I20260709 12:58:51.252954 19451 consensus_queue.cc:237] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.238186 19474 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:51.225265 19481 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.275460 19481 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:51.275342 19474 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:51.280997 19478 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:51.284257 19384 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:51.297653 14211 leader_election.cc:336] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.279814 13440 catalog_manager.cc:5697] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:51.312940 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 74102.2us
I20260709 12:58:51.314679 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:51.321376 19444 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:58:51.321831 19444 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (detected failure of leader ad2fdb1517c542439654bd3f7c72dad0)
I20260709 12:58:51.322199 19444 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:58:51.324632 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 56309.2us
I20260709 12:58:51.345840 19444 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.347853 13441 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 10 to 11, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 11 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:51.349283 19444 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:58:51.350595 19384 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:51.354171 13658 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:51.352177 13584 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.359377 13658 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
W20260709 12:58:51.363204 13658 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.363735 13658 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:58:51.363860 19480 raft_consensus.cc:2804] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:58:51.364653 19480 raft_consensus.cc:697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:58:51.364830 13658 leader_election.cc:336] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.365159 19458 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:58:51.365612 19480 consensus_queue.cc:237] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.365700 13440 catalog_manager.cc:5697] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 14 to 15, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 15 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:51.365741 13658 leader_election.cc:336] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:51.368623 13658 leader_election.cc:336] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.358884 19490 raft_consensus.cc:2804] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:51.369495 19490 raft_consensus.cc:697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:51.370458 19490 consensus_queue.cc:237] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:51.374943 13658 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:51.376993 13584 leader_election.cc:336] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.390038 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:58:51.390966 14244 raft_consensus.cc:3060] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:58:51.426707 13611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c88adb75300c48e98d20928bce53551d"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55274
I20260709 12:58:51.427500 13611 raft_consensus.cc:606] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Received request to transfer leadership
I20260709 12:58:51.429418 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:51.430351 13690 raft_consensus.cc:2393] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d35f063cd3f942b0bf559b7c6e9b299f in current term 16: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:58:51.432278 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:51.432893 13692 raft_consensus.cc:3060] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:58:51.436129 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:58:51.436774 13693 raft_consensus.cc:3060] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:58:51.440448 14244 raft_consensus.cc:2468] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:58:51.442010 13611 raft_consensus.cc:1275] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:58:51.449638 13693 raft_consensus.cc:2468] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:58:51.450345 19491 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:51.450924 19491 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.453145 13617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c88adb75300c48e98d20928bce53551d"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55274
I20260709 12:58:51.453346 19491 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:51.453708 13617 raft_consensus.cc:606] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Received request to transfer leadership
I20260709 12:58:51.459514 13617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c88adb75300c48e98d20928bce53551d"
dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55274
I20260709 12:58:51.460117 13617 raft_consensus.cc:606] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Received request to transfer leadership
W20260709 12:58:51.467587 13489 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:58:51.468052 13489 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for c88adb75300c48e98d20928bce53551d already in progress
W20260709 12:58:51.468323 13489 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for c88adb75300c48e98d20928bce53551d already in progress
I20260709 12:58:51.473345 19480 consensus_queue.cc:1048] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:58:51.477020 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:51.477895 13612 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:51.478156 13692 raft_consensus.cc:2468] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
I20260709 12:58:51.486601 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.489493 19493 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:58:51.514673 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56f36011554a4cd78d7b17b92a823c8f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:51.516253 13658 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:58:51.523164 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:58:51.523706 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:51.524111 19491 raft_consensus.cc:2804] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:58:51.526574 13658 leader_election.cc:336] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.530994 19491 raft_consensus.cc:697] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:51.524394 13582 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.532110 19491 consensus_queue.cc:237] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.531101 14209 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.536612 19488 raft_consensus.cc:2804] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:58:51.537133 19488 raft_consensus.cc:697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:58:51.538081 19488 consensus_queue.cc:237] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:51.533957 19493 raft_consensus.cc:2804] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:58:51.541275 19493 raft_consensus.cc:697] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:58:51.544337 19493 consensus_queue.cc:237] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.531059 19480 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:58:51.547262 19480 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:51.547637 19480 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:51.548997 13440 catalog_manager.cc:5697] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 17 to 18, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 18 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:51.550810 13440 catalog_manager.cc:5697] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:51.558027 13440 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 16 to 17, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67), now has a pending config: VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65), VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66), VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67), VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:51.567162 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:51.567982 13658 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:51.583681 12409 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13565)
I20260709 12:58:51.590862 19480 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.603269 13441 catalog_manager.cc:5697] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 14, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 14 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:51.612720 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:51.638970 13619 raft_consensus.cc:1275] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:58:51.643013 19451 consensus_queue.cc:1048] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:58:51.658048 19480 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:58:51.665616 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:51.665706 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:58:51.666359 13658 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:51.666564 13612 raft_consensus.cc:3060] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:58:51.678710 13611 raft_consensus.cc:1275] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:51.682554 13611 raft_consensus.cc:1275] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:58:51.681958 19356 consensus_queue.cc:1048] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:58:51.686009 19356 consensus_queue.cc:1048] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:51.696970 13612 raft_consensus.cc:2468] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 18.
I20260709 12:58:51.703161 13440 catalog_manager.cc:5697] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 14 to 15, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 15 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:51.707372 13659 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:58:51.715680 19356 raft_consensus.cc:2804] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:58:51.716799 19356 raft_consensus.cc:697] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 18 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:58:51.716955 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:51.718498 19356 consensus_queue.cc:237] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:51.764382 13692 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:58:51.772534 13619 raft_consensus.cc:1275] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:58:51.788492 19451 consensus_queue.cc:1048] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:58:51.804176 19493 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:58:51.822479 13441 catalog_manager.cc:5697] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 13, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:51.855211 13611 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:58:51.858404 19509 ts_tablet_manager.cc:933] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:58:51.869547 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:51.881287 14238 raft_consensus.cc:1275] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:58:51.910259 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:51.922763 13584 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:51.925810 19508 log.cc:927] Time spent T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 12:58:51.942710 13584 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c88adb75300c48e98d20928bce53551d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:51.946154 19450 consensus_queue.cc:1048] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:58:51.948421 19444 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:58:51.952857 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:51.955843 19509 tablet_copy_client.cc:323] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
I20260709 12:58:51.965803 14244 raft_consensus.cc:1275] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:58:52.000649 13702 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet 56f36011554a4cd78d7b17b92a823c8f from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:58:52.001543 13702 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet 56f36011554a4cd78d7b17b92a823c8f from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-56f36011554a4cd78d7b17b92a823c8f
I20260709 12:58:52.001906 13440 catalog_manager.cc:5697] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 16 to 18, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 18 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:52.015810 13702 tablet_copy_source_session.cc:215] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:52.058086 13693 raft_consensus.cc:1275] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:58:52.059244 19458 consensus_queue.cc:1048] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:58:52.067159 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:52.099851 19384 raft_consensus.cc:993] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: : Instructing follower 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:52.100560 19407 raft_consensus.cc:1081] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Signalling peer 058c47afa94546dba814cf345f414f4a to start an election
I20260709 12:58:52.105229 19493 consensus_queue.cc:1048] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:58:52.115378 13687 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c88adb75300c48e98d20928bce53551d"
dest_uuid: "058c47afa94546dba814cf345f414f4a"
 from {username='slave'} at 127.0.0.1:45328
I20260709 12:58:52.116214 13687 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:52.116659 13687 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:52.146919 13687 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
W20260709 12:58:52.161168 19515 log.cc:927] Time spent T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: Append to log took a long time: real 0.171s	user 0.003s	sys 0.000s
I20260709 12:58:52.177516 13687 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:52.183954 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:52.184583 13616 raft_consensus.cc:3055] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:58:52.184924 13616 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
W20260709 12:58:52.169795 13427 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 104884us
W20260709 12:58:52.196024 13658 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:52.204617 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 245122us
I20260709 12:58:52.185745 13616 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:52.206755 13616 raft_consensus.cc:3060] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:58:52.218742 13616 raft_consensus.cc:2468] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:58:52.220713 13659 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:58:52.226168 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:52.226754 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.236145 13584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111) [suppressed 362 similar messages]
I20260709 12:58:52.239995 19490 raft_consensus.cc:2955] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:52.254896 19450 raft_consensus.cc:2804] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:58:52.280373 19450 raft_consensus.cc:697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:52.281246 13687 raft_consensus.cc:1275] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 12:58:52.270447 13427 catalog_manager.cc:4755] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6dd04e83486049ae859032792f652869: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:52.273432 19509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f36011554a4cd78d7b17b92a823c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:52.281738 19450 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:52.297124 14242 raft_consensus.cc:2955] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:52.307998 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:52.308447 13611 raft_consensus.cc:1275] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:58:52.313071 14244 raft_consensus.cc:1275] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:58:52.316510 19524 consensus_queue.cc:1048] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:58:52.318712 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:52.335167 13438 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 11 to 12, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:58:52.366161 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 45487.4us
I20260709 12:58:52.367587 19356 consensus_queue.cc:1048] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:52.371978 19480 consensus_queue.cc:1048] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:58:52.394273 13619 raft_consensus.cc:2955] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:52.400182 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:52.406268 13441 catalog_manager.cc:5697] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 11 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65), VOTER d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66), VOTER 058c47afa94546dba814cf345f414f4a (127.12.30.67), VOTER 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:52.427758 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:52.430054 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:52.484026 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.511762 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.536459 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:52.543748 19509 tablet_copy_client.cc:806] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:58:52.546622 19509 tablet_copy_client.cc:670] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:58:52.559900 19509 tablet_copy_client.cc:538] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:58:52.563897 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.581704 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.584364 12409 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 13565)
W20260709 12:58:52.601329 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:52.609290 19509 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
W20260709 12:58:52.635578 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.657059 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.763914 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527): Couldn't send request to peer 0f34cda059124f839711dbef61bb2c28. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c88adb75300c48e98d20928bce53551d. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:52.771250 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.823122 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.855824 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:52.912693 13611 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:58:52.915799 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:52.923214 19490 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:58:52.928961 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.934165 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.935055 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.939292 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.946550 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.947314 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.967887 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.991930 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:52.992509 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:53.012698 19509 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:58:53.013721 19509 tablet_bootstrap.cc:492] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
I20260709 12:58:53.028851 19509 ts_tablet_manager.cc:1403] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.420s	user 0.055s	sys 0.014s
I20260709 12:58:53.033455 19509 raft_consensus.cc:359] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:53.034437 19509 raft_consensus.cc:740] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:58:53.045152 19509 consensus_queue.cc:260] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:53.048060 19509 ts_tablet_manager.cc:1434] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:58:53.080933 13702 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-56f36011554a4cd78d7b17b92a823c8f received from {username='slave'} at 127.0.0.1:40872
I20260709 12:58:53.081602 13702 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-56f36011554a4cd78d7b17b92a823c8f on tablet 56f36011554a4cd78d7b17b92a823c8f with peer 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:53.134513 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:53.195279 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:53.198585 19480 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:53.199360 19480 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:53.201658 19480 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:53.202883 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:58:53.203837 13619 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 16.
I20260709 12:58:53.205389 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:53.214469 19491 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:58:53.214968 19491 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:53.215327 19491 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:58:53.262645 19491 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:53.266201 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:58:53.266844 13619 raft_consensus.cc:3060] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:58:53.284214 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:53.325223 13619 raft_consensus.cc:2468] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
I20260709 12:58:53.340408 19491 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Requested vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:58:53.349285 13659 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:58:53.350503 19450 raft_consensus.cc:2804] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:58:53.351066 19450 raft_consensus.cc:697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:58:53.352569 19450 consensus_queue.cc:237] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:58:53.385486 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:53.387595 19509 ts_tablet_manager.cc:933] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Initiating tablet copy from peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:53.395027 14238 raft_consensus.cc:1217] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 LEARNER]: Deduplicated request from leader. Original: 18.12->[19.13-19.13]   Dedup: 19.13->[]
W20260709 12:58:53.420971 13658 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:53.425578 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:53.436893 13658 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:53.456346 13427 catalog_manager.cc:4755] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6dd04e83486049ae859032792f652869: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:58:53.458226 13441 catalog_manager.cc:5697] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 15 to 17, leader changed from ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 17 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:53.491884 19509 tablet_copy_client.cc:323] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Beginning tablet copy session from remote peer at address 127.12.30.67:41755
W20260709 12:58:53.530882 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 91175.2us
W20260709 12:58:53.584852 12409 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 13565)
W20260709 12:58:53.587193 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 92900.6us
W20260709 12:58:53.590701 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:53.593184 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 65573us
I20260709 12:58:53.596640 13702 tablet_copy_service.cc:140] P 058c47afa94546dba814cf345f414f4a: Received BeginTabletCopySession request for tablet c88adb75300c48e98d20928bce53551d from peer 0f34cda059124f839711dbef61bb2c28 ({username='slave'} at 127.0.0.1:40872)
I20260709 12:58:53.597333 13702 tablet_copy_service.cc:161] P 058c47afa94546dba814cf345f414f4a: Beginning new tablet copy session on tablet c88adb75300c48e98d20928bce53551d from peer 0f34cda059124f839711dbef61bb2c28 at {username='slave'} at 127.0.0.1:40872: session id = 0f34cda059124f839711dbef61bb2c28-c88adb75300c48e98d20928bce53551d
I20260709 12:58:53.626760 13702 tablet_copy_source_session.cc:215] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:58:53.645354 19509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c88adb75300c48e98d20928bce53551d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:58:53.720732 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:53.724532 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:58:53.725771 19509 tablet_copy_client.cc:806] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 0 data blocks...
I20260709 12:58:53.726413 19509 tablet_copy_client.cc:670] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Starting download of 1 WAL segments...
I20260709 12:58:53.749390 19509 tablet_copy_client.cc:538] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:58:53.767400 19509 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Bootstrap starting.
I20260709 12:58:53.768719 13619 raft_consensus.cc:1275] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:58:53.806931 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:53.811583 19539 consensus_queue.cc:1048] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:58:53.862471 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:53.989449 19539 raft_consensus.cc:1064] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
W20260709 12:58:53.997416 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:54.017632 19539 consensus_queue.cc:237] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:54.077178 19509 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:58:54.078307 19509 tablet_bootstrap.cc:492] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Bootstrap complete.
W20260709 12:58:54.108079 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 46088.6us
W20260709 12:58:54.133656 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 47223.6us
I20260709 12:58:54.134613 19509 ts_tablet_manager.cc:1403] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Time spent bootstrapping tablet: real 0.367s	user 0.043s	sys 0.012s
I20260709 12:58:54.158308 19509 raft_consensus.cc:359] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:54.159125 19509 raft_consensus.cc:740] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Initialized, Role: LEARNER
I20260709 12:58:54.175670 19509 consensus_queue.cc:260] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: NON_VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: true } }
I20260709 12:58:54.180513 14244 raft_consensus.cc:1217] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 LEARNER]: Deduplicated request from leader. Original: 11.10->[12.11-12.11]   Dedup: 12.11->[]
I20260709 12:58:54.201844 19509 ts_tablet_manager.cc:1434] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.001s
I20260709 12:58:54.212160 14238 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:58:54.227105 13702 tablet_copy_service.cc:342] P 058c47afa94546dba814cf345f414f4a: Request end of tablet copy session 0f34cda059124f839711dbef61bb2c28-c88adb75300c48e98d20928bce53551d received from {username='slave'} at 127.0.0.1:40872
I20260709 12:58:54.227610 13702 tablet_copy_service.cc:434] P 058c47afa94546dba814cf345f414f4a: ending tablet copy session 0f34cda059124f839711dbef61bb2c28-c88adb75300c48e98d20928bce53551d on tablet c88adb75300c48e98d20928bce53551d with peer 0f34cda059124f839711dbef61bb2c28
W20260709 12:58:54.257853 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:54.312704 19541 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:58:54.348218 13612 raft_consensus.cc:1275] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:58:54.371081 19356 consensus_queue.cc:1048] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:58:54.405295 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:54.444074 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:58:54.456501 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:54.523460 19552 raft_consensus.cc:2955] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:54.540894 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:54.575515 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:54.578038 13441 catalog_manager.cc:5697] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 12 to 14, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:54.585434 12409 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 13565)
I20260709 12:58:54.595072 13611 raft_consensus.cc:2955] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:54.598029 14244 raft_consensus.cc:2955] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:54.656755 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:54.668206 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:54.693185 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:54.777113 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:54.787686 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:58:54.797900 19541 raft_consensus.cc:1064] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: attempting to promote NON_VOTER 0f34cda059124f839711dbef61bb2c28 to VOTER
I20260709 12:58:54.829610 19541 consensus_queue.cc:237] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 10, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:58:54.861727 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:54.938918 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:58:54.969008 14238 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 LEARNER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:58:55.025501 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.043515 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.051304 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:58:55.056766 13616 raft_consensus.cc:1275] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:58:55.058202 19491 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:58:55.086581 19491 consensus_queue.cc:1048] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:58:55.134322 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:58:55.144948 19561 raft_consensus.cc:2955] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Committing config change with OpId 12.12: config changed from index 9 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
W20260709 12:58:55.164944 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:58:55.169709 14238 raft_consensus.cc:2955] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Committing config change with OpId 12.12: config changed from index 9 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:55.173620 13619 raft_consensus.cc:2955] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Committing config change with OpId 12.12: config changed from index 9 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } } }
I20260709 12:58:55.182832 13440 catalog_manager.cc:5697] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a reported cstate change: config changed from index 9 to 12, 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:58:55.192281 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.206117 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.213186 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.218703 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:55.295480 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.303905 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.326552 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.339972 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.346170 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.385473 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.391036 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.409477 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.446874 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.511196 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.580610 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.586802 12409 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 13565)
W20260709 12:58:55.602105 13427 catalog_manager.cc:4755] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6dd04e83486049ae859032792f652869: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:58:55.638742 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.681720 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.728605 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.777416 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.799525 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:55.823581 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.845798 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:55.868546 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:56.160329 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:56.336236 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 105471us
W20260709 12:58:56.433445 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:56.437199 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:56.461694 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:56.587634 12409 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 13565)
W20260709 12:58:56.769347 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:56.787068 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:56.929217 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:57.040444 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:57.069420 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 71214us
W20260709 12:58:57.074012 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 92513.1us
W20260709 12:58:57.111243 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:57.119607 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:57.123661 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:57.157783 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.241835 13658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111) [suppressed 447 similar messages]
W20260709 12:58:57.245217 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.335048 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:57.388355 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.409981 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.412065 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.421864 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.501474 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.561328 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.570464 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.581993 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:57.588178 12409 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 13565)
W20260709 12:58:57.588449 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.617828 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:58:57.722054 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.742419 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.768795 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.791671 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.802398 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.858479 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.888844 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.890347 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.911736 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:57.938864 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.015427 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.041077 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.048982 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:58:58.061671 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.080119 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.130865 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.180773 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.394922 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.564191 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:58:58.588701 12409 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 13565)
W20260709 12:58:58.640148 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:58:59.589607 12409 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 13565)
I20260709 12:58:59.725706 19589 raft_consensus.cc:493] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:59.909565 19583 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:59.910617 19583 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:59.934762 19591 raft_consensus.cc:493] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:59.935575 19593 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:59.943507 19578 raft_consensus.cc:493] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:58:59.956262 19578 raft_consensus.cc:515] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:59.945413 19595 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:59.976177 19583 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:58:59.976182 19593 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:59.978518 19591 raft_consensus.cc:515] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:59.989089 19571 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:59.898618 19589 raft_consensus.cc:515] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:58:59.990161 19594 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:58:59.990849 19595 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.000423 19595 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.002338 19583 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.003511 19583 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:58:59.997231 19578 leader_election.cc:290] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.032212 19599 raft_consensus.cc:493] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.032841 19589 leader_election.cc:290] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.033588 19589 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.034394 19589 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.040165 19594 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:00.033156 19599 raft_consensus.cc:515] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.012849 19595 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.113801 19611 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.043113 19571 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:00.135574 19609 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.042248 19591 leader_election.cc:290] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.136780 19615 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.137725 19619 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.147197 19615 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.138756 19617 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.148849 19617 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.151604 19615 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.074338 19612 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.152594 19615 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.147523 19619 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.153326 19615 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.157946 19571 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:00.104461 13714 debug-util.cc:398] Leaking SignalData structure 0x7b08002fe700 after lost signal to thread 12412
I20260709 12:59:00.109529 19583 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.111339 19601 raft_consensus.cc:493] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.134073 19611 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.140394 19594 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.106648 19600 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.042860 19593 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.169265 19591 raft_consensus.cc:493] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.043479 19586 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.108047 19602 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.139814 19618 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.073918 19585 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.073899 19606 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.195964 19620 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.197058 19616 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.197382 19593 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.198194 19616 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.195973 19601 raft_consensus.cc:515] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.201121 19606 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.197072 19620 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.204022 19595 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.204540 19620 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.180847 19583 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.205183 19620 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.205811 19583 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.206094 19620 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.206400 19595 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.198444 19593 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.208231 19602 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.208345 19586 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.209918 19593 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.201669 19601 leader_election.cc:290] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.210918 19602 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.211494 19586 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.211894 19611 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.212111 19591 raft_consensus.cc:515] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:00.212925 19609 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.213557 19620 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:00.215948 13714 debug-util.cc:398] Leaking SignalData structure 0x7b0800059e60 after lost signal to thread 13475
I20260709 12:59:00.216475 19624 raft_consensus.cc:493] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.227660 19599 leader_election.cc:290] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.231293 19607 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.231755 19585 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.232178 19607 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.232921 19609 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.233961 19589 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.211498 19596 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.234449 19585 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.234933 19591 leader_election.cc:290] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.235246 19615 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.234998 19596 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.217264 19600 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.236186 19588 raft_consensus.cc:493] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.236933 19588 raft_consensus.cc:515] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.237900 19600 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.239995 19624 raft_consensus.cc:515] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.240901 19618 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.242749 19624 leader_election.cc:290] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.243618 19617 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.244118 19613 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.245332 19613 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.225193 19606 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:00.261040 13714 debug-util.cc:398] Leaking SignalData structure 0x7b0800151100 after lost signal to thread 13643
I20260709 12:59:00.268203 19626 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.268561 19619 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.269145 19626 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.271030 19583 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.271629 19626 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.274911 19616 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.275873 19618 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.276999 19613 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.262104 19571 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.277921 19607 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.237900 19596 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.278802 19597 raft_consensus.cc:493] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.279596 19597 raft_consensus.cc:515] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.152946 19612 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.282889 19612 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.283236 19597 leader_election.cc:290] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.278304 19592 raft_consensus.cc:493] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.278527 19571 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.283878 19597 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.303380 19597 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:59:00.284144 13714 debug-util.cc:398] Leaking SignalData structure 0x7b080014f480 after lost signal to thread 13717
W20260709 12:59:00.308239 13714 debug-util.cc:398] Leaking SignalData structure 0x7b0800226800 after lost signal to thread 14270
W20260709 12:59:00.308969 13714 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9340 after lost signal to thread 19576
I20260709 12:59:00.284858 19592 raft_consensus.cc:515] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.313684 19592 leader_election.cc:290] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.286091 19619 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.315351 19619 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.286274 19583 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.295720 19610 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.318707 19583 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.318951 19610 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.321521 19610 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.323731 19591 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.324981 19591 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.325556 19583 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.295274 19590 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.326803 19590 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.329874 19590 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.294595 19594 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.331195 19589 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.296017 19605 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.334335 19589 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.334873 19614 raft_consensus.cc:493] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.334818 19605 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.335852 19614 raft_consensus.cc:515] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:00.337795 19589 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.337697 19605 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.296197 19599 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.288380 19571 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.338840 19602 raft_consensus.cc:493] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.339354 19599 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.339562 19602 raft_consensus.cc:515] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.340716 19571 raft_consensus.cc:493] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.341370 19571 raft_consensus.cc:515] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.341888 19602 leader_election.cc:290] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.295445 19611 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.342973 19578 raft_consensus.cc:493] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.296197 19588 leader_election.cc:290] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.331885 19596 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.343765 19578 raft_consensus.cc:515] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } }
I20260709 12:59:00.344631 19596 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.334059 19601 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.346570 19578 leader_election.cc:290] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209)
I20260709 12:59:00.333215 19619 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.346498 19601 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.341945 19599 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.348224 19591 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.350081 19601 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.343364 19611 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.331244 19594 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:00.352469 19610 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.352649 19611 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.353487 19611 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:00.353754 19594 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.354336 19594 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.354398 19611 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.354897 19594 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.353353 19610 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.343794 19571 leader_election.cc:290] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.342628 19602 raft_consensus.cc:493] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.362025 19611 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.362227 19594 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.359508 19597 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:00.363231 19602 raft_consensus.cc:515] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:00.364418 19610 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.338292 19614 leader_election.cc:290] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.373498 19602 leader_election.cc:290] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.375842 19596 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:59:00.374804 14211 leader_election.cc:336] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:00.381007 14211 leader_election.cc:336] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:00.381937 14211 leader_election.cc:336] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:00.382704 14211 leader_election.cc:336] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:00.383186 14211 leader_election.cc:336] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.396270 19623 raft_consensus.cc:493] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.397352 19623 raft_consensus.cc:515] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
W20260709 12:59:00.383970 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:59:00.405773 19596 raft_consensus.cc:493] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.406490 19596 raft_consensus.cc:515] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.411388 19596 leader_election.cc:290] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.412148 19596 raft_consensus.cc:493] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.413028 19596 raft_consensus.cc:515] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.417623 19602 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.418437 19602 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:00.408300 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 51089.3us
W20260709 12:59:00.435757 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:59:00.442262 19589 raft_consensus.cc:493] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.443507 19589 raft_consensus.cc:515] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.449530 19589 leader_election.cc:290] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.453702 19602 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.457867 19628 raft_consensus.cc:493] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.458667 19628 raft_consensus.cc:515] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:00.468722 19623 leader_election.cc:290] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.485075 19623 raft_consensus.cc:493] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.485864 19623 raft_consensus.cc:515] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.479391 19596 leader_election.cc:290] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.459378 19589 raft_consensus.cc:493] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.490281 19589 raft_consensus.cc:515] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.460283 19632 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.492125 19632 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.493432 19589 leader_election.cc:290] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.488750 19623 leader_election.cc:290] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.498003 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c827d49183b8406b9adff9666f6d83a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.501364 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.500423 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b48fb5973b84b06a9ad858a9b3967f8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:59:00.497076 14211 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.547001 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.548988 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68566bdf08224971a44a28240a8bf9e4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:59:00.549862 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 52833.5us
I20260709 12:59:00.552503 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.514461 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.553391 13617 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 7.
I20260709 12:59:00.553857 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:59:00.552724 14211 leader_election.cc:336] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.511955 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.551247 13616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa892a8dfec433d8aa859d442e3e309" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.561811 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5406eb61fd534169b3c1645f80144779" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.504165 19628 leader_election.cc:290] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:00.566784 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.570514 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6f9228cbd74e95b5192c80f7f4a2b6" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.559145 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.551128 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2c2e5a7dac4eb4b3bacc18402ffa47" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.568075 13614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.572181 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee08c4df55f41c7a18951168c44fb5a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:59:00.573475 14211 leader_election.cc:336] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.505275 19632 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.578622 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.579139 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.586156 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.588526 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.589457 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:59:00.590701 12409 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 13565)
I20260709 12:59:00.598256 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.600466 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b8156356cf4b709a2903d6861a117c" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.601909 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b47f4cbc5b245adbdd9ebed9c3175f1" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.603641 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9eb8a2c5d7846099d30c0839aefc5a9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.600854 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.604393 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.607944 13618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.609639 13612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
W20260709 12:59:00.611053 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46508.8us
I20260709 12:59:00.614387 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a3f49b0e164d27994cfe440af6f514" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.614825 13619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.615633 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.621913 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7d1ab18f48a4689857964b7200b72c9" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.632377 13615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.615418 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.666523 13610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:00.666975 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.668886 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.674340 19619 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:59:00.672417 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 63213us
I20260709 12:59:00.682037 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eb0ac09bf7466899637997aae42111" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.678247 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.685304 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.686985 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.687384 14240 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:59:00.687804 14238 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:59:00.688635 19612 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:59:00.688730 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.686547 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.690784 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.691610 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.692608 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fe463242d40e5b82ddb5898323fc5" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.693899 19606 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:59:00.694555 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.695022 19606 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:00.695220 14242 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 11.
I20260709 12:59:00.699755 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c636bac0584e2e92ff66a089d880ee" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.703403 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86a0c010e8cc4b828fec8e8547e37902" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:00.703735 14211 leader_election.cc:336] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.705936 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "019c73868e744f4eba4ac7d97d34a137" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:00.709229 14211 leader_election.cc:336] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.713753 14211 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.715067 19619 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:00.719075 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.719311 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.720078 13692 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:59:00.723099 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2ce7a788b8c40629fc1921a4286d084" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.724370 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.725018 13691 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 13.
W20260709 12:59:00.728003 14211 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.730098 14211 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.733171 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d95a7092b0445eb00a28a1fdb8d239" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:00.735798 14211 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.737208 14211 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.737912 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.740345 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.741096 14244 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
W20260709 12:59:00.743722 14211 leader_election.cc:336] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:00.744390 14211 leader_election.cc:336] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.736907 19615 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:59:00.743098 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.746192 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.750960 14237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d73171142594388972ed22eae44f95f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.751402 14238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.754339 19619 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:59:00.755879 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.758100 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c9f20cfde5343c9b91c4e2adc5320ce" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:00.762445 14211 leader_election.cc:336] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.746766 13687 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 7.
I20260709 12:59:00.759192 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f2068be68d418fb0b123e59c3db757" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.763161 14211 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:00.767843 19615 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:00.770175 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be11851ed11044eea9f0fc3dbdbe0599" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.771275 13688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.771731 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6af1705dc4aa4e738a8416263f5af754" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.772011 13688 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 15.
W20260709 12:59:00.773594 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:00.774209 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:00.774680 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:59:00.759629 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973866cfc22e48ea931cb1a6fb16872f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.760068 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.760497 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.760828 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.761598 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.761750 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.778901 13690 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 14.
I20260709 12:59:00.761821 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c665edf3d7834e18a67df3f75b30856a" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.782832 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88adb75300c48e98d20928bce53551d" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.785928 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.786338 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f125a6a76f974409807014b9ddaa1b8c" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.786788 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.799324 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:00.797037 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:00.798432 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f7e1f6a8119482c9f8056bb615edfa8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:00.800783 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:00.807622 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0863151795784affb539e2b7a55fc2b8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:00.813928 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:00.797509 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "048528e413ac4c67a1cfd3dfe70673c7" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.820669 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a689534693d4469befa622d6b4917e3" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.825135 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.822914 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:00.829023 19589 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:59:00.829485 19589 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:00.829896 19589 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:59:00.837934 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:00.838930 19599 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:59:00.839395 19599 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:00.839807 19599 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:59:00.847828 13427 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 64134us
I20260709 12:59:00.851435 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.853344 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82077fd60c1346d7b057a65a778ee2f5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.857673 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a8c869a6b840e591423340bc32af00" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.860555 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f97a6462cd4341ddbd7058ce9537c1eb" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.861480 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:00.866829 13584 leader_election.cc:336] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:00.867633 13584 leader_election.cc:336] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:00.868789 13584 leader_election.cc:336] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.878829 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.885061 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.887984 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1c4c42ef7ad4d8fb669a6a5fe183f6f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.888584 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be389b2236484464bc3e9f558d7b4230" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.889163 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f36011554a4cd78d7b17b92a823c8f" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.889837 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c5bef52669b468c8765408698f5884d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.891144 19599 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.891736 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d739d3dda4e4370b33be67fca1c66d5" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.887899 13689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:00.900707 13427 catalog_manager.cc:4755] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6dd04e83486049ae859032792f652869: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:00.887423 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75988f1a2dda4bd6962cd5f3ad1c304a" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.895161 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "921257a2480244e88f3ff036fb3be8d2" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.902671 19599 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.895884 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.903872 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d2db0b5674796bbd0426953d99538" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.905344 19589 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:00.906188 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:00.908474 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.908790 19589 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:00.913075 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:00.916913 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:00.932021 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 48939.7us
W20260709 12:59:00.976449 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:00.976502 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:00.978906 14239 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:59:00.994918 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
W20260709 12:59:01.009075 13584 leader_election.cc:336] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.009682 13584 leader_election.cc:304] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.011232 13582 leader_election.cc:304] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.016419 19614 raft_consensus.cc:2749] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.016909 19589 raft_consensus.cc:2749] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:59:01.017773 13582 leader_election.cc:304] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:59:01.018564 13584 leader_election.cc:336] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.019002 13582 leader_election.cc:304] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.019891 19627 raft_consensus.cc:2749] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:59:01.018615 19589 raft_consensus.cc:2749] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:01.023293 13582 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.024474 19628 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:59:01.025655 19628 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:01.026157 19628 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:59:01.032239 13584 leader_election.cc:336] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.032958 13584 leader_election.cc:304] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.040946 19614 raft_consensus.cc:2749] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:59:01.041095 13584 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.041697 13584 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.042915 19589 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:59:01.043239 13584 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.043824 13582 leader_election.cc:304] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.052865 19589 raft_consensus.cc:2749] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:59:01.053409 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.054186 19589 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:59:01.054683 19589 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:01.055079 19589 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:59:01.044906 13584 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.058885 19627 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:59:01.063140 13584 leader_election.cc:336] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:01.063719 13584 leader_election.cc:336] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.073077 19628 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.087147 19628 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.098766 14209 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.100525 19589 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.077724 14239 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:59:01.101174 14244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:01.117599 14242 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16. Ops: []
I20260709 12:59:01.101478 19615 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:01.102069 13617 raft_consensus.cc:1240] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 16. Current term is 17. Ops: []
I20260709 12:59:01.109112 14209 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.064122 13584 leader_election.cc:304] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.121968 19615 consensus_queue.cc:1059] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:59:01.123160 19615 raft_consensus.cc:3055] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:59:01.123539 19615 raft_consensus.cc:740] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:01.124815 19615 consensus_queue.cc:260] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.126657 19615 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:59:01.128417 19612 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:01.130066 14209 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.130856 19612 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.132194 14209 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.133143 19612 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:59:01.134725 19589 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.138571 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.152454 19605 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:01.153676 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.155006 19586 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:59:01.155256 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 91537.6us
W20260709 12:59:01.156013 13584 leader_election.cc:336] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:01.156491 13584 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.156927 13584 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.162315 19628 raft_consensus.cc:2749] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.163043 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.161799 13658 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.164353 19586 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:59:01.164738 19586 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:01.165132 19586 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:59:01.165684 19612 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:01.163187 19628 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:59:01.074476 13582 leader_election.cc:304] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.170378 19628 raft_consensus.cc:2749] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:59:01.171590 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 97199.7us
I20260709 12:59:01.172542 13582 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.173324 19628 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:59:01.173723 19628 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:01.174095 19628 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:59:01.176265 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.179171 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.183421 19605 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:59:01.183832 19605 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:01.181576 13582 leader_election.cc:304] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.184269 19612 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.184243 19605 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:59:01.188895 19586 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.187366 19628 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:01.162269 13584 leader_election.cc:336] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.190167 14209 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.187623 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.197625 19612 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:59:01.197645 13617 raft_consensus.cc:1240] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 7. Current term is 8. Ops: []
I20260709 12:59:01.198067 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.201040 19619 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.201476 14209 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.201782 19619 consensus_queue.cc:1059] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:59:01.203043 19619 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.203225 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:59:01.198408 13584 leader_election.cc:336] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:01.204210 13584 leader_election.cc:336] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.204416 19606 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:01.204628 13584 leader_election.cc:304] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.199774 19586 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.195411 19628 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.212819 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.213596 19599 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:01.214257 19599 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:01.215695 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.216449 19599 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:59:01.216823 19599 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:01.217193 19599 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:59:01.203758 19619 raft_consensus.cc:3055] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:59:01.221095 19619 raft_consensus.cc:740] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:01.222031 19619 consensus_queue.cc:260] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.223522 19619 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:59:01.225958 19599 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.229058 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.229915 19606 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:01.230298 14209 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.231025 19606 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:01.231438 14209 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.232506 19599 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.233400 19589 raft_consensus.cc:2749] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:59:01.234228 13584 leader_election.cc:336] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.235146 19589 raft_consensus.cc:2749] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:01.233511 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:59:01.236187 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 54694.4us
I20260709 12:59:01.236528 13658 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.237219 19586 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.238006 19586 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.238854 13582 leader_election.cc:304] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.239065 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:59:01.239499 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:01.239602 19614 raft_consensus.cc:2749] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:01.239730 19586 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.241732 13582 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.235159 13584 leader_election.cc:304] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.244010 19590 consensus_queue.cc:1059] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:59:01.244764 19614 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:59:01.245173 19614 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:01.245416 19586 raft_consensus.cc:3055] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:59:01.245595 19614 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:59:01.245831 19586 raft_consensus.cc:740] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:59:01.247107 19586 consensus_queue.cc:260] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.249208 19586 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:59:01.254118 13584 leader_election.cc:336] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.256067 13584 leader_election.cc:304] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.256806 19628 raft_consensus.cc:2749] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:01.257615 19628 raft_consensus.cc:2749] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:01.259080 14209 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.259999 19606 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:01.260115 19612 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.271265 14209 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.272545 14209 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.273566 19606 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.274150 19615 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:59:01.275458 13584 leader_election.cc:336] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:01.275486 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:01.277794 13658 leader_election.cc:336] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.278187 13658 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.281147 13584 leader_election.cc:304] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.283783 13582 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.287021 19614 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.288130 19589 raft_consensus.cc:2749] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:59:01.289160 13584 leader_election.cc:336] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.289577 13584 leader_election.cc:304] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.291010 19590 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:59:01.292932 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:59:01.293586 13658 leader_election.cc:336] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.294250 13658 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.295586 19628 raft_consensus.cc:2749] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:01.295862 19590 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:59:01.303776 13584 leader_election.cc:336] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.305150 19589 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:01.307873 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.309206 13612 raft_consensus.cc:1240] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 15. Current term is 16. Ops: []
I20260709 12:59:01.312371 19606 consensus_queue.cc:1059] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:01.313475 19606 raft_consensus.cc:3055] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:59:01.313800 19606 raft_consensus.cc:740] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:01.304369 13584 leader_election.cc:304] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:59:01.316565 13658 leader_election.cc:336] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.320140 13658 leader_election.cc:304] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.335744 19586 raft_consensus.cc:2749] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:59:01.351310 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 47573.9us
W20260709 12:59:01.354004 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 46228.1us
I20260709 12:59:01.354751 13582 leader_election.cc:304] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:59:01.353076 13658 leader_election.cc:336] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.355641 13658 leader_election.cc:304] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.324952 19589 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:59:01.359233 13584 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:01.361261 13612 raft_consensus.cc:1240] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 13. Current term is 14. Ops: []
I20260709 12:59:01.367991 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:01.353575 19628 raft_consensus.cc:2749] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:01.371598 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:01.273870 19605 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.373925 19605 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.376120 19599 raft_consensus.cc:2749] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:01.340508 19606 consensus_queue.cc:260] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.381244 14241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:59:01.382684 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:01.360070 19627 raft_consensus.cc:2749] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:59:01.393534 19627 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:59:01.393999 19627 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:59:01.394894 19627 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.378873 19589 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:01.397871 19614 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.385878 19606 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:59:01.399335 19589 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:59:01.414597 14246 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 8.
W20260709 12:59:01.438941 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 73275.9us
W20260709 12:59:01.442502 13659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 87065us
I20260709 12:59:01.438683 19589 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.447629 19589 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Requested vote from peers 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:01.450737 14240 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 17.
W20260709 12:59:01.456645 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 97486.3us
I20260709 12:59:01.464100 19612 consensus_queue.cc:1059] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:59:01.465395 19612 raft_consensus.cc:3055] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:59:01.465786 19612 raft_consensus.cc:740] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:01.466708 19612 consensus_queue.cc:260] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.477994 13612 raft_consensus.cc:1240] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Rejecting Update request from peer 058c47afa94546dba814cf345f414f4a for earlier term 15. Current term is 16. Ops: []
I20260709 12:59:01.476284 13659 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.481153 19605 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:59:01.476894 19612 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:59:01.496703 13441 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 15 to 16, leader changed from 058c47afa94546dba814cf345f414f4a (127.12.30.67) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 16 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:01.523237 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 170169us
W20260709 12:59:01.528337 13658 leader_election.cc:336] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.528849 13658 leader_election.cc:304] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.537251 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:01.538966 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:01.540608 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:01.553853 14240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:01.554414 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:59:01.557580 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:01.558189 13658 leader_election.cc:336] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.558564 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.564606 19590 raft_consensus.cc:2749] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:01.565392 19590 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:01.568840 13612 raft_consensus.cc:1240] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 14. Current term is 15. Ops: []
I20260709 12:59:01.571857 19606 consensus_queue.cc:1059] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:59:01.573058 19606 raft_consensus.cc:3055] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:59:01.573412 19606 raft_consensus.cc:740] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:01.574340 19606 consensus_queue.cc:260] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:01.574661 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:01.575742 19606 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:59:01.584133 13658 leader_election.cc:336] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.584669 13658 leader_election.cc:304] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.585392 19649 raft_consensus.cc:2749] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:59:01.586522 13658 leader_election.cc:336] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.586957 13658 leader_election.cc:304] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:01.587731 19586 raft_consensus.cc:2749] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:59:01.587886 13658 leader_election.cc:336] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:01.588435 13658 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.589149 19586 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:59:01.591166 12409 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 13565)
I20260709 12:59:01.595176 13692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:01.595646 14243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:01.597033 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:01.598563 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:01.600142 13686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:01.601930 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:01.606359 13689 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:59:01.607803 13687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:01.609661 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:01.611732 13693 raft_consensus.cc:1240] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 11. Current term is 12. Ops: []
I20260709 12:59:01.587971 14240 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:59:01.619473 13692 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35f063cd3f942b0bf559b7c6e9b299f in term 16.
I20260709 12:59:01.624233 13690 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Rejecting Update request from peer d35f063cd3f942b0bf559b7c6e9b299f for earlier term 12. Current term is 13. Ops: []
I20260709 12:59:01.637347 13584 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:01.638799 13584 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
W20260709 12:59:01.644809 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:59:01.646209 19639 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:59:01.648097 19639 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:59:01.644984 19589 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:59:01.656073 19589 raft_consensus.cc:697] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:59:01.659874 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:01.661419 19639 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.657660 19589 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.667052 13582 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.667960 19642 raft_consensus.cc:2749] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:59:01.669245 19642 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:01.670389 13584 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:59:01.670476 19641 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:59:01.671617 19642 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:59:01.672041 19641 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:59:01.672937 13582 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:01.674744 19641 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.675410 19628 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:59:01.676590 19641 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:59:01.673491 19642 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Becoming Leader. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:59:01.679567 19642 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.684329 13658 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.686694 13658 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:01.687609 19638 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:59:01.686121 19649 raft_consensus.cc:2749] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:59:01.715420 13658 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:59:01.752920 14212 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:59:01.724985 19586 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:59:01.761945 19589 consensus_queue.cc:1059] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:59:01.764318 19614 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:59:01.764928 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:01.765506 19614 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:59:01.765879 19614 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:59:01.770484 19589 raft_consensus.cc:3055] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:59:01.770989 19589 raft_consensus.cc:740] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d35f063cd3f942b0bf559b7c6e9b299f, State: Running, Role: LEADER
I20260709 12:59:01.771870 19589 consensus_queue.cc:260] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.773417 19589 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:59:01.766808 19614 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:01.776700 19614 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:59:01.783490 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 46963.9us
I20260709 12:59:01.798002 13441 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 16 to 17, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 17 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:01.800546 14209 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:59:01.800096 13441 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 7 to 8, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 8 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:01.825747 13441 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f reported cstate change: term changed from 15 to 16, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66). New cstate: current_term: 16 leader_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:01.891973 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:01.910252 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:01.910964 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:59:01.911458 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:01.947261 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:59:01.972685 14246 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:59:01.974857 19642 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:59:01.981539 13686 raft_consensus.cc:1275] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:59:02.002442 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:02.029069 19589 consensus_queue.cc:1048] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:59:02.090821 13684 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:59:02.108902 19614 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:59:02.119899 14246 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:59:02.147877 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:02.129603 19642 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:59:02.159179 14246 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:59:02.174881 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:02.188388 14242 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:59:02.204149 19642 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:59:02.209554 19642 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
W20260709 12:59:02.229146 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:02.249325 13686 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:59:02.250478 13684 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:59:02.255600 13658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111) [suppressed 317 similar messages]
I20260709 12:59:02.282478 13684 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:59:02.289434 13686 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer d35f063cd3f942b0bf559b7c6e9b299f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:59:02.308176 19589 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:02.309641 19589 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:59:02.317185 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:02.462212 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.493084 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:02.506727 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.527034 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.561877 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.562704 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.564160 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:02.592406 12409 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:02.602322 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.621153 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.662003 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:02.837159 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:59:02.858351 19669 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:02.859027 19669 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:02.864162 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:02.864959 13617 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:59:02.867058 19669 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:02.868584 14212 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:02.871582 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:02.877864 19670 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:59:02.878410 19670 raft_consensus.cc:493] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:02.878790 19670 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:59:02.887897 19666 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:02.888700 19666 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:02.901626 19612 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:02.901849 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:02.902190 19612 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:02.902638 13617 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:59:02.904557 14212 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:02.905835 19612 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:02.906684 19612 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:59:02.907469 19612 raft_consensus.cc:493] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:02.907981 19612 raft_consensus.cc:3060] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:59:02.904675 19671 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:02.916251 19671 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:02.918769 19671 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:02.920740 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:02.921806 13617 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:59:02.925789 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:02.929560 19671 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:59:02.930094 19671 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:02.930478 19671 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:59:02.931119 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:59:02.934444 19612 raft_consensus.cc:515] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:02.935843 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:02.936496 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:59:02.936324 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:02.937371 13684 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:59:02.937673 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:02.938524 13691 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:59:02.939668 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:02.940479 13690 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:59:02.942714 19671 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:02.927783 19666 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:02.955446 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:02.956167 13617 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:59:02.958428 19671 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:02.985605 19612 leader_election.cc:290] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:02.992985 19670 raft_consensus.cc:515] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:02.999475 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:02.999644 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
W20260709 12:59:03.000379 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:03.026985 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 72075.1us
I20260709 12:59:03.032677 13690 raft_consensus.cc:2393] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 13: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
W20260709 12:59:03.033465 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:03.040871 14212 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 058c47afa94546dba814cf345f414f4a
I20260709 12:59:03.042253 19666 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:59:03.042701 19666 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:03.043054 19666 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:59:03.032701 13684 raft_consensus.cc:2393] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:59:03.047683 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:03.052829 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd04e83486049ae859032792f652869" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:03.082033 13613 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:59:03.085637 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:03.098101 13611 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:59:03.080463 19612 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:03.104328 19612 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.107089 19612 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:03.111686 13611 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:59:03.114253 14212 leader_election.cc:304] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.115134 19612 raft_consensus.cc:2804] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:59:03.115718 19612 raft_consensus.cc:697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:03.116734 19612 consensus_queue.cc:237] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.118548 19670 leader_election.cc:290] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:03.122349 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:03.126271 13611 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
W20260709 12:59:03.125262 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 89423.2us
I20260709 12:59:03.124594 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.152561 19670 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:59:03.154446 19670 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:03.155978 19670 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:03.156895 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:59:03.158577 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.165246 19672 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:59:03.165742 19672 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:03.166093 19672 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:59:03.173197 13617 raft_consensus.cc:2468] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:59:03.137975 19666 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:03.180702 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:03.182773 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:03.184509 19666 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:03.185946 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:03.194659 14212 leader_election.cc:304] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.206100 19612 raft_consensus.cc:2804] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:59:03.206844 19612 raft_consensus.cc:697] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:03.208882 19612 consensus_queue.cc:237] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.198422 19672 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:03.231367 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 84072.5us
I20260709 12:59:03.239238 13617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:03.239866 13617 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:59:03.241089 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:59:03.239883 13611 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 12.
I20260709 12:59:03.243111 14212 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.244635 19673 raft_consensus.cc:493] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0f34cda059124f839711dbef61bb2c28)
I20260709 12:59:03.245546 19673 raft_consensus.cc:515] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.249070 19673 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:59:03.237996 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:03.258298 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:03.242080 19672 leader_election.cc:290] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:03.265410 19612 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:59:03.266400 13617 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:59:03.266239 19612 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:03.267895 19612 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.269160 14212 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.270143 19666 raft_consensus.cc:2804] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:59:03.270846 19666 raft_consensus.cc:697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:03.271823 19666 consensus_queue.cc:237] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.272382 13441 catalog_manager.cc:5697] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:03.274102 13441 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 12 to 13, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:03.275831 13441 catalog_manager.cc:5697] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 11 to 12, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 12 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:59:03.294009 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:03.371645 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:59:03.389990 13684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f14b8d35f741a28ad886b19614c4c8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:03.390744 13684 raft_consensus.cc:3060] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:59:03.392071 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:03.393709 13685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:03.392966 13690 raft_consensus.cc:2468] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:59:03.394752 13691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:03.395473 13691 raft_consensus.cc:2393] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f34cda059124f839711dbef61bb2c28 in current term 12: Already voted for candidate 058c47afa94546dba814cf345f414f4a in this term.
I20260709 12:59:03.417975 13684 raft_consensus.cc:2468] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 15.
I20260709 12:59:03.416545 19599 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:03.419910 19599 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.422590 19599 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:03.424000 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:03.424711 13613 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:59:03.429977 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.430953 19599 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:59:03.431357 19599 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:03.431730 19599 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:59:03.433470 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5db10903ab7d403ba303d72b7f26dfed" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:03.446759 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:03.450250 19599 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.453732 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:03.453938 19599 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:03.454370 13613 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:59:03.463877 13613 raft_consensus.cc:2468] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:59:03.466200 13659 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.468185 19599 raft_consensus.cc:2804] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:59:03.468966 19599 raft_consensus.cc:697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:59:03.469883 19599 consensus_queue.cc:237] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:03.494441 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:59:03.498172 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:03.507810 19681 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:03.508366 19681 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.511755 13658 leader_election.cc:304] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:03.513432 19638 raft_consensus.cc:2749] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:03.514732 13613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" is_pre_election: true
I20260709 12:59:03.512650 19681 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:03.515403 13613 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:59:03.516595 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 
I20260709 12:59:03.517449 19599 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:59:03.517872 19599 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:03.519500 19599 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:59:03.523460 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c25c2050d4547c39924ceb739727ae8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:59:03.529093 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:59:03.548544 14239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:03.555858 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:03.569171 13613 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:59:03.570941 19672 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:59:03.556298 19599 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.586107 19599 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:03.587582 13611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d35f063cd3f942b0bf559b7c6e9b299f"
I20260709 12:59:03.588305 13611 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:59:03.592924 12409 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 13565)
I20260709 12:59:03.600279 13440 catalog_manager.cc:5697] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 11 to 13, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.612345 13611 raft_consensus.cc:2468] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:59:03.613270 13612 raft_consensus.cc:1240] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 13. Current term is 14. Ops: []
I20260709 12:59:03.615573 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12822fb99cf24dbcac8dbb9d77e56af8" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
W20260709 12:59:03.609376 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:59:03.632241 19672 consensus_queue.cc:1059] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:03.633162 19672 raft_consensus.cc:3055] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:59:03.633531 19672 raft_consensus.cc:740] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:03.636369 13659 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f; no voters: 0f34cda059124f839711dbef61bb2c28
I20260709 12:59:03.638336 19672 consensus_queue.cc:260] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.639727 19672 raft_consensus.cc:3060] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:59:03.641278 13611 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:59:03.642882 19612 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:59:03.646425 19599 raft_consensus.cc:2804] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:59:03.647032 19599 raft_consensus.cc:697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:59:03.647882 19599 consensus_queue.cc:237] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.661975 13438 catalog_manager.cc:5697] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:03.672533 13612 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:59:03.680352 19669 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:59:03.704654 13440 catalog_manager.cc:5697] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 13 to 14, leader changed from 0f34cda059124f839711dbef61bb2c28 (127.12.30.68) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 14 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.727456 13691 raft_consensus.cc:1275] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:59:03.728387 13684 raft_consensus.cc:1275] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:59:03.732049 13617 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:59:03.734977 19612 consensus_queue.cc:1048] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:59:03.736084 19669 consensus_queue.cc:1048] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:59:03.736937 19612 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:03.737939 19612 raft_consensus.cc:3055] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:59:03.738315 19612 raft_consensus.cc:740] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:03.765605 13690 raft_consensus.cc:3060] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:59:03.818050 13693 raft_consensus.cc:1240] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Rejecting Update request from peer 0f34cda059124f839711dbef61bb2c28 for earlier term 12. Current term is 13. Ops: []
I20260709 12:59:03.823850 19669 consensus_queue.cc:1059] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:03.824702 19669 consensus_queue.cc:1243] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Peer 058c47afa94546dba814cf345f414f4a log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:59:03.828434 19612 consensus_queue.cc:260] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:03.830017 19612 raft_consensus.cc:3060] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:59:03.815794 13690 raft_consensus.cc:1275] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:59:03.837880 19669 consensus_queue.cc:1048] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:59:03.854167 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:04.018743 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:04.185115 14242 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:59:04.185740 14242 pending_rounds.cc:85] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 11
W20260709 12:59:04.186427 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:59:04.186009 14242 pending_rounds.cc:107] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 13.12
W20260709 12:59:04.170604 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 132506us
I20260709 12:59:04.192121 14242 raft_consensus.cc:2889] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:59:04.180687 19700 log.cc:927] Time spent T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:59:04.277482 13617 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:59:04.282199 14246 raft_consensus.cc:1275] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 12:59:04.282975 14246 pending_rounds.cc:85] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Aborting all ops after (but not including) 11
W20260709 12:59:04.282337 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 108281us
I20260709 12:59:04.283363 14246 pending_rounds.cc:107] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:59:04.283660 14246 raft_consensus.cc:2889] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:59:04.287775 13611 raft_consensus.cc:1275] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:59:04.312434 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.319533 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.321249 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:59:04.323642 19705 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:04.335206 19638 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:59:04.335431 19649 consensus_queue.cc:1048] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 12:59:04.417337 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:04.418121 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:59:04.418012 19649 consensus_queue.cc:1048] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:59:04.493559 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:04.515690 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.555620 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:04.589336 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.593504 12409 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:04.615273 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.755571 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.762621 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:04.802197 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.871387 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.884620 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:04.937386 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:04.953756 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.034556 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.095680 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.263661 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.285964 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:05.408214 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:05.429376 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 64129.4us
W20260709 12:59:05.442610 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:05.476527 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.556219 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:05.577972 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.594012 12409 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:05.632243 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.639092 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:05.672955 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:05.696085 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.722402 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.732192 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.787811 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:05.809652 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:05.825923 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.876946 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:05.947149 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:05.947908 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:06.067028 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:06.240550 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:59:06.398662 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49963.7us
W20260709 12:59:06.402328 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 49583.7us
W20260709 12:59:06.414573 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:06.424880 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:06.493081 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:06.558100 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:06.574219 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:06.594619 12409 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:06.623816 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:06.714073 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:06.879557 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:06.927495 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:06.984582 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:06.985106 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.060132 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.107013 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:07.107537 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:07.144702 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:07.145272 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:07.197517 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.257903 13658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111) [suppressed 432 similar messages]
W20260709 12:59:07.307828 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:07.398523 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.464275 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 75617.7us
W20260709 12:59:07.531515 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.532074 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.595176 12409 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:07.780133 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.784006 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.784953 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.828387 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:07.861775 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 44372us
W20260709 12:59:07.956923 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:07.984618 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:07.995261 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:08.038020 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:08.044440 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.092069 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:08.112577 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:08.176896 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.192416 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:08.205026 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.212108 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.240885 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.252498 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:08.323201 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.411953 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.415481 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:08.469955 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.519495 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:08.566602 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:59:08.595710 12409 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:08.624049 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:08.777329 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:09.039233 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:09.125957 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:09.134279 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:09.141116 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:09.150856 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:09.287809 13427 catalog_manager.cc:4755] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6dd04e83486049ae859032792f652869: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:09.415551 14211 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:39209, user_credentials={real_user=slave}} blocked reactor thread for 59480.7us
W20260709 12:59:09.420490 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:09.530576 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:09.596436 12409 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:09.593940 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:09.597995 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:09.622081 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:09.622803 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:09.623329 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:09.659238 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:09.763762 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:09.783805 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:09.790985 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:09.843827 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:09.966974 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.080286 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.153223 14212 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39367, user_credentials={real_user=slave}} blocked reactor thread for 80355.1us
W20260709 12:59:10.189544 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.208273 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.244254 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.270220 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.320559 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:10.368872 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:10.409297 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:10.464988 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:10.489990 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:10.541205 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.554157 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:10.569413 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.597235 12409 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:10.662885 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:10.692742 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.745639 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:10.818742 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.855293 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.858489 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:10.991674 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:11.011044 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:11.014880 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:11.019649 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:11.077697 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:11.103658 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:11.223215 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:59:11.309878 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:11.459118 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:11.533880 13582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 52994.6us
W20260709 12:59:11.537057 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:11.546227 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:11.559178 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:11.559870 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:11.597986 12409 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:11.654095 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:11.810487 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:11.820254 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:11.876161 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:11.932833 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:11.948939 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:12.032714 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:12.222297 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.239732 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:59:12.264458 14211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111) [suppressed 445 similar messages]
W20260709 12:59:12.301555 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:12.324373 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:12.377278 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:12.483337 13584 consensus_peers.cc:597] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.527998 13584 consensus_peers.cc:597] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.542869 13584 consensus_peers.cc:597] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.549084 13584 consensus_peers.cc:597] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.567654 13584 consensus_peers.cc:597] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.598521 12409 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 13565)
W20260709 12:59:12.632988 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.862936 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:12.933552 13584 consensus_peers.cc:597] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:12.935086 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:12.961124 13584 consensus_peers.cc:597] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:13.046568 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:13.120126 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.141510 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:13.168226 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:13.222645 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.225796 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.245043 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:13.254122 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:59:13.289971 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
W20260709 12:59:13.341738 13584 consensus_peers.cc:597] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.343425 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:13.361256 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.462054 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:59:13.529273 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:13.675149 13584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 58168.8us
W20260709 12:59:13.711330 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:13.761179 13584 consensus_peers.cc:597] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.765808 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.795400 13584 consensus_peers.cc:597] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:13.809549 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:59:13.856112 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
W20260709 12:59:13.880185 13659 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:13.884457 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:13.894600 14212 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:13.898188 13659 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:13.959260 13659 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:13.977519 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:13.982971 13659 consensus_peers.cc:597] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.008726 14212 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.017503 14212 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.022992 13659 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.027257 13659 consensus_peers.cc:597] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.034106 13659 consensus_peers.cc:597] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.044744 13659 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.073145 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:14.094043 13659 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.115837 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:14.142949 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:14.171010 14212 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.174785 13659 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.177824 13659 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.178653 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:59:14.182690 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:14.184870 12409 tablet_replica.cc:333] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.186123 12409 raft_consensus.cc:2243] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.187302 12409 raft_consensus.cc:2272] T 3f7e1f6a8119482c9f8056bb615edfa8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.187448 14212 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.194446 12409 tablet_replica.cc:333] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
W20260709 12:59:14.194869 14212 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.201064 13659 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.202468 12409 raft_consensus.cc:2243] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 12:59:14.206290 13659 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.209014 13659 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.222832 14212 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.203253 12409 raft_consensus.cc:2272] T c665edf3d7834e18a67df3f75b30856a P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.241657 12409 tablet_replica.cc:333] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
W20260709 12:59:14.246747 14212 consensus_peers.cc:597] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.249840 12409 raft_consensus.cc:2243] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.259680 12409 raft_consensus.cc:2272] T e634cc18febe4680bb70a3673cabbae4 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.267963 12409 tablet_replica.cc:333] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.269071 12409 raft_consensus.cc:2243] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.274936 12409 raft_consensus.cc:2272] T 1d73171142594388972ed22eae44f95f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.282330 13658 consensus_peers.cc:597] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:14.293301 12409 tablet_replica.cc:333] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
W20260709 12:59:14.294024 14212 consensus_peers.cc:597] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.294618 12409 raft_consensus.cc:2243] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.295472 12409 raft_consensus.cc:2272] T 019c73868e744f4eba4ac7d97d34a137 P d35f063cd3f942b0bf559b7c6e9b299f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.300482 12409 tablet_replica.cc:333] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
W20260709 12:59:14.309505 13659 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.315342 13659 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.316323 12409 raft_consensus.cc:2243] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.360492 12409 raft_consensus.cc:2272] T 3f54dde33fa845098e64673b576d780c P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.325621 13659 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.369117 14212 consensus_peers.cc:597] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.363854 12409 tablet_replica.cc:333] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.373498 12409 raft_consensus.cc:2243] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.374446 12409 raft_consensus.cc:2272] T 56f36011554a4cd78d7b17b92a823c8f P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.379455 13659 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.380585 13659 consensus_peers.cc:597] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:14.381353 14212 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:14.383968 12409 tablet_replica.cc:333] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.385192 12409 raft_consensus.cc:2243] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.386483 12409 raft_consensus.cc:2272] T 9aa892a8dfec433d8aa859d442e3e309 P d35f063cd3f942b0bf559b7c6e9b299f [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.389734 12409 tablet_replica.cc:333] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.390568 12409 raft_consensus.cc:2243] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.391223 12409 raft_consensus.cc:2272] T 86d7b5c530f14daaade417c2e3903072 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.404564 12409 tablet_replica.cc:333] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.416738 12409 raft_consensus.cc:2243] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.417990 12409 raft_consensus.cc:2272] T 2c5bef52669b468c8765408698f5884d P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.421195 12409 tablet_replica.cc:333] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.422029 12409 raft_consensus.cc:2243] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.422717 12409 raft_consensus.cc:2272] T 22f14b8d35f741a28ad886b19614c4c8 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.423863 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:59:14.425575 12409 tablet_replica.cc:333] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.426483 12409 raft_consensus.cc:2243] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.427626 12409 raft_consensus.cc:2272] T c827d49183b8406b9adff9666f6d83a9 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.442605 12409 tablet_replica.cc:333] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.443565 12409 raft_consensus.cc:2243] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.444526 12409 raft_consensus.cc:2272] T f9f9e2a62b8a498d93662b589c6a6870 P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.448670 12409 tablet_replica.cc:333] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.450024 12409 raft_consensus.cc:2243] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.450703 12409 raft_consensus.cc:2272] T 921257a2480244e88f3ff036fb3be8d2 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.453544 12409 tablet_replica.cc:333] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.454404 12409 raft_consensus.cc:2243] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.460378 12409 raft_consensus.cc:2272] T 6b48fb5973b84b06a9ad858a9b3967f8 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.463442 12409 tablet_replica.cc:333] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.464282 12409 raft_consensus.cc:2243] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.464891 12409 raft_consensus.cc:2272] T 5c25c2050d4547c39924ceb739727ae8 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.467967 12409 tablet_replica.cc:333] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.468744 12409 raft_consensus.cc:2243] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.480615 12409 raft_consensus.cc:2272] T 6af1705dc4aa4e738a8416263f5af754 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.492954 12409 tablet_replica.cc:333] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.493827 12409 raft_consensus.cc:2243] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.495386 12409 raft_consensus.cc:2272] T f125a6a76f974409807014b9ddaa1b8c P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.497089 13658 consensus_peers.cc:597] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:14.498399 12409 tablet_replica.cc:333] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.499264 12409 raft_consensus.cc:2243] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.500435 12409 raft_consensus.cc:2272] T 4b9bdd77e564429c9c0434b358f48302 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.503135 12409 tablet_replica.cc:333] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.515990 12409 raft_consensus.cc:2243] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.516816 12409 raft_consensus.cc:2272] T 1c9f20cfde5343c9b91c4e2adc5320ce P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.530047 12409 tablet_replica.cc:333] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.530840 12409 raft_consensus.cc:2243] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.531595 12409 raft_consensus.cc:2272] T f97a6462cd4341ddbd7058ce9537c1eb P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.539582 12409 tablet_replica.cc:333] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.540423 12409 raft_consensus.cc:2243] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.541057 12409 raft_consensus.cc:2272] T 0863151795784affb539e2b7a55fc2b8 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.543756 12409 tablet_replica.cc:333] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
W20260709 12:59:14.544638 13658 consensus_peers.cc:597] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:59:14.553335 12409 raft_consensus.cc:2243] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.554163 12409 raft_consensus.cc:2272] T 86a0c010e8cc4b828fec8e8547e37902 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.566102 12409 tablet_replica.cc:333] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.566937 12409 raft_consensus.cc:2243] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.567682 12409 raft_consensus.cc:2272] T 12c636bac0584e2e92ff66a089d880ee P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.570657 12409 tablet_replica.cc:333] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.571461 12409 raft_consensus.cc:2243] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.572343 12409 raft_consensus.cc:2272] T f2ce7a788b8c40629fc1921a4286d084 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.575124 12409 tablet_replica.cc:333] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.587328 12409 raft_consensus.cc:2243] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.588536 12409 raft_consensus.cc:2272] T fa97d994ab11478b9f735ced13bc6d48 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.596145 12409 tablet_replica.cc:333] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.599061 12409 raft_consensus.cc:2243] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.605245 12409 raft_consensus.cc:2272] T 5db10903ab7d403ba303d72b7f26dfed P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.608695 12409 tablet_replica.cc:333] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.621713 12409 raft_consensus.cc:2243] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Raft consensus shutting down.
W20260709 12:59:14.624583 13658 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:14.632791 12409 raft_consensus.cc:2272] T 5406eb61fd534169b3c1645f80144779 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.643283 12409 tablet_replica.cc:333] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.644392 12409 raft_consensus.cc:2243] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.645164 12409 raft_consensus.cc:2272] T be11851ed11044eea9f0fc3dbdbe0599 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.654325 13658 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:59:14.655128 12409 tablet_replica.cc:333] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.657111 12409 raft_consensus.cc:2243] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.660605 12409 raft_consensus.cc:2272] T 048528e413ac4c67a1cfd3dfe70673c7 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.668174 12409 tablet_replica.cc:333] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.672655 12409 raft_consensus.cc:2243] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.673478 12409 raft_consensus.cc:2272] T 973866cfc22e48ea931cb1a6fb16872f P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.678828 12409 tablet_replica.cc:333] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.683874 12409 raft_consensus.cc:2243] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.684906 12409 raft_consensus.cc:2272] T 72d95a7092b0445eb00a28a1fdb8d239 P d35f063cd3f942b0bf559b7c6e9b299f [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.695906 13658 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:59:14.718582 12409 tablet_replica.cc:333] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.719723 12409 raft_consensus.cc:2243] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.721065 12409 raft_consensus.cc:2272] T 2ee08c4df55f41c7a18951168c44fb5a P d35f063cd3f942b0bf559b7c6e9b299f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.729065 12409 tablet_replica.cc:333] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.732563 12409 raft_consensus.cc:2243] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.737018 12409 raft_consensus.cc:2272] T 6dd04e83486049ae859032792f652869 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.742091 12409 tablet_replica.cc:333] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.743079 12409 raft_consensus.cc:2243] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.744463 12409 raft_consensus.cc:2272] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.747704 12409 tablet_replica.cc:333] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.748585 12409 raft_consensus.cc:2243] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 8 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.751545 12409 raft_consensus.cc:2272] T 2181ddb48f3d44d583ad7ecf793222fa P d35f063cd3f942b0bf559b7c6e9b299f [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.765260 12409 tablet_replica.cc:333] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.766331 12409 raft_consensus.cc:2243] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.768878 12409 raft_consensus.cc:2272] T 91a3f49b0e164d27994cfe440af6f514 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.773649 12409 tablet_replica.cc:333] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.782546 12409 raft_consensus.cc:2243] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.783447 12409 raft_consensus.cc:2272] T 35f2068be68d418fb0b123e59c3db757 P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.786674 12409 tablet_replica.cc:333] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
W20260709 12:59:14.773861 13658 consensus_peers.cc:597] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:59:14.799546 12409 raft_consensus.cc:2243] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.805881 12409 raft_consensus.cc:2272] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P d35f063cd3f942b0bf559b7c6e9b299f [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.818204 12409 tablet_replica.cc:333] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
W20260709 12:59:14.807833 13658 consensus_peers.cc:597] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:59:14.826079 12409 raft_consensus.cc:2243] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.832370 12409 raft_consensus.cc:2272] T a315f47e47184081bd93e74c72dba511 P d35f063cd3f942b0bf559b7c6e9b299f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.835618 12409 tablet_replica.cc:333] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.836651 12409 raft_consensus.cc:2243] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.848364 12409 raft_consensus.cc:2272] T bcd2fb69161943208c2aef45acb3c33d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.851846 12409 tablet_replica.cc:333] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.852768 12409 raft_consensus.cc:2243] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.854431 12409 raft_consensus.cc:2272] T c88adb75300c48e98d20928bce53551d P d35f063cd3f942b0bf559b7c6e9b299f [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.857643 12409 tablet_replica.cc:333] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.866274 12409 raft_consensus.cc:2243] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.867532 12409 raft_consensus.cc:2272] T bc9a7a0b2fe343e48ea3ff1bae778069 P d35f063cd3f942b0bf559b7c6e9b299f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.870219 12409 tablet_replica.cc:333] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.871018 12409 raft_consensus.cc:2243] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.876029 12409 raft_consensus.cc:2272] T 68566bdf08224971a44a28240a8bf9e4 P d35f063cd3f942b0bf559b7c6e9b299f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.879006 12409 tablet_replica.cc:333] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.884708 12409 raft_consensus.cc:2243] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:14.886062 12409 raft_consensus.cc:2272] T 32b8156356cf4b709a2903d6861a117c P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.888826 12409 tablet_replica.cc:333] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.911573 12409 raft_consensus.cc:2243] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.912488 12409 raft_consensus.cc:2272] T 1a689534693d4469befa622d6b4917e3 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:14.915769 12409 tablet_replica.cc:333] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f: stopping tablet replica
I20260709 12:59:14.928591 12409 raft_consensus.cc:2243] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:14.929445 12409 raft_consensus.cc:2272] T 12822fb99cf24dbcac8dbb9d77e56af8 P d35f063cd3f942b0bf559b7c6e9b299f [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:14.950621 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:14.961803 13658 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:59:14.991683 13658 consensus_peers.cc:597] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:15.235440 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:59:15.269794 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:59:15.353041 19763 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.353729 19763 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.360369 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
W20260709 12:59:15.362394 13659 leader_election.cc:336] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.363351 19763 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:15.367432 13658 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.372793 19772 raft_consensus.cc:2749] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:59:15.503733 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:15.518802 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:59:15.557812 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:59:15.577695 19778 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.578423 19778 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:15.585980 14212 leader_election.cc:336] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.587541 19778 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:15.594601 19772 raft_consensus.cc:493] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.595280 19772 raft_consensus.cc:515] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:15.596547 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:15.597211 13693 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:59:15.597662 19772 leader_election.cc:290] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:15.601534 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.603865 19778 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:59:15.604302 19778 raft_consensus.cc:493] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.607065 19778 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:59:15.607612 13658 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:15.608237 13658 leader_election.cc:336] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:15.612174 13659 leader_election.cc:336] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.612727 13659 leader_election.cc:304] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.613528 19763 raft_consensus.cc:2749] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:15.616199 19778 raft_consensus.cc:515] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.626952 19763 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:59:15.630610 13658 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:15.630350 19763 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.632663 19763 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:15.633663 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:15.636791 19776 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:59:15.637486 13659 leader_election.cc:336] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.637320 19776 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:15.639104 14212 leader_election.cc:336] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.639585 19776 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:15.642457 13658 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.645287 19756 raft_consensus.cc:2749] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:59:15.646009 14211 leader_election.cc:336] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:15.647225 14212 leader_election.cc:336] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.647739 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.649854 19776 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:59:15.650915 13658 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:15.656528 19778 leader_election.cc:290] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:15.657759 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e634cc18febe4680bb70a3673cabbae4" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:15.661136 13693 raft_consensus.cc:3060] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:59:15.663467 19763 raft_consensus.cc:493] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.664299 19763 raft_consensus.cc:515] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:15.669629 19763 leader_election.cc:290] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:15.671214 13693 raft_consensus.cc:2468] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 17.
W20260709 12:59:15.671751 13658 leader_election.cc:336] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:15.684321 13659 leader_election.cc:336] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.685257 14209 leader_election.cc:304] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.685606 13659 leader_election.cc:304] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.686088 19778 raft_consensus.cc:2804] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:59:15.686719 19778 raft_consensus.cc:697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 17 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:15.687737 19778 consensus_queue.cc:237] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.687871 19763 raft_consensus.cc:2749] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:15.714604 13440 catalog_manager.cc:5697] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 16 to 17, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 17 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:59:15.751274 13658 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:15.808132 19778 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.808905 19778 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.816934 19778 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:15.820055 14212 leader_election.cc:336] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.823491 19778 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.824779 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:15.825573 13693 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 13.
I20260709 12:59:15.827754 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.828934 19776 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:59:15.829321 19776 raft_consensus.cc:493] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.829694 19776 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:59:15.824261 19778 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:15.838371 13658 consensus_peers.cc:597] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:15.840621 19772 raft_consensus.cc:493] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.841171 19772 raft_consensus.cc:515] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:15.843533 19772 leader_election.cc:290] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:15.843536 14211 leader_election.cc:336] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:15.844973 14212 leader_election.cc:336] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.845876 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.847718 19779 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:59:15.849210 13658 leader_election.cc:336] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:15.851953 13659 leader_election.cc:336] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.852456 13659 leader_election.cc:304] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.852649 19778 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:15.851624 19776 raft_consensus.cc:515] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.853915 19772 raft_consensus.cc:2749] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:15.857492 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9bdd77e564429c9c0434b358f48302" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:15.858150 13693 raft_consensus.cc:3060] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:59:15.859663 14212 leader_election.cc:336] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.862633 19779 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.863229 19779 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.869355 19779 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:15.870146 14212 leader_election.cc:336] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.871484 13693 raft_consensus.cc:2468] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 14.
I20260709 12:59:15.856863 19776 leader_election.cc:290] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:15.874423 14209 leader_election.cc:304] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.875314 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:15.876008 19776 raft_consensus.cc:2804] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:59:15.879367 19776 raft_consensus.cc:697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:15.883275 14209 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.880545 19776 consensus_queue.cc:237] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.884534 19779 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:59:15.924197 13440 catalog_manager.cc:5697] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 13 to 14, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 14 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:15.937585 19776 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:59:15.939893 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:59:15.938171 19776 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:15.947844 14212 leader_election.cc:336] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:15.953418 14211 leader_election.cc:336] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:15.953933 14211 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.954838 19779 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:15.966962 19772 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.967526 19772 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.970399 19776 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:15.976694 13659 leader_election.cc:336] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.978776 19772 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:15.978892 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:15.980504 13658 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: 0f34cda059124f839711dbef61bb2c28, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.982014 19763 raft_consensus.cc:2749] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:15.983865 19776 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:15.984464 19776 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:15.986553 19776 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:15.991701 14211 leader_election.cc:336] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:15.993407 14212 leader_election.cc:336] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:15.993865 14212 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:15.995118 19776 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:16.013612 19778 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.014192 19778 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.016273 19778 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:16.020282 13658 consensus_peers.cc:597] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:59:16.021802 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:16.024761 14212 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.026557 19778 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.027117 19778 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.028436 14209 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.029659 19779 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:16.031741 19776 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.032275 19776 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.034651 19776 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:16.041270 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:16.044962 14212 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.051895 14209 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.056305 19779 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:59:16.056821 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
W20260709 12:59:16.062529 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:59:16.063629 19778 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:16.066388 19772 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.067231 19772 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.076339 19772 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:16.076977 19763 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.077608 19763 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.079747 19763 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:59:16.089118 13659 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:16.091881 13659 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:16.092927 14212 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:16.093765 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:59:16.093873 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:16.094319 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:16.094563 14242 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:59:16.094971 14246 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 8.
I20260709 12:59:16.098428 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
W20260709 12:59:16.099257 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 45854.6us
W20260709 12:59:16.100885 13658 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:59:16.101197 14209 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.102947 13693 raft_consensus.cc:1275] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:59:16.102856 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.104061 19763 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:59:16.104417 19763 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.104496 19779 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:16.104799 19763 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:59:16.106227 19778 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.106701 19778 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.108856 19778 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:16.114564 19763 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.104527 13658 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.116830 19756 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:59:16.117174 19756 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.117530 19756 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:59:16.116379 19772 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.124271 19772 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:16.125451 14211 leader_election.cc:336] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:16.126350 14212 leader_election.cc:336] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.126513 19772 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:16.126785 14212 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.130036 19776 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:59:16.152005 19756 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.154289 19756 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:16.168562 19763 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:16.188833 19789 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.189644 19789 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:16.198637 13658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:39527, user_credentials={real_user=slave}} blocked reactor thread for 81454.5us
W20260709 12:59:16.201656 14211 leader_election.cc:336] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:16.200830 19789 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:16.201077 13659 leader_election.cc:336] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.204980 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:16.205654 14242 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 17.
W20260709 12:59:16.204552 14212 leader_election.cc:336] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.206596 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.207202 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2181ddb48f3d44d583ad7ecf793222fa" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:16.208001 14246 raft_consensus.cc:3060] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:59:16.209278 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa97d994ab11478b9f735ced13bc6d48" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:16.209851 14242 raft_consensus.cc:3060] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:59:16.212603 19778 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.213236 19778 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.216642 19789 consensus_queue.cc:1048] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:59:16.215799 19778 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:16.220420 13658 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:59:16.220417 19790 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
W20260709 12:59:16.222290 13659 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:16.227258 13659 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.227509 19778 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:16.225631 19790 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.234776 19790 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:16.218632 14242 raft_consensus.cc:2468] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 14.
I20260709 12:59:16.248241 13693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a" is_pre_election: true
I20260709 12:59:16.248463 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.248968 13693 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 16.
I20260709 12:59:16.251833 19792 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:59:16.252312 19792 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.252678 19792 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:59:16.236197 14212 leader_election.cc:336] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.254199 13658 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:59:16.257539 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:59:16.258551 14246 raft_consensus.cc:2468] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 9.
W20260709 12:59:16.263873 14212 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.264125 19790 raft_consensus.cc:2804] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:59:16.264683 19790 raft_consensus.cc:697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:59:16.265642 19790 consensus_queue.cc:237] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.270596 13658 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.274617 19763 raft_consensus.cc:2804] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:59:16.275204 19763 raft_consensus.cc:697] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
W20260709 12:59:16.276889 13659 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:16.277590 13659 consensus_peers.cc:597] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.282589 19763 consensus_queue.cc:237] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.276938 14246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28" is_pre_election: true
I20260709 12:59:16.286616 14246 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 12.
I20260709 12:59:16.288733 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.290045 19789 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:59:16.290411 19789 raft_consensus.cc:493] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.290755 19789 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:59:16.294090 19792 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:16.316931 14212 consensus_peers.cc:597] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.317771 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.320050 19772 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:59:16.318226 19789 raft_consensus.cc:515] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.320461 19772 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Starting leader election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.320909 19772 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:59:16.322005 19789 leader_election.cc:290] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:16.323328 19792 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
W20260709 12:59:16.328411 13659 consensus_peers.cc:597] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.335722 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9a7a0b2fe343e48ea3ff1bae778069" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:16.336550 14242 raft_consensus.cc:3060] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:59:16.337764 13690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a315f47e47184081bd93e74c72dba511" candidate_uuid: "0f34cda059124f839711dbef61bb2c28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "058c47afa94546dba814cf345f414f4a"
I20260709 12:59:16.338363 13690 raft_consensus.cc:3060] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:59:16.338889 19772 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.345160 19792 raft_consensus.cc:493] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:16.345809 19792 raft_consensus.cc:515] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } }
I20260709 12:59:16.347782 19792 leader_election.cc:290] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
I20260709 12:59:16.348693 13690 raft_consensus.cc:2468] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34cda059124f839711dbef61bb2c28 in term 17.
W20260709 12:59:16.350697 13659 leader_election.cc:336] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.356467 14242 raft_consensus.cc:2468] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 18.
W20260709 12:59:16.363271 14212 leader_election.cc:336] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:16.364290 19772 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Requested vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 0f34cda059124f839711dbef61bb2c28 (127.12.30.68:39527)
I20260709 12:59:16.368880 13658 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.370350 19772 raft_consensus.cc:2804] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:59:16.370858 19772 raft_consensus.cc:697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 18 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:59:16.375562 14209 leader_election.cc:304] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.375679 13440 catalog_manager.cc:5697] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 13 to 14, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 14 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.377867 13440 catalog_manager.cc:5697] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 8 to 9, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 9 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.378878 19789 raft_consensus.cc:2804] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:59:16.379452 19789 raft_consensus.cc:697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 17 LEADER]: Becoming Leader. State: Replica: 0f34cda059124f839711dbef61bb2c28, State: Running, Role: LEADER
I20260709 12:59:16.373027 19772 consensus_queue.cc:237] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.380447 19789 consensus_queue.cc:237] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:16.387339 14242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcd2fb69161943208c2aef45acb3c33d" candidate_uuid: "058c47afa94546dba814cf345f414f4a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0f34cda059124f839711dbef61bb2c28"
I20260709 12:59:16.388361 14242 raft_consensus.cc:3060] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:59:16.392992 13659 leader_election.cc:336] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:16.401454 13658 leader_election.cc:336] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:16.408891 13658 leader_election.cc:304] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 058c47afa94546dba814cf345f414f4a; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:16.397254 14242 raft_consensus.cc:2468] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 058c47afa94546dba814cf345f414f4a in term 13.
I20260709 12:59:16.411759 19772 raft_consensus.cc:2749] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:16.413303 13658 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058c47afa94546dba814cf345f414f4a, 0f34cda059124f839711dbef61bb2c28; no voters: 
I20260709 12:59:16.416160 19772 raft_consensus.cc:2804] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:59:16.417351 19772 raft_consensus.cc:697] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Becoming Leader. State: Replica: 058c47afa94546dba814cf345f414f4a, State: Running, Role: LEADER
I20260709 12:59:16.418843 19772 consensus_queue.cc:237] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:16.429113 13659 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:16.432391 13658 consensus_peers.cc:597] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:16.434255 14212 consensus_peers.cc:597] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.434300 13440 catalog_manager.cc:5697] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 reported cstate change: term changed from 16 to 17, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 0f34cda059124f839711dbef61bb2c28 (127.12.30.68). New cstate: current_term: 17 leader_uuid: "0f34cda059124f839711dbef61bb2c28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:59:16.439270 13659 consensus_peers.cc:597] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.460806 13690 raft_consensus.cc:1275] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Refusing update from remote peer 0f34cda059124f839711dbef61bb2c28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:59:16.461272 13440 catalog_manager.cc:5697] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 17 to 18, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 18 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:16.462364 19789 consensus_queue.cc:1048] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:59:16.462988 13440 catalog_manager.cc:5697] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a reported cstate change: term changed from 12 to 13, leader changed from d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66) to 058c47afa94546dba814cf345f414f4a (127.12.30.67). New cstate: current_term: 13 leader_uuid: "058c47afa94546dba814cf345f414f4a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:16.471568 14212 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.475481 13659 consensus_peers.cc:597] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.485108 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:59:16.485543 13659 consensus_peers.cc:597] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.486078 13659 consensus_peers.cc:597] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.505272 13659 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.511745 13659 consensus_peers.cc:597] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.572741 13659 consensus_peers.cc:597] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.581528 14212 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.583983 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
W20260709 12:59:16.591836 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:59:16.598501 14209 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:16.630052 14209 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:16.631880 13659 consensus_peers.cc:597] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.655874 14212 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.659570 13658 consensus_peers.cc:597] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:16.663450 14209 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:16.698148 13659 consensus_peers.cc:597] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:16.703399 13659 consensus_peers.cc:597] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.711081 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:59:16.727164 14212 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.728314 12409 tablet_replica.cc:333] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.729396 12409 raft_consensus.cc:2243] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.730628 12409 raft_consensus.cc:2272] T c665edf3d7834e18a67df3f75b30856a P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.733610 12409 tablet_replica.cc:333] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.734498 12409 raft_consensus.cc:2243] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:16.735088 12409 raft_consensus.cc:2272] T 835fe463242d40e5b82ddb5898323fc5 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.738162 12409 tablet_replica.cc:333] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
W20260709 12:59:16.738828 14212 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.740223 13658 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:59:16.742544 13659 consensus_peers.cc:597] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.738966 12409 raft_consensus.cc:2243] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:16.759413 12409 raft_consensus.cc:2272] T d9eb8a2c5d7846099d30c0839aefc5a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.752486 14209 consensus_peers.cc:597] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.754045 14242 raft_consensus.cc:1275] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:59:16.761620 19772 consensus_queue.cc:1048] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:59:16.762241 12409 tablet_replica.cc:333] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.763135 12409 raft_consensus.cc:2243] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.764213 12409 raft_consensus.cc:2272] T 75988f1a2dda4bd6962cd5f3ad1c304a P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.767160 12409 tablet_replica.cc:333] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.768307 12409 raft_consensus.cc:2243] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:16.769201 12409 raft_consensus.cc:2272] T e634cc18febe4680bb70a3673cabbae4 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.775856 12409 tablet_replica.cc:333] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.782794 12409 raft_consensus.cc:2243] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.785943 12409 raft_consensus.cc:2272] T 1c2d2db0b5674796bbd0426953d99538 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.796826 14212 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.797019 12409 tablet_replica.cc:333] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.798075 12409 raft_consensus.cc:2243] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.799253 12409 raft_consensus.cc:2272] T 3f54dde33fa845098e64673b576d780c P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.802227 12409 tablet_replica.cc:333] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.803200 12409 raft_consensus.cc:2243] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Raft consensus shutting down.
W20260709 12:59:16.810207 13659 consensus_peers.cc:597] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.810644 12409 pending_rounds.cc:70] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Trying to abort 1 pending ops.
I20260709 12:59:16.810926 12409 pending_rounds.cc:77] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a: Aborting op as it isn't in flight: id { term: 13 index: 10 } timestamp: 7305633613507678208 op_type: NO_OP noop_request { }
I20260709 12:59:16.811326 12409 raft_consensus.cc:2889] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:59:16.811661 12409 raft_consensus.cc:2272] T bcd2fb69161943208c2aef45acb3c33d P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.815013 13659 consensus_peers.cc:597] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.819658 12409 tablet_replica.cc:333] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.820595 12409 raft_consensus.cc:2243] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.822009 12409 raft_consensus.cc:2272] T 56f36011554a4cd78d7b17b92a823c8f P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.824923 12409 tablet_replica.cc:333] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.834766 12409 raft_consensus.cc:2243] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:16.835476 12409 raft_consensus.cc:2272] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.836462 13659 consensus_peers.cc:597] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:16.836995 13659 consensus_peers.cc:597] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.838632 12409 tablet_replica.cc:333] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.839607 12409 raft_consensus.cc:2243] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.840777 12409 raft_consensus.cc:2272] T 9d739d3dda4e4370b33be67fca1c66d5 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.841060 14242 raft_consensus.cc:1275] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:59:16.842716 19792 consensus_queue.cc:1048] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:59:16.843797 12409 tablet_replica.cc:333] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.852972 12409 raft_consensus.cc:2243] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.854902 12409 raft_consensus.cc:2272] T 2c5bef52669b468c8765408698f5884d P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.862504 14212 consensus_peers.cc:597] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.862542 12409 tablet_replica.cc:333] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.869462 12409 raft_consensus.cc:2243] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:16.870815 12409 raft_consensus.cc:2272] T 22f14b8d35f741a28ad886b19614c4c8 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.873980 12409 tablet_replica.cc:333] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.875013 12409 raft_consensus.cc:2243] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.876197 12409 raft_consensus.cc:2272] T 86d7b5c530f14daaade417c2e3903072 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.882474 12409 tablet_replica.cc:333] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.883452 12409 raft_consensus.cc:2243] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:16.892009 12409 raft_consensus.cc:2272] T c827d49183b8406b9adff9666f6d83a9 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.893704 13658 consensus_peers.cc:597] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:59:16.894709 12409 tablet_replica.cc:333] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.895648 12409 raft_consensus.cc:2243] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.896803 12409 raft_consensus.cc:2272] T f9f9e2a62b8a498d93662b589c6a6870 P 058c47afa94546dba814cf345f414f4a [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.897553 14211 consensus_peers.cc:597] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:59:16.899662 12409 tablet_replica.cc:333] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.904605 12409 raft_consensus.cc:2243] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 LEADER]: Raft consensus shutting down.
W20260709 12:59:16.905982 14209 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.906518 12409 raft_consensus.cc:2272] T 921257a2480244e88f3ff036fb3be8d2 P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.910841 14212 consensus_peers.cc:597] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:16.906725 14209 consensus_peers.cc:597] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:16.913209 13659 consensus_peers.cc:597] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.914897 12409 tablet_replica.cc:333] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.916484 12409 raft_consensus.cc:2243] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.924183 12409 raft_consensus.cc:2272] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.928984 12409 tablet_replica.cc:333] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.929945 12409 raft_consensus.cc:2243] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:16.930702 12409 raft_consensus.cc:2272] T 6b48fb5973b84b06a9ad858a9b3967f8 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.934868 12409 tablet_replica.cc:333] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.935786 12409 raft_consensus.cc:2243] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Raft consensus shutting down.
W20260709 12:59:16.936225 14209 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:16.937080 14212 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:16.937150 12409 raft_consensus.cc:2272] T 5c25c2050d4547c39924ceb739727ae8 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.940704 12409 tablet_replica.cc:333] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
W20260709 12:59:16.940912 14212 consensus_peers.cc:597] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.941728 12409 raft_consensus.cc:2243] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.943565 12409 raft_consensus.cc:2272] T 6af1705dc4aa4e738a8416263f5af754 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.947906 12409 tablet_replica.cc:333] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.949934 12409 raft_consensus.cc:2243] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Raft consensus shutting down.
W20260709 12:59:16.949864 14209 consensus_peers.cc:597] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.961416 12409 raft_consensus.cc:2272] T be389b2236484464bc3e9f558d7b4230 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.966526 12409 tablet_replica.cc:333] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.972604 12409 raft_consensus.cc:2243] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:16.973793 12409 raft_consensus.cc:2272] T f2ce7a788b8c40629fc1921a4286d084 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:16.974467 14209 consensus_peers.cc:597] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.977084 14246 raft_consensus.cc:1275] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Refusing update from remote peer 058c47afa94546dba814cf345f414f4a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:59:16.978646 19792 consensus_queue.cc:1048] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:59:16.980844 12409 tablet_replica.cc:333] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.981819 12409 raft_consensus.cc:2243] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Raft consensus shutting down.
W20260709 12:59:16.982995 14209 consensus_peers.cc:597] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:16.994091 12409 raft_consensus.cc:2272] T 87a8c869a6b840e591423340bc32af00 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:16.998821 12409 tablet_replica.cc:333] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:16.999614 12409 raft_consensus.cc:2243] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.008427 12409 raft_consensus.cc:2272] T fa97d994ab11478b9f735ced13bc6d48 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.012816 12409 tablet_replica.cc:333] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.013737 12409 raft_consensus.cc:2243] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.014847 12409 raft_consensus.cc:2272] T be11851ed11044eea9f0fc3dbdbe0599 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:17.049522 14212 consensus_peers.cc:597] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:17.054986 12409 tablet_replica.cc:333] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.056344 12409 raft_consensus.cc:2243] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.063318 12409 raft_consensus.cc:2272] T 82077fd60c1346d7b057a65a778ee2f5 P 058c47afa94546dba814cf345f414f4a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.067133 12409 tablet_replica.cc:333] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.068439 12409 raft_consensus.cc:2243] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.069185 12409 raft_consensus.cc:2272] T 4b9bdd77e564429c9c0434b358f48302 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.073879 12409 tablet_replica.cc:333] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.074676 12409 raft_consensus.cc:2243] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.075995 12409 raft_consensus.cc:2272] T 1c9f20cfde5343c9b91c4e2adc5320ce P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.092371 12409 tablet_replica.cc:333] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.093230 12409 raft_consensus.cc:2243] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.098397 12409 raft_consensus.cc:2272] T bc9a7a0b2fe343e48ea3ff1bae778069 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.101933 12409 tablet_replica.cc:333] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.102813 12409 raft_consensus.cc:2243] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.104610 12409 raft_consensus.cc:2272] T c88adb75300c48e98d20928bce53551d P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.108027 12409 tablet_replica.cc:333] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.109341 12409 raft_consensus.cc:2243] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.110386 12409 raft_consensus.cc:2272] T e7d1ab18f48a4689857964b7200b72c9 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.116569 12409 tablet_replica.cc:333] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.117343 12409 raft_consensus.cc:2243] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.117934 12409 raft_consensus.cc:2272] T a315f47e47184081bd93e74c72dba511 P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.120843 12409 tablet_replica.cc:333] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
W20260709 12:59:17.122088 14209 consensus_peers.cc:597] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:17.122949 12409 raft_consensus.cc:2243] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.123621 12409 raft_consensus.cc:2272] T 5db10903ab7d403ba303d72b7f26dfed P 058c47afa94546dba814cf345f414f4a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.126418 12409 tablet_replica.cc:333] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.133221 12409 raft_consensus.cc:2243] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.140190 12409 raft_consensus.cc:2272] T 5406eb61fd534169b3c1645f80144779 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.143129 12409 tablet_replica.cc:333] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.143953 12409 raft_consensus.cc:2243] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.157625 12409 raft_consensus.cc:2272] T 86a0c010e8cc4b828fec8e8547e37902 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.160351 12409 tablet_replica.cc:333] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.161593 12409 raft_consensus.cc:2243] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.162722 12409 raft_consensus.cc:2272] T 12c636bac0584e2e92ff66a089d880ee P 058c47afa94546dba814cf345f414f4a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.165587 12409 tablet_replica.cc:333] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.167160 12409 raft_consensus.cc:2243] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.167778 12409 raft_consensus.cc:2272] T 68566bdf08224971a44a28240a8bf9e4 P 058c47afa94546dba814cf345f414f4a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.170879 12409 tablet_replica.cc:333] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.171877 12409 raft_consensus.cc:2243] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.173014 12409 raft_consensus.cc:2272] T 12822fb99cf24dbcac8dbb9d77e56af8 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.176091 12409 tablet_replica.cc:333] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.181663 12409 raft_consensus.cc:2243] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.184175 12409 raft_consensus.cc:2272] T 35f2068be68d418fb0b123e59c3db757 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.187664 12409 tablet_replica.cc:333] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.188584 12409 raft_consensus.cc:2243] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.197918 12409 raft_consensus.cc:2272] T 2181ddb48f3d44d583ad7ecf793222fa P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.219543 12409 tablet_replica.cc:333] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.221664 12409 raft_consensus.cc:2243] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.222980 12409 raft_consensus.cc:2272] T 6dd04e83486049ae859032792f652869 P 058c47afa94546dba814cf345f414f4a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.226614 12409 tablet_replica.cc:333] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.227348 12409 raft_consensus.cc:2243] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:59:17.244374 12409 raft_consensus.cc:2272] T 019c73868e744f4eba4ac7d97d34a137 P 058c47afa94546dba814cf345f414f4a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.264681 12409 tablet_replica.cc:333] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.265360 12409 raft_consensus.cc:2243] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 LEADER]: Raft consensus shutting down.
W20260709 12:59:17.265437 14209 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 637 similar messages]
I20260709 12:59:17.266654 12409 raft_consensus.cc:2272] T f97a6462cd4341ddbd7058ce9537c1eb P 058c47afa94546dba814cf345f414f4a [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.276927 12409 tablet_replica.cc:333] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a: stopping tablet replica
I20260709 12:59:17.277627 12409 raft_consensus.cc:2243] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:17.278229 12409 raft_consensus.cc:2272] T 85eb0ac09bf7466899637997aae42111 P 058c47afa94546dba814cf345f414f4a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:17.551187 19779 raft_consensus.cc:493] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:17.551743 19779 raft_consensus.cc:515] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:17.554076 19779 leader_election.cc:290] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:17.563669 14211 leader_election.cc:336] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:17.564438 14212 leader_election.cc:336] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:17.564842 14212 leader_election.cc:304] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:17.566329 19806 raft_consensus.cc:2749] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:59:17.610725 14211 consensus_peers.cc:597] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:59:17.621758 19779 raft_consensus.cc:493] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:17.622502 19779 raft_consensus.cc:515] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:17.625161 19779 leader_election.cc:290] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:17.631222 14212 leader_election.cc:336] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:17.642896 14211 leader_election.cc:336] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:17.643400 14211 leader_election.cc:304] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:17.645143 19779 raft_consensus.cc:2749] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:17.645273 19806 raft_consensus.cc:493] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:17.645888 19806 raft_consensus.cc:515] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:17.648061 19806 leader_election.cc:290] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:17.650179 14211 consensus_peers.cc:597] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:59:17.652814 14212 leader_election.cc:336] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:17.660897 14211 leader_election.cc:336] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:17.661460 14211 leader_election.cc:304] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:17.663288 19779 raft_consensus.cc:2749] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:59:17.736851 19779 raft_consensus.cc:493] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:17.737488 19779 raft_consensus.cc:515] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:17.739768 19779 leader_election.cc:290] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:17.750046 14211 leader_election.cc:336] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:17.751050 14212 leader_election.cc:336] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:17.751555 14212 leader_election.cc:304] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:17.752713 19779 raft_consensus.cc:2749] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:59:17.835719 19779 raft_consensus.cc:493] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:17.836400 19779 raft_consensus.cc:515] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:17.844659 14211 consensus_peers.cc:597] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:17.845389 14211 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:17.846397 19779 leader_election.cc:290] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:17.850078 14209 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:17.850538 14209 leader_election.cc:304] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:17.851289 19779 raft_consensus.cc:2749] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:59:17.851485 14212 leader_election.cc:336] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:17.854557 19779 raft_consensus.cc:493] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:17.855185 19779 raft_consensus.cc:515] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:17.857582 19779 leader_election.cc:290] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:17.863948 14211 leader_election.cc:336] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:17.866439 14209 leader_election.cc:336] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:17.866871 14209 leader_election.cc:304] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:17.867633 19779 raft_consensus.cc:2749] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:59:17.877744 19779 raft_consensus.cc:493] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:17.878369 19779 raft_consensus.cc:515] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:17.881199 19779 leader_election.cc:290] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:17.885838 14211 consensus_peers.cc:597] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:59:17.888799 14212 leader_election.cc:336] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:17.893139 19779 raft_consensus.cc:493] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:59:17.893594 14211 leader_election.cc:336] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:17.894145 14211 leader_election.cc:304] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:17.894935 19789 raft_consensus.cc:2749] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:59:17.893730 19779 raft_consensus.cc:515] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:17.897634 19779 leader_election.cc:290] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:17.903221 14209 leader_election.cc:336] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
W20260709 12:59:17.904039 14211 leader_election.cc:336] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:17.904553 14211 leader_election.cc:304] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:17.905408 19779 raft_consensus.cc:2749] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:59:17.967002 14211 consensus_peers.cc:597] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:17.984368 14211 consensus_peers.cc:597] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 12:59:18.077569 19779 raft_consensus.cc:493] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.078285 19779 raft_consensus.cc:515] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.085631 19779 leader_election.cc:290] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.099493 14209 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
W20260709 12:59:18.100369 14211 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:18.100798 14211 leader_election.cc:304] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:18.101649 19779 raft_consensus.cc:493] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.102210 19779 raft_consensus.cc:515] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.104195 19806 raft_consensus.cc:2749] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:59:18.104811 19779 leader_election.cc:290] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:18.104990 19806 raft_consensus.cc:493] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.105410 19779 raft_consensus.cc:493] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d35f063cd3f942b0bf559b7c6e9b299f)
I20260709 12:59:18.105542 19806 raft_consensus.cc:515] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.105917 19779 raft_consensus.cc:515] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.107882 19806 leader_election.cc:290] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:18.108108 19779 leader_election.cc:290] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367)
W20260709 12:59:18.120821 14209 leader_election.cc:336] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
W20260709 12:59:18.121608 14209 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
W20260709 12:59:18.122623 14211 leader_election.cc:336] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:18.123252 14211 leader_election.cc:304] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:18.124282 19808 raft_consensus.cc:2749] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:59:18.124336 14211 leader_election.cc:336] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:18.125156 14211 leader_election.cc:304] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:59:18.126219 14211 leader_election.cc:336] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:18.126245 19808 raft_consensus.cc:2749] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:59:18.127202 14212 leader_election.cc:336] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:18.130882 14212 leader_election.cc:336] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.131313 14212 leader_election.cc:304] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: ad2fdb1517c542439654bd3f7c72dad0, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:18.132253 19808 raft_consensus.cc:2749] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:59:18.254477 14209 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:41755, user_credentials={real_user=slave}} blocked reactor thread for 65771.6us
I20260709 12:59:18.282500 19809 raft_consensus.cc:493] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.283106 19809 raft_consensus.cc:515] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } attrs { replace: true } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.288522 19809 leader_election.cc:290] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:18.297650 19806 raft_consensus.cc:493] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.298321 19806 raft_consensus.cc:515] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.299144 19789 raft_consensus.cc:493] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.299803 19789 raft_consensus.cc:515] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.302253 19789 leader_election.cc:290] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:18.312435 19806 leader_election.cc:290] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.332399 14209 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.333806 19809 raft_consensus.cc:493] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.334403 19809 raft_consensus.cc:515] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:18.337273 14209 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.338322 19808 raft_consensus.cc:493] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
W20260709 12:59:18.338708 14209 leader_election.cc:336] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.339058 19808 raft_consensus.cc:515] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:18.339843 14211 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:18.340315 14211 leader_election.cc:304] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:18.341073 19779 raft_consensus.cc:2749] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:59:18.343190 19808 leader_election.cc:290] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.343394 14211 consensus_peers.cc:597] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 12:59:18.346423 19809 leader_election.cc:290] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.349829 14211 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:59:18.350760 14211 leader_election.cc:336] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:18.351305 14211 leader_election.cc:304] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:18.352121 19789 raft_consensus.cc:2749] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:59:18.354650 14211 leader_election.cc:336] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
W20260709 12:59:18.369980 14209 leader_election.cc:336] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
W20260709 12:59:18.371538 14212 leader_election.cc:336] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:18.371861 14209 leader_election.cc:336] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.372097 14212 leader_election.cc:304] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:18.372280 19789 raft_consensus.cc:493] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.372351 14209 leader_election.cc:304] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
I20260709 12:59:18.372922 19809 raft_consensus.cc:2749] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:18.373322 19808 raft_consensus.cc:2749] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:18.373214 19789 raft_consensus.cc:515] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:18.373886 14212 leader_election.cc:336] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.371454 19811 raft_consensus.cc:493] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.374956 19811 raft_consensus.cc:515] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.375700 19789 leader_election.cc:290] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:18.377121 19811 leader_election.cc:290] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.388837 14209 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
W20260709 12:59:18.389274 14212 leader_election.cc:336] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
W20260709 12:59:18.389472 14209 leader_election.cc:336] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.389799 14212 leader_election.cc:304] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:18.390576 19811 raft_consensus.cc:2749] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:59:18.390867 14212 leader_election.cc:336] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.396606 14212 leader_election.cc:304] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:18.402671 19811 raft_consensus.cc:2749] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:59:18.405543 14212 leader_election.cc:336] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.406070 14212 leader_election.cc:304] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:18.407568 19811 raft_consensus.cc:2749] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:59:18.425696 19811 raft_consensus.cc:493] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.426411 19811 raft_consensus.cc:515] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:18.437960 14209 leader_election.cc:336] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.439160 19811 leader_election.cc:290] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.449890 14211 consensus_peers.cc:597] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:59:18.453469 14212 leader_election.cc:336] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.454133 14212 leader_election.cc:304] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:18.455140 19811 raft_consensus.cc:2749] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:18.494376 19811 raft_consensus.cc:493] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.495149 19811 raft_consensus.cc:515] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad2fdb1517c542439654bd3f7c72dad0" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 39209 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
W20260709 12:59:18.504014 14211 leader_election.cc:336] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:18.523618 19810 raft_consensus.cc:493] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:18.524238 19810 raft_consensus.cc:515] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.527004 19810 leader_election.cc:290] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
I20260709 12:59:18.528084 19811 leader_election.cc:290] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.536911 14209 leader_election.cc:336] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.537626 14209 leader_election.cc:304] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, ad2fdb1517c542439654bd3f7c72dad0
W20260709 12:59:18.538820 14209 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
W20260709 12:59:18.529311 14211 consensus_peers.cc:597] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:59:18.539801 19811 raft_consensus.cc:2749] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:18.541855 19789 raft_consensus.cc:493] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.544349 19789 raft_consensus.cc:515] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.546784 19789 leader_election.cc:290] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.559317 14209 leader_election.cc:336] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.561457 19811 raft_consensus.cc:493] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 058c47afa94546dba814cf345f414f4a)
I20260709 12:59:18.572062 19811 raft_consensus.cc:515] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d35f063cd3f942b0bf559b7c6e9b299f" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39367 } } peers { permanent_uuid: "058c47afa94546dba814cf345f414f4a" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41755 } } peers { permanent_uuid: "0f34cda059124f839711dbef61bb2c28" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 39527 } attrs { promote: false } }
I20260709 12:59:18.577797 19811 leader_election.cc:290] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367), 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755)
W20260709 12:59:18.587029 14212 leader_election.cc:336] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.592839 14212 leader_election.cc:304] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:59:18.593925 14212 leader_election.cc:336] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.596088 14212 leader_election.cc:304] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
W20260709 12:59:18.595144 14209 leader_election.cc:336] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111)
I20260709 12:59:18.594461 19811 raft_consensus.cc:2749] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:59:18.598803 19810 raft_consensus.cc:2749] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:59:18.613023 14212 leader_election.cc:336] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111)
I20260709 12:59:18.613642 14212 leader_election.cc:304] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f34cda059124f839711dbef61bb2c28; no voters: 058c47afa94546dba814cf345f414f4a, d35f063cd3f942b0bf559b7c6e9b299f
I20260709 12:59:18.614483 19811 raft_consensus.cc:2749] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:59:18.629746 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
W20260709 12:59:18.748588 14212 consensus_peers.cc:597] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:18.773561 14211 consensus_peers.cc:597] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 12:59:18.797991 12409 tablet_server.cc:179] TabletServer@127.12.30.68:0 shutting down...
W20260709 12:59:18.822996 14211 consensus_peers.cc:597] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 -> Peer ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): Couldn't send request to peer ad2fdb1517c542439654bd3f7c72dad0. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:59:18.867796 14212 consensus_peers.cc:597] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:59:18.909726 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:18.911001 12409 tablet_replica.cc:333] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.912251 12409 raft_consensus.cc:2243] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.913157 12409 raft_consensus.cc:2272] T 56f36011554a4cd78d7b17b92a823c8f P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.916661 12409 tablet_replica.cc:333] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.917709 12409 raft_consensus.cc:2243] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.918464 12409 raft_consensus.cc:2272] T 2c5bef52669b468c8765408698f5884d P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.922073 12409 tablet_replica.cc:333] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.923041 12409 raft_consensus.cc:2243] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.923841 12409 raft_consensus.cc:2272] T 2f6f9228cbd74e95b5192c80f7f4a2b6 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:18.925076 14212 consensus_peers.cc:597] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 -> Peer d35f063cd3f942b0bf559b7c6e9b299f (127.12.30.66:39367): Couldn't send request to peer d35f063cd3f942b0bf559b7c6e9b299f. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:18.927388 12409 tablet_replica.cc:333] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.928428 12409 raft_consensus.cc:2243] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.929538 12409 raft_consensus.cc:2272] T 6dd04e83486049ae859032792f652869 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.932986 12409 tablet_replica.cc:333] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.933889 12409 raft_consensus.cc:2243] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.934589 12409 raft_consensus.cc:2272] T 2ee08c4df55f41c7a18951168c44fb5a P 0f34cda059124f839711dbef61bb2c28 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.937871 12409 tablet_replica.cc:333] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.938807 12409 raft_consensus.cc:2243] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.939878 12409 raft_consensus.cc:2272] T 72d95a7092b0445eb00a28a1fdb8d239 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.942901 12409 tablet_replica.cc:333] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.943986 12409 raft_consensus.cc:2243] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.945256 12409 raft_consensus.cc:2272] T 835fe463242d40e5b82ddb5898323fc5 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.948870 12409 tablet_replica.cc:333] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.950883 12409 raft_consensus.cc:2243] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.951761 12409 raft_consensus.cc:2272] T be389b2236484464bc3e9f558d7b4230 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.955235 12409 tablet_replica.cc:333] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.956159 12409 raft_consensus.cc:2243] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.957418 12409 raft_consensus.cc:2272] T 1d73171142594388972ed22eae44f95f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.960636 12409 tablet_replica.cc:333] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.961561 12409 raft_consensus.cc:2243] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.962689 12409 raft_consensus.cc:2272] T e634cc18febe4680bb70a3673cabbae4 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.965875 12409 tablet_replica.cc:333] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.966828 12409 raft_consensus.cc:2243] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.967449 12409 raft_consensus.cc:2272] T 9aa892a8dfec433d8aa859d442e3e309 P 0f34cda059124f839711dbef61bb2c28 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.970727 12409 tablet_replica.cc:333] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.971629 12409 raft_consensus.cc:2243] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.972496 12409 raft_consensus.cc:2272] T 9d739d3dda4e4370b33be67fca1c66d5 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.975533 12409 tablet_replica.cc:333] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.976454 12409 raft_consensus.cc:2243] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.977664 12409 raft_consensus.cc:2272] T 1a689534693d4469befa622d6b4917e3 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.981292 12409 tablet_replica.cc:333] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.982491 12409 raft_consensus.cc:2243] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus shutting down.
W20260709 12:59:18.982498 14209 consensus_peers.cc:597] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 -> Peer 058c47afa94546dba814cf345f414f4a (127.12.30.67:41755): Couldn't send request to peer 058c47afa94546dba814cf345f414f4a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.67:41755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:18.983367 12409 raft_consensus.cc:2272] T 32b8156356cf4b709a2903d6861a117c P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.986809 12409 tablet_replica.cc:333] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.987768 12409 raft_consensus.cc:2243] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.988547 12409 raft_consensus.cc:2272] T 87a8c869a6b840e591423340bc32af00 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.991859 12409 tablet_replica.cc:333] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.992864 12409 raft_consensus.cc:2243] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.994127 12409 raft_consensus.cc:2272] T d9eb8a2c5d7846099d30c0839aefc5a9 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.997331 12409 tablet_replica.cc:333] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:18.998327 12409 raft_consensus.cc:2243] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.998992 12409 raft_consensus.cc:2272] T 2181ddb48f3d44d583ad7ecf793222fa P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.002246 12409 tablet_replica.cc:333] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.003175 12409 raft_consensus.cc:2243] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.003904 12409 raft_consensus.cc:2272] T 91a3f49b0e164d27994cfe440af6f514 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.007303 12409 tablet_replica.cc:333] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.008347 12409 raft_consensus.cc:2243] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.009189 12409 raft_consensus.cc:2272] T 921257a2480244e88f3ff036fb3be8d2 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.013006 12409 tablet_replica.cc:333] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.013922 12409 raft_consensus.cc:2243] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.014634 12409 raft_consensus.cc:2272] T d1c4c42ef7ad4d8fb669a6a5fe183f6f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.018285 12409 tablet_replica.cc:333] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.019246 12409 raft_consensus.cc:2243] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.020380 12409 pending_rounds.cc:70] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Trying to abort 1 pending ops.
I20260709 12:59:19.020637 12409 pending_rounds.cc:77] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28: Aborting op as it isn't in flight: id { term: 17 index: 14 } timestamp: 7305633613365657600 op_type: NO_OP noop_request { }
I20260709 12:59:19.021004 12409 raft_consensus.cc:2889] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 17 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:59:19.021376 12409 raft_consensus.cc:2272] T a315f47e47184081bd93e74c72dba511 P 0f34cda059124f839711dbef61bb2c28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.024260 12409 tablet_replica.cc:333] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.025154 12409 raft_consensus.cc:2243] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.025758 12409 raft_consensus.cc:2272] T ba2c2e5a7dac4eb4b3bacc18402ffa47 P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.028891 12409 tablet_replica.cc:333] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.029815 12409 raft_consensus.cc:2243] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.030550 12409 raft_consensus.cc:2272] T c88adb75300c48e98d20928bce53551d P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.034155 12409 tablet_replica.cc:333] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.035046 12409 raft_consensus.cc:2243] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.036536 12409 raft_consensus.cc:2272] T f125a6a76f974409807014b9ddaa1b8c P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.044198 12409 tablet_replica.cc:333] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.045411 12409 raft_consensus.cc:2243] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.046264 12409 raft_consensus.cc:2272] T bcd2fb69161943208c2aef45acb3c33d P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.050123 12409 tablet_replica.cc:333] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.051429 12409 raft_consensus.cc:2243] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.053309 12409 raft_consensus.cc:2272] T 3f7e1f6a8119482c9f8056bb615edfa8 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.057415 12409 tablet_replica.cc:333] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.058348 12409 raft_consensus.cc:2243] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.059070 12409 raft_consensus.cc:2272] T 12822fb99cf24dbcac8dbb9d77e56af8 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.063241 12409 tablet_replica.cc:333] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.064246 12409 raft_consensus.cc:2243] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.065640 12409 raft_consensus.cc:2272] T e7d1ab18f48a4689857964b7200b72c9 P 0f34cda059124f839711dbef61bb2c28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.069214 12409 tablet_replica.cc:333] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.070058 12409 raft_consensus.cc:2243] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.070813 12409 raft_consensus.cc:2272] T 1c2d2db0b5674796bbd0426953d99538 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.074154 12409 tablet_replica.cc:333] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.075027 12409 raft_consensus.cc:2243] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.075774 12409 raft_consensus.cc:2272] T bc9a7a0b2fe343e48ea3ff1bae778069 P 0f34cda059124f839711dbef61bb2c28 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.078745 12409 tablet_replica.cc:333] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.079612 12409 raft_consensus.cc:2243] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.081265 12409 raft_consensus.cc:2272] T 048528e413ac4c67a1cfd3dfe70673c7 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.084393 12409 tablet_replica.cc:333] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.085218 12409 raft_consensus.cc:2243] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.085857 12409 raft_consensus.cc:2272] T 82077fd60c1346d7b057a65a778ee2f5 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.088932 12409 tablet_replica.cc:333] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.089764 12409 raft_consensus.cc:2243] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.090775 12409 raft_consensus.cc:2272] T 75988f1a2dda4bd6962cd5f3ad1c304a P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.093763 12409 tablet_replica.cc:333] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.094583 12409 raft_consensus.cc:2243] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.095667 12409 raft_consensus.cc:2272] T 4b9bdd77e564429c9c0434b358f48302 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.098539 12409 tablet_replica.cc:333] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.099334 12409 raft_consensus.cc:2243] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.100682 12409 raft_consensus.cc:2272] T 85eb0ac09bf7466899637997aae42111 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.103767 12409 tablet_replica.cc:333] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.104666 12409 raft_consensus.cc:2243] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.105310 12409 raft_consensus.cc:2272] T fa97d994ab11478b9f735ced13bc6d48 P 0f34cda059124f839711dbef61bb2c28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.108600 12409 tablet_replica.cc:333] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.109377 12409 raft_consensus.cc:2243] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.110149 12409 raft_consensus.cc:2272] T 5c25c2050d4547c39924ceb739727ae8 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.113731 12409 tablet_replica.cc:333] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.114627 12409 raft_consensus.cc:2243] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.116061 12409 raft_consensus.cc:2272] T 0863151795784affb539e2b7a55fc2b8 P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.119097 12409 tablet_replica.cc:333] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.119967 12409 raft_consensus.cc:2243] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:19.120848 12409 raft_consensus.cc:2272] T f97a6462cd4341ddbd7058ce9537c1eb P 0f34cda059124f839711dbef61bb2c28 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.124043 12409 tablet_replica.cc:333] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.124827 12409 raft_consensus.cc:2243] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.125952 12409 raft_consensus.cc:2272] T 1b47f4cbc5b245adbdd9ebed9c3175f1 P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.128712 12409 tablet_replica.cc:333] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.129443 12409 raft_consensus.cc:2243] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.130620 12409 raft_consensus.cc:2272] T 5db10903ab7d403ba303d72b7f26dfed P 0f34cda059124f839711dbef61bb2c28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.133455 12409 tablet_replica.cc:333] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.134171 12409 raft_consensus.cc:2243] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.135222 12409 raft_consensus.cc:2272] T 22f14b8d35f741a28ad886b19614c4c8 P 0f34cda059124f839711dbef61bb2c28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:19.138017 12409 tablet_replica.cc:333] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28: stopping tablet replica
I20260709 12:59:19.138664 12409 raft_consensus.cc:2243] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:59:19.139729 12409 raft_consensus.cc:2272] T 973866cfc22e48ea931cb1a6fb16872f P 0f34cda059124f839711dbef61bb2c28 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:19.195802 13472 debug-util.cc:398] Leaking SignalData structure 0x7b080033a0c0 after lost signal to thread 12412
W20260709 12:59:19.197139 13472 debug-util.cc:398] Leaking SignalData structure 0x7b0800369200 after lost signal to thread 13475
W20260709 12:59:19.198881 13472 debug-util.cc:398] Leaking SignalData structure 0x7b08003115a0 after lost signal to thread 14270
W20260709 12:59:20.239114 12409 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 14267)
I20260709 12:59:20.681044 12409 tablet_server.cc:196] TabletServer@127.12.30.68:0 shutdown complete.
I20260709 12:59:20.835351 12409 master.cc:562] Master@127.12.30.126:40047 shutting down...
W20260709 12:59:20.880762 13489 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:59:20.881184 13489 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:59:20.881435 13489 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:59:25.728261 13427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111) [suppressed 189 similar messages]
W20260709 12:59:25.732260 13427 catalog_manager.cc:4755] TS ad2fdb1517c542439654bd3f7c72dad0 (127.12.30.65:39209): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 6dd04e83486049ae859032792f652869: Network error: Client connection negotiation failed: client connection to 127.12.30.65:39209: connect: Connection refused (error 111)
I20260709 12:59:26.486929 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:26.488113 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:26.488564 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7c7c6d1a2ec946878b513e7eb3a47a90: stopping tablet replica
I20260709 12:59:26.710062 12409 master.cc:584] Master@127.12.30.126:40047 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (214422 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:59:26.846022 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:46523
I20260709 12:59:26.847399 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:26.854614 19815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:26.854771 19816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:26.856534 19818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:26.858879 12409 server_base.cc:1061] running on GCE node
I20260709 12:59:26.863081 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:26.863317 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:26.863538 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601966863516 us; error 0 us; skew 500 ppm
I20260709 12:59:26.864168 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:26.867202 12409 webserver.cc:533] Webserver started at http://127.12.30.126:41095/ using document root <none> and password file <none>
I20260709 12:59:26.867785 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:26.868043 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:26.868376 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:26.869721 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "d84e5f78c3264c738b7966fc230d4878"
format_stamp: "Formatted at 2026-07-09 12:59:26 on dist-test-slave-mlt7"
I20260709 12:59:26.874938 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:59:26.878939 19823 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:26.880028 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:59:26.880383 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "d84e5f78c3264c738b7966fc230d4878"
format_stamp: "Formatted at 2026-07-09 12:59:26 on dist-test-slave-mlt7"
I20260709 12:59:26.880759 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:26.902159 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:26.903481 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:26.949487 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:46523
I20260709 12:59:26.949723 19874 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:46523 every 8 connection(s)
I20260709 12:59:26.954362 19875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:26.970480 19875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Bootstrap starting.
I20260709 12:59:26.977053 19875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.982249 19875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: No bootstrap required, opened a new log
I20260709 12:59:26.985038 19875 raft_consensus.cc:359] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84e5f78c3264c738b7966fc230d4878" member_type: VOTER }
I20260709 12:59:26.985548 19875 raft_consensus.cc:385] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.985812 19875 raft_consensus.cc:740] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84e5f78c3264c738b7966fc230d4878, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.986462 19875 consensus_queue.cc:260] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84e5f78c3264c738b7966fc230d4878" member_type: VOTER }
I20260709 12:59:26.987004 19875 raft_consensus.cc:399] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:59:26.987218 19875 raft_consensus.cc:493] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:59:26.987563 19875 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.993656 19875 raft_consensus.cc:515] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84e5f78c3264c738b7966fc230d4878" member_type: VOTER }
I20260709 12:59:26.994309 19875 leader_election.cc:304] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d84e5f78c3264c738b7966fc230d4878; no voters: 
I20260709 12:59:26.995744 19875 leader_election.cc:290] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:59:26.996275 19878 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.997963 19878 raft_consensus.cc:697] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 1 LEADER]: Becoming Leader. State: Replica: d84e5f78c3264c738b7966fc230d4878, State: Running, Role: LEADER
I20260709 12:59:26.998778 19878 consensus_queue.cc:237] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84e5f78c3264c738b7966fc230d4878" member_type: VOTER }
I20260709 12:59:27.000063 19875 sys_catalog.cc:565] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:59:27.006529 19879 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d84e5f78c3264c738b7966fc230d4878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84e5f78c3264c738b7966fc230d4878" member_type: VOTER } }
I20260709 12:59:27.007339 19879 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:27.007423 19880 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84e5f78c3264c738b7966fc230d4878. Latest consensus state: current_term: 1 leader_uuid: "d84e5f78c3264c738b7966fc230d4878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84e5f78c3264c738b7966fc230d4878" member_type: VOTER } }
I20260709 12:59:27.008404 19880 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:27.010412 19886 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:59:27.019532 19886 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:59:27.020996 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:59:27.030948 19886 catalog_manager.cc:1383] Generated new cluster ID: e52159c795254200bd43d8366fae7443
I20260709 12:59:27.031272 19886 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:59:27.054728 19886 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:59:27.056412 19886 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:59:27.074154 19886 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Generated new TSK 0
I20260709 12:59:27.074929 19886 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:59:27.088398 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:27.096045 19896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:27.098359 19897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:27.100957 12409 server_base.cc:1061] running on GCE node
W20260709 12:59:27.100754 19899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:27.104202 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:27.104426 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:27.104628 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601967104613 us; error 0 us; skew 500 ppm
I20260709 12:59:27.105281 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:27.108134 12409 webserver.cc:533] Webserver started at http://127.12.30.65:38037/ using document root <none> and password file <none>
I20260709 12:59:27.108628 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:27.108822 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:27.109131 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:27.110419 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "e5eb27db496b47129df3e689e546d94a"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.115777 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:59:27.119721 19904 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.120700 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:59:27.121089 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "e5eb27db496b47129df3e689e546d94a"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.121354 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:27.139438 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:27.140993 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:27.142668 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:27.145365 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:27.145605 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.145875 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:27.146075 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.192667 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:38607
I20260709 12:59:27.192795 19966 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:38607 every 8 connection(s)
I20260709 12:59:27.198021 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:27.205775 19971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:27.206256 19972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:27.209818 19974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:27.210639 12409 server_base.cc:1061] running on GCE node
I20260709 12:59:27.213208 19967 heartbeater.cc:344] Connected to a master server at 127.12.30.126:46523
I20260709 12:59:27.213611 19967 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:27.213963 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:27.214228 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:27.214411 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601967214397 us; error 0 us; skew 500 ppm
I20260709 12:59:27.214475 19967 heartbeater.cc:507] Master 127.12.30.126:46523 requested a full tablet report, sending...
I20260709 12:59:27.215070 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:27.216959 19840 ts_manager.cc:194] Registered new tserver with Master: e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:27.217989 12409 webserver.cc:533] Webserver started at http://127.12.30.66:36195/ using document root <none> and password file <none>
I20260709 12:59:27.218696 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:27.218951 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:27.219259 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:27.219695 19840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60996
I20260709 12:59:27.221006 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "48da76e971b84219a6ba80d1ba83b7eb"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.225814 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 12:59:27.229321 19979 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.230137 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:59:27.230435 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "48da76e971b84219a6ba80d1ba83b7eb"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.230758 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:27.242578 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:27.243769 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:27.245270 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:27.247622 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:27.247843 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.248147 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:27.248315 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.292768 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:39003
I20260709 12:59:27.292912 20041 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:39003 every 8 connection(s)
I20260709 12:59:27.297801 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:27.305471 20045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:27.307030 20046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:27.310256 20042 heartbeater.cc:344] Connected to a master server at 127.12.30.126:46523
W20260709 12:59:27.310405 20048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:27.311113 20042 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:27.311657 12409 server_base.cc:1061] running on GCE node
I20260709 12:59:27.312255 20042 heartbeater.cc:507] Master 127.12.30.126:46523 requested a full tablet report, sending...
I20260709 12:59:27.312723 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:27.312995 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:27.313199 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601967313180 us; error 0 us; skew 500 ppm
I20260709 12:59:27.313916 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:27.314438 19840 ts_manager.cc:194] Registered new tserver with Master: 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:27.315807 19840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32772
I20260709 12:59:27.316823 12409 webserver.cc:533] Webserver started at http://127.12.30.67:44843/ using document root <none> and password file <none>
I20260709 12:59:27.317421 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:27.317634 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:27.317907 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:27.319126 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "41af8b41f8a348c5b60ec12d2e323698"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.323776 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:59:27.327205 20053 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.328050 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:59:27.328370 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "41af8b41f8a348c5b60ec12d2e323698"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.328682 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:27.344455 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:27.345731 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:27.347247 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:27.349551 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:27.349790 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.350047 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:27.350222 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.394907 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:39831
I20260709 12:59:27.395058 20115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:39831 every 8 connection(s)
I20260709 12:59:27.400043 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:27.408293 20119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:27.409760 20120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:27.413465 12409 server_base.cc:1061] running on GCE node
W20260709 12:59:27.414096 20122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:27.415292 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:59:27.415319 20116 heartbeater.cc:344] Connected to a master server at 127.12.30.126:46523
W20260709 12:59:27.415640 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:27.415897 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601967415881 us; error 0 us; skew 500 ppm
I20260709 12:59:27.416000 20116 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:27.416657 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:27.417047 20116 heartbeater.cc:507] Master 127.12.30.126:46523 requested a full tablet report, sending...
I20260709 12:59:27.419647 12409 webserver.cc:533] Webserver started at http://127.12.30.68:36063/ using document root <none> and password file <none>
I20260709 12:59:27.419643 19840 ts_manager.cc:194] Registered new tserver with Master: 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:27.420351 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:27.420614 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:27.420905 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:27.421869 19840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32774
I20260709 12:59:27.422080 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-3-root/instance:
uuid: "8b5b4e2a92b046d096f2661c54321d41"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.428458 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:27.432665 20127 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.433506 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:59:27.433854 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-3-root
uuid: "8b5b4e2a92b046d096f2661c54321d41"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:27.434122 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:27.466054 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:27.467347 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:27.468918 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:27.471189 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:27.471434 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.471683 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:27.471863 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:27.518287 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.68:46223
I20260709 12:59:27.518388 20189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.68:46223 every 8 connection(s)
I20260709 12:59:27.532889 20190 heartbeater.cc:344] Connected to a master server at 127.12.30.126:46523
I20260709 12:59:27.533344 20190 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:27.534221 20190 heartbeater.cc:507] Master 127.12.30.126:46523 requested a full tablet report, sending...
I20260709 12:59:27.536334 19840 ts_manager.cc:194] Registered new tserver with Master: 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:27.537310 12409 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015280211s
I20260709 12:59:27.537765 19840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32784
I20260709 12:59:27.574188 19840 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32792:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:59:27.825194 20155 tablet_service.cc:1511] Processing CreateTablet for tablet 4ccd50eb99764ffc91bd06202fd0a5e7 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:27.826892 20155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ccd50eb99764ffc91bd06202fd0a5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.827881 20006 tablet_service.cc:1511] Processing CreateTablet for tablet af7bc64cbbc04798be25ec85d5de3b0d (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:27.829456 20006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af7bc64cbbc04798be25ec85d5de3b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.829524 20007 tablet_service.cc:1511] Processing CreateTablet for tablet 4ccd50eb99764ffc91bd06202fd0a5e7 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:27.832978 20154 tablet_service.cc:1511] Processing CreateTablet for tablet 1c733c25de16421ba3746f8c6de93bd5 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:27.834619 20154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c733c25de16421ba3746f8c6de93bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.835845 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ccd50eb99764ffc91bd06202fd0a5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.842370 20005 tablet_service.cc:1511] Processing CreateTablet for tablet 313fdc28d1e44d6393c8388c74843a28 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:27.845059 20005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 313fdc28d1e44d6393c8388c74843a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.845646 19932 tablet_service.cc:1511] Processing CreateTablet for tablet 4ccd50eb99764ffc91bd06202fd0a5e7 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:27.846972 19931 tablet_service.cc:1511] Processing CreateTablet for tablet af7bc64cbbc04798be25ec85d5de3b0d (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:27.847884 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ccd50eb99764ffc91bd06202fd0a5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.849157 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 1c733c25de16421ba3746f8c6de93bd5 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:27.850193 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb7d330cd8b4195aba1f8c1348bfcc2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:27.851285 19931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af7bc64cbbc04798be25ec85d5de3b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.852178 20153 tablet_service.cc:1511] Processing CreateTablet for tablet 313fdc28d1e44d6393c8388c74843a28 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:27.853184 20004 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb7d330cd8b4195aba1f8c1348bfcc2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:27.853828 20153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 313fdc28d1e44d6393c8388c74843a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.854895 20004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb7d330cd8b4195aba1f8c1348bfcc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.858168 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb7d330cd8b4195aba1f8c1348bfcc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.858769 20152 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe4ca3a058c431f8e6ecbfc90468313 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:27.861850 20151 tablet_service.cc:1511] Processing CreateTablet for tablet 054afb193dbf43d48b1b0ba6ce809ee3 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:27.862800 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c733c25de16421ba3746f8c6de93bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.863366 20151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054afb193dbf43d48b1b0ba6ce809ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.865387 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe4ca3a058c431f8e6ecbfc90468313 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:27.867105 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe4ca3a058c431f8e6ecbfc90468313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.869076 19928 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe4ca3a058c431f8e6ecbfc90468313 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:27.869306 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 32a78ca0f82940f0b3698fceba7b2ce4 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:27.869505 20002 tablet_service.cc:1511] Processing CreateTablet for tablet 32a78ca0f82940f0b3698fceba7b2ce4 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:27.870785 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32a78ca0f82940f0b3698fceba7b2ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.871299 20002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32a78ca0f82940f0b3698fceba7b2ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.872486 19927 tablet_service.cc:1511] Processing CreateTablet for tablet 054afb193dbf43d48b1b0ba6ce809ee3 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:27.875589 20152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe4ca3a058c431f8e6ecbfc90468313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.875399 20149 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1a8f437ee9497c987ac7a6206d2338 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:27.876444 19925 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1a8f437ee9497c987ac7a6206d2338 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:27.877975 19926 tablet_service.cc:1511] Processing CreateTablet for tablet 111cd0966cde4d87a8a86d90eabf9c0a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:27.870811 19928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe4ca3a058c431f8e6ecbfc90468313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.881178 20148 tablet_service.cc:1511] Processing CreateTablet for tablet 6240627eb43d4d039f730bc326fdc93e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:27.882946 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6240627eb43d4d039f730bc326fdc93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.884537 19926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 111cd0966cde4d87a8a86d90eabf9c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.885330 19927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054afb193dbf43d48b1b0ba6ce809ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.888731 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1a8f437ee9497c987ac7a6206d2338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.894409 20001 tablet_service.cc:1511] Processing CreateTablet for tablet 111cd0966cde4d87a8a86d90eabf9c0a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:27.897511 20000 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1a8f437ee9497c987ac7a6206d2338 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:27.898649 20147 tablet_service.cc:1511] Processing CreateTablet for tablet 9dadb6b575d44e06ba9ab0804b1d51c6 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:27.900350 19999 tablet_service.cc:1511] Processing CreateTablet for tablet 9dadb6b575d44e06ba9ab0804b1d51c6 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:27.901273 20146 tablet_service.cc:1511] Processing CreateTablet for tablet bb22f25deefa4dce94a6ca7ffecaee7c (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:27.905476 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 6240627eb43d4d039f730bc326fdc93e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:27.906816 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1a8f437ee9497c987ac7a6206d2338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.908702 19923 tablet_service.cc:1511] Processing CreateTablet for tablet f4ea05922db6473289d3a25f7b32e86e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:27.910403 19923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ea05922db6473289d3a25f7b32e86e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.909209 20001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 111cd0966cde4d87a8a86d90eabf9c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.915396 20147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dadb6b575d44e06ba9ab0804b1d51c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.917838 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6240627eb43d4d039f730bc326fdc93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.915441 20000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1a8f437ee9497c987ac7a6206d2338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.920445 20146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb22f25deefa4dce94a6ca7ffecaee7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.920720 19999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dadb6b575d44e06ba9ab0804b1d51c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.926232 19998 tablet_service.cc:1511] Processing CreateTablet for tablet f4ea05922db6473289d3a25f7b32e86e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:27.928509 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ea05922db6473289d3a25f7b32e86e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.968739 20081 tablet_service.cc:1511] Processing CreateTablet for tablet af7bc64cbbc04798be25ec85d5de3b0d (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:27.970685 20081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af7bc64cbbc04798be25ec85d5de3b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.973989 20080 tablet_service.cc:1511] Processing CreateTablet for tablet 1c733c25de16421ba3746f8c6de93bd5 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:27.976050 20080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c733c25de16421ba3746f8c6de93bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:27.997973 20079 tablet_service.cc:1511] Processing CreateTablet for tablet 313fdc28d1e44d6393c8388c74843a28 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:28.000416 20079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 313fdc28d1e44d6393c8388c74843a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.010601 20210 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:28.010192 20078 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb7d330cd8b4195aba1f8c1348bfcc2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:28.029950 20078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb7d330cd8b4195aba1f8c1348bfcc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.030186 20072 tablet_service.cc:1511] Processing CreateTablet for tablet f4ea05922db6473289d3a25f7b32e86e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:28.031813 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ea05922db6473289d3a25f7b32e86e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:28.014523 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:28.016134 20003 tablet_service.cc:1511] Processing CreateTablet for tablet bb22f25deefa4dce94a6ca7ffecaee7c (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:28.016749 20077 tablet_service.cc:1511] Processing CreateTablet for tablet 054afb193dbf43d48b1b0ba6ce809ee3 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:28.037177 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb22f25deefa4dce94a6ca7ffecaee7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.037304 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054afb193dbf43d48b1b0ba6ce809ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.019351 20076 tablet_service.cc:1511] Processing CreateTablet for tablet 32a78ca0f82940f0b3698fceba7b2ce4 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:28.040918 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32a78ca0f82940f0b3698fceba7b2ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.041374 20209 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:28.021759 20075 tablet_service.cc:1511] Processing CreateTablet for tablet 111cd0966cde4d87a8a86d90eabf9c0a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:28.045087 20075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 111cd0966cde4d87a8a86d90eabf9c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.024611 20074 tablet_service.cc:1511] Processing CreateTablet for tablet 6240627eb43d4d039f730bc326fdc93e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:28.049966 20074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6240627eb43d4d039f730bc326fdc93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.027354 20073 tablet_service.cc:1511] Processing CreateTablet for tablet 9dadb6b575d44e06ba9ab0804b1d51c6 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:28.053246 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dadb6b575d44e06ba9ab0804b1d51c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.054394 20153 tablet_service.cc:1511] Processing CreateTablet for tablet 630bae767d5b40dfa02289e9b25f7c15 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:28.055732 20153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630bae767d5b40dfa02289e9b25f7c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.061328 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 45dd40d980c94ccdab3d167b33984c44 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:28.076876 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45dd40d980c94ccdab3d167b33984c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.081152 20006 tablet_service.cc:1511] Processing CreateTablet for tablet 630bae767d5b40dfa02289e9b25f7c15 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:28.082899 20006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630bae767d5b40dfa02289e9b25f7c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.071285 20210 tablet_bootstrap.cc:654] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.053143 20149 tablet_service.cc:1511] Processing CreateTablet for tablet 6c7a345503524963ae2cf82630978b5a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:28.096665 20209 tablet_bootstrap.cc:654] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.096835 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c7a345503524963ae2cf82630978b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.100539 19932 tablet_service.cc:1511] Processing CreateTablet for tablet 6c7a345503524963ae2cf82630978b5a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:28.101933 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c7a345503524963ae2cf82630978b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.105168 19924 tablet_service.cc:1511] Processing CreateTablet for tablet bb22f25deefa4dce94a6ca7ffecaee7c (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:28.106736 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb22f25deefa4dce94a6ca7ffecaee7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.109138 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 45dd40d980c94ccdab3d167b33984c44 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:28.110776 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45dd40d980c94ccdab3d167b33984c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.111352 19925 tablet_service.cc:1511] Processing CreateTablet for tablet f761e6c8cc73410eb153cfbcb79afb90 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:28.112939 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f761e6c8cc73410eb153cfbcb79afb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.122018 20211 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:28.131611 20007 tablet_service.cc:1511] Processing CreateTablet for tablet f761e6c8cc73410eb153cfbcb79afb90 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:28.146490 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f761e6c8cc73410eb153cfbcb79afb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.157856 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 45dd40d980c94ccdab3d167b33984c44 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:28.159528 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45dd40d980c94ccdab3d167b33984c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.176892 20211 tablet_bootstrap.cc:654] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.193555 20149 tablet_service.cc:1511] Processing CreateTablet for tablet 59c71a9c3bb24e9d92023242a19a363d (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:28.195668 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59c71a9c3bb24e9d92023242a19a363d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.216099 20073 tablet_service.cc:1511] Processing CreateTablet for tablet 6c7a345503524963ae2cf82630978b5a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:28.217532 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c7a345503524963ae2cf82630978b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.222728 19967 heartbeater.cc:499] Master 127.12.30.126:46523 was elected leader, sending a full tablet report...
I20260709 12:59:28.234625 20073 tablet_service.cc:1511] Processing CreateTablet for tablet 630bae767d5b40dfa02289e9b25f7c15 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:28.236232 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630bae767d5b40dfa02289e9b25f7c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.243309 19925 tablet_service.cc:1511] Processing CreateTablet for tablet 59c71a9c3bb24e9d92023242a19a363d (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:28.245404 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59c71a9c3bb24e9d92023242a19a363d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.250272 20007 tablet_service.cc:1511] Processing CreateTablet for tablet 7296fb4f71a34242af6ae5071d9f508e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:28.251713 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7296fb4f71a34242af6ae5071d9f508e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.262850 20076 tablet_service.cc:1511] Processing CreateTablet for tablet f761e6c8cc73410eb153cfbcb79afb90 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:28.263456 20073 tablet_service.cc:1511] Processing CreateTablet for tablet 59c71a9c3bb24e9d92023242a19a363d (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:28.270771 20210 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:28.271354 20210 ts_tablet_manager.cc:1403] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.262s	user 0.010s	sys 0.011s
I20260709 12:59:28.271472 20209 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:28.272307 20209 ts_tablet_manager.cc:1403] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.231s	user 0.036s	sys 0.084s
I20260709 12:59:28.277525 20209 raft_consensus.cc:359] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.278812 20209 raft_consensus.cc:385] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.279309 20209 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.279165 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 746d924541574d6086a6ae086476a1e7 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:28.280719 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746d924541574d6086a6ae086476a1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.280718 20209 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.285347 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f761e6c8cc73410eb153cfbcb79afb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.289698 20152 tablet_service.cc:1511] Processing CreateTablet for tablet 7296fb4f71a34242af6ae5071d9f508e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:28.290128 19998 tablet_service.cc:1511] Processing CreateTablet for tablet 746d924541574d6086a6ae086476a1e7 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:28.291126 20152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7296fb4f71a34242af6ae5071d9f508e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.291319 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746d924541574d6086a6ae086476a1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.275408 20210 raft_consensus.cc:359] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.296869 20210 raft_consensus.cc:385] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.306525 20214 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:28.307559 20072 tablet_service.cc:1511] Processing CreateTablet for tablet 7296fb4f71a34242af6ae5071d9f508e (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:28.308561 20077 tablet_service.cc:1511] Processing CreateTablet for tablet 746d924541574d6086a6ae086476a1e7 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:28.308815 20148 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd1b4e7346b47ffaa37a1f24e287d3a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:28.310881 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd1b4e7346b47ffaa37a1f24e287d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.312870 19932 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd1b4e7346b47ffaa37a1f24e287d3a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:28.314330 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd1b4e7346b47ffaa37a1f24e287d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.318534 20042 heartbeater.cc:499] Master 127.12.30.126:46523 was elected leader, sending a full tablet report...
I20260709 12:59:28.321290 20210 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.328554 20210 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.325994 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59c71a9c3bb24e9d92023242a19a363d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.358383 20211 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:28.358739 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7296fb4f71a34242af6ae5071d9f508e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.358973 20211 ts_tablet_manager.cc:1403] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.237s	user 0.018s	sys 0.014s
I20260709 12:59:28.362035 20211 raft_consensus.cc:359] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:28.362772 20211 raft_consensus.cc:385] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.363075 20211 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.368368 20209 ts_tablet_manager.cc:1434] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.096s	user 0.017s	sys 0.025s
I20260709 12:59:28.369326 20190 heartbeater.cc:499] Master 127.12.30.126:46523 was elected leader, sending a full tablet report...
I20260709 12:59:28.368710 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746d924541574d6086a6ae086476a1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.372352 20211 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:28.369652 20209 tablet_bootstrap.cc:492] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:28.392611 20210 ts_tablet_manager.cc:1434] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.121s	user 0.010s	sys 0.015s
I20260709 12:59:28.393567 20210 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:28.395296 19998 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd1b4e7346b47ffaa37a1f24e287d3a (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:28.396672 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd1b4e7346b47ffaa37a1f24e287d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.399115 20076 tablet_service.cc:1511] Processing CreateTablet for tablet 0af76b6f84f54d249f7949cf3c104e32 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:28.400408 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af76b6f84f54d249f7949cf3c104e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.406558 19999 tablet_service.cc:1511] Processing CreateTablet for tablet 153ccb361669440fa940c70fa6ae8d76 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:28.408591 19999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153ccb361669440fa940c70fa6ae8d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.417779 20209 tablet_bootstrap.cc:654] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.420475 20152 tablet_service.cc:1511] Processing CreateTablet for tablet 0af76b6f84f54d249f7949cf3c104e32 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:28.422195 20152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af76b6f84f54d249f7949cf3c104e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.423025 20214 tablet_bootstrap.cc:654] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.424839 20116 heartbeater.cc:499] Master 127.12.30.126:46523 was elected leader, sending a full tablet report...
I20260709 12:59:28.425031 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 0af76b6f84f54d249f7949cf3c104e32 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:28.426402 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af76b6f84f54d249f7949cf3c104e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.438606 20210 tablet_bootstrap.cc:654] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.442049 20081 tablet_service.cc:1511] Processing CreateTablet for tablet 153ccb361669440fa940c70fa6ae8d76 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:28.443677 20081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153ccb361669440fa940c70fa6ae8d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.451088 20214 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:28.451910 20214 ts_tablet_manager.cc:1403] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.156s	user 0.006s	sys 0.011s
I20260709 12:59:28.460565 20214 raft_consensus.cc:359] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:28.464814 20214 raft_consensus.cc:385] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.470793 20148 tablet_service.cc:1511] Processing CreateTablet for tablet 153ccb361669440fa940c70fa6ae8d76 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:28.472064 20214 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.472810 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153ccb361669440fa940c70fa6ae8d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.473173 20211 ts_tablet_manager.cc:1434] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.114s	user 0.037s	sys 0.038s
I20260709 12:59:28.472980 20214 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:28.474229 20211 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:28.481060 20211 tablet_bootstrap.cc:654] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.493857 19925 tablet_service.cc:1511] Processing CreateTablet for tablet d8a881d176a0426b9337a6b7c224b88c (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:28.495472 20210 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:28.499990 20209 tablet_bootstrap.cc:492] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:28.500659 20209 ts_tablet_manager.cc:1403] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.131s	user 0.015s	sys 0.004s
I20260709 12:59:28.495592 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a881d176a0426b9337a6b7c224b88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.505796 20209 raft_consensus.cc:359] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:28.506518 20209 raft_consensus.cc:385] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.506840 20209 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.508769 20209 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:28.512610 20209 ts_tablet_manager.cc:1434] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 12:59:28.513548 20209 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:28.515300 20214 ts_tablet_manager.cc:1434] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.063s	user 0.013s	sys 0.002s
I20260709 12:59:28.516747 20214 tablet_bootstrap.cc:492] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:28.520066 20210 ts_tablet_manager.cc:1403] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.127s	user 0.011s	sys 0.006s
I20260709 12:59:28.521159 20209 tablet_bootstrap.cc:654] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.523355 20218 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:28.524101 20191 tablet.cc:2406] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:59:28.523900 20218 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:28.527820 20218 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:28.558398 19999 tablet_service.cc:1511] Processing CreateTablet for tablet d8a881d176a0426b9337a6b7c224b88c (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:28.559974 20152 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8749de35944e3d92b86f7d4c472de2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:28.560602 19999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a881d176a0426b9337a6b7c224b88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.563310 19998 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8749de35944e3d92b86f7d4c472de2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:28.564767 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8749de35944e3d92b86f7d4c472de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.564841 20076 tablet_service.cc:1511] Processing CreateTablet for tablet d8a881d176a0426b9337a6b7c224b88c (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:28.566336 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a881d176a0426b9337a6b7c224b88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.571846 20152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8749de35944e3d92b86f7d4c472de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.572654 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8749de35944e3d92b86f7d4c472de2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:28.574209 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8749de35944e3d92b86f7d4c472de2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.553592 20210 raft_consensus.cc:359] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.580366 20210 raft_consensus.cc:385] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.580825 20210 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.582018 20210 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.585052 20210 ts_tablet_manager.cc:1434] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.064s	user 0.004s	sys 0.006s
I20260709 12:59:28.586541 20210 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:28.602411 20211 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:28.603210 20211 ts_tablet_manager.cc:1403] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.129s	user 0.025s	sys 0.029s
I20260709 12:59:28.604801 20072 tablet_service.cc:1511] Processing CreateTablet for tablet bde87ca1d98f4bd8b90d7f1502cc67f1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:28.606405 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bde87ca1d98f4bd8b90d7f1502cc67f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.607025 20211 raft_consensus.cc:359] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:28.612429 20148 tablet_service.cc:1511] Processing CreateTablet for tablet bde87ca1d98f4bd8b90d7f1502cc67f1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:28.614612 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bde87ca1d98f4bd8b90d7f1502cc67f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.607901 20211 raft_consensus.cc:385] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.617252 20211 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.617719 19932 tablet_service.cc:1511] Processing CreateTablet for tablet bde87ca1d98f4bd8b90d7f1502cc67f1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:28.618644 20216 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.619279 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bde87ca1d98f4bd8b90d7f1502cc67f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.619319 20216 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.621893 20211 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:28.625623 20211 ts_tablet_manager.cc:1434] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:59:28.637863 19998 tablet_service.cc:1511] Processing CreateTablet for tablet eb6e04fb0be94cccad001ccc48f9bd85 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:28.639572 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6e04fb0be94cccad001ccc48f9bd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.639185 20077 tablet_service.cc:1511] Processing CreateTablet for tablet eb6e04fb0be94cccad001ccc48f9bd85 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:28.640846 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6e04fb0be94cccad001ccc48f9bd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.644609 20149 tablet_service.cc:1511] Processing CreateTablet for tablet eb6e04fb0be94cccad001ccc48f9bd85 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:28.646032 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6e04fb0be94cccad001ccc48f9bd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.646533 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 62a5f678444241d5a158a35557f9a25b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:28.647964 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62a5f678444241d5a158a35557f9a25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.666976 19998 tablet_service.cc:1511] Processing CreateTablet for tablet 62a5f678444241d5a158a35557f9a25b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:28.668938 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62a5f678444241d5a158a35557f9a25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.669040 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 2d767704576e43bb9f6bcd864fa9a6fe (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:28.670368 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d767704576e43bb9f6bcd864fa9a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.678961 20214 tablet_bootstrap.cc:654] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.683723 20217 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.684353 20217 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.631100 20216 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:28.697293 20211 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:28.700716 20209 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:28.701284 20209 ts_tablet_manager.cc:1403] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.188s	user 0.018s	sys 0.035s
I20260709 12:59:28.704478 20211 tablet_bootstrap.cc:654] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.722048 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 2d767704576e43bb9f6bcd864fa9a6fe (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:28.723604 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d767704576e43bb9f6bcd864fa9a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.729246 20153 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2cace87d4c4d26be2f8bffdb5b6d50 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:28.730020 20081 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2cace87d4c4d26be2f8bffdb5b6d50 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:28.730682 20153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2cace87d4c4d26be2f8bffdb5b6d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.732528 20081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2cace87d4c4d26be2f8bffdb5b6d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.739575 20222 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.740199 20217 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:28.740492 20222 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:28.743067 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:28.747014 19925 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2cace87d4c4d26be2f8bffdb5b6d50 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20260709 12:59:28.747953 19906 leader_election.cc:343] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.748999 20072 tablet_service.cc:1511] Processing CreateTablet for tablet 62a5f678444241d5a158a35557f9a25b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:28.759645 20007 tablet_service.cc:1511] Processing CreateTablet for tablet 4864bdb85f20457e9ef1bfb12f991554 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:28.749100 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2cace87d4c4d26be2f8bffdb5b6d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.704756 20209 raft_consensus.cc:359] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.770262 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4864bdb85f20457e9ef1bfb12f991554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.770892 20210 tablet_bootstrap.cc:654] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.771368 20222 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:28.784415 20216 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.785213 20216 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:28.786792 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:28.773022 20209 raft_consensus.cc:385] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:28.794163 19907 leader_election.cc:343] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.794845 19907 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:28.796056 20218 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.800235 20209 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.821130 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62a5f678444241d5a158a35557f9a25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.840588 19929 tablet_service.cc:1511] Processing CreateTablet for tablet d94fa4e16c4f424c86e23d82c9408473 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:28.836576 20146 tablet_service.cc:1511] Processing CreateTablet for tablet 4864bdb85f20457e9ef1bfb12f991554 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:28.843218 20146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4864bdb85f20457e9ef1bfb12f991554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.842327 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94fa4e16c4f424c86e23d82c9408473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.844638 20216 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:28.866814 20073 tablet_service.cc:1511] Processing CreateTablet for tablet d94fa4e16c4f424c86e23d82c9408473 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:28.868335 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94fa4e16c4f424c86e23d82c9408473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.875589 20153 tablet_service.cc:1511] Processing CreateTablet for tablet eb11384495c2485aaab243a655f75c86 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:28.876031 20076 tablet_service.cc:1511] Processing CreateTablet for tablet 4864bdb85f20457e9ef1bfb12f991554 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:28.879773 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4864bdb85f20457e9ef1bfb12f991554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.879840 19927 tablet_service.cc:1511] Processing CreateTablet for tablet eb11384495c2485aaab243a655f75c86 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:28.880494 20217 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.880749 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:28.881325 19927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb11384495c2485aaab243a655f75c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.881017 20217 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.882025 20017 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:28.883574 20217 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:28.884395 20153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb11384495c2485aaab243a655f75c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.884497 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:28.884733 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:28.885632 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:28.881704 20209 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:28.887391 19981 leader_election.cc:343] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.888091 20209 ts_tablet_manager.cc:1434] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.186s	user 0.007s	sys 0.007s
I20260709 12:59:28.908591 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:28.918582 20214 tablet_bootstrap.cc:492] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:28.919381 20214 ts_tablet_manager.cc:1403] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.403s	user 0.029s	sys 0.019s
I20260709 12:59:28.924680 20214 raft_consensus.cc:359] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:28.925514 20214 raft_consensus.cc:385] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.925837 20214 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.926649 20214 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:28.927853 20211 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:28.928560 20211 ts_tablet_manager.cc:1403] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.245s	user 0.014s	sys 0.007s
I20260709 12:59:28.932116 20216 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.932578 20216 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.932924 20216 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.933104 20210 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:28.933784 20210 ts_tablet_manager.cc:1403] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.348s	user 0.018s	sys 0.014s
W20260709 12:59:28.943415 20129 leader_election.cc:343] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.948632 20209 tablet_bootstrap.cc:492] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:28.961184 20210 raft_consensus.cc:359] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.963590 20210 raft_consensus.cc:385] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.964061 20210 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.903126 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 2d767704576e43bb9f6bcd864fa9a6fe (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:28.966468 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:28.968142 20210 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.968752 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d767704576e43bb9f6bcd864fa9a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:28.970417 20055 leader_election.cc:343] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.973059 20214 ts_tablet_manager.cc:1434] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.053s	user 0.010s	sys 0.001s
I20260709 12:59:28.974138 20214 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:28.978652 20148 tablet_service.cc:1511] Processing CreateTablet for tablet 77ca1b6b08b74f04948397dece34f882 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:28.980046 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ca1b6b08b74f04948397dece34f882. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:28.959081 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74581.7us
W20260709 12:59:28.981278 20130 leader_election.cc:343] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.983326 20003 tablet_service.cc:1511] Processing CreateTablet for tablet d94fa4e16c4f424c86e23d82c9408473 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:28.984709 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94fa4e16c4f424c86e23d82c9408473. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.980073 20211 raft_consensus.cc:359] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:28.990690 20211 raft_consensus.cc:385] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.990993 20211 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.992512 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 77ca1b6b08b74f04948397dece34f882 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:28.992859 19999 tablet_service.cc:1511] Processing CreateTablet for tablet eb11384495c2485aaab243a655f75c86 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:28.993892 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ca1b6b08b74f04948397dece34f882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:28.994208 19999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb11384495c2485aaab243a655f75c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.004467 20222 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.005044 20222 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:29.006728 20081 tablet_service.cc:1511] Processing CreateTablet for tablet 77ca1b6b08b74f04948397dece34f882 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:29.007406 20222 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:29.008298 20081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ca1b6b08b74f04948397dece34f882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.008773 20211 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.015573 20077 tablet_service.cc:1511] Processing CreateTablet for tablet a42b8897c4224305af65104f541be1fc (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:29.018584 20210 ts_tablet_manager.cc:1434] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.001s
I20260709 12:59:29.018982 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a42b8897c4224305af65104f541be1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.024495 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:29.025281 19942 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:29.026000 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:29.023473 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:29.027931 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 12:59:29.029222 19981 leader_election.cc:343] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.029922 20217 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:29.030439 20217 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:29.030856 20217 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.036618 20214 tablet_bootstrap.cc:654] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.037982 20211 ts_tablet_manager.cc:1434] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.000s
W20260709 12:59:29.038916 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:29.044806 20217 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.047397 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:29.047766 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:29.049932 19998 tablet_service.cc:1511] Processing CreateTablet for tablet a42b8897c4224305af65104f541be1fc (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:29.051365 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a42b8897c4224305af65104f541be1fc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:29.052407 20129 leader_election.cc:343] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.037719 20216 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.052933 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:29.055529 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:29.055802 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:29.056499 20017 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:29.057096 20129 leader_election.cc:343] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.057965 20244 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.065052 20209 tablet_bootstrap.cc:654] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.019810 20210 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:29.071877 20150 tablet_service.cc:1511] Processing CreateTablet for tablet a42b8897c4224305af65104f541be1fc (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:29.072716 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:29.073467 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a42b8897c4224305af65104f541be1fc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:29.074254 19981 leader_election.cc:343] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.080521 20017 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:29.081733 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:29.082638 20210 tablet_bootstrap.cc:654] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.082823 20244 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:29.083407 20211 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:29.069659 20216 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 12:59:29.081240 20056 leader_election.cc:343] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.102788 20218 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.103467 20218 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.106166 20218 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:29.109203 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:29.110383 20222 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.110549 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:29.112860 19907 leader_election.cc:343] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.113492 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 7b9595f271b64d9caeb8e95eb6890e38 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:29.115231 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b9595f271b64d9caeb8e95eb6890e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.125466 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:29.126358 19941 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.132628 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 12:59:29.134704 19907 leader_election.cc:343] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.135289 19907 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:29.136031 20217 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:29.136710 20218 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.140704 20007 tablet_service.cc:1511] Processing CreateTablet for tablet 7b9595f271b64d9caeb8e95eb6890e38 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 12:59:29.141443 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60475us
I20260709 12:59:29.142663 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b9595f271b64d9caeb8e95eb6890e38. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:29.143399 20056 leader_election.cc:343] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.109229 20211 tablet_bootstrap.cc:654] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.159277 20209 tablet_bootstrap.cc:492] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:29.159870 20209 ts_tablet_manager.cc:1403] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.221s	user 0.018s	sys 0.007s
I20260709 12:59:29.163023 20214 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:29.164049 19941 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:29.168377 20244 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:29.171016 20214 ts_tablet_manager.cc:1403] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.197s	user 0.026s	sys 0.022s
I20260709 12:59:29.171321 20244 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.172012 20072 tablet_service.cc:1511] Processing CreateTablet for tablet 7b9595f271b64d9caeb8e95eb6890e38 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:29.172443 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:29.173395 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b9595f271b64d9caeb8e95eb6890e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.173704 20217 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:29.177863 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.180073 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.183229 20249 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:29.184475 20056 leader_election.cc:343] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.184617 20249 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.185236 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:29.185686 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:29.186259 20222 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.190069 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2ab233d0664594955baf948686386b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:29.195145 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2ab233d0664594955baf948686386b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.204497 20214 raft_consensus.cc:359] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.225925 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2ab233d0664594955baf948686386b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:29.226621 20210 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:29.227154 20210 ts_tablet_manager.cc:1403] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.208s	user 0.011s	sys 0.029s
I20260709 12:59:29.227392 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2ab233d0664594955baf948686386b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.228717 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:29.230180 20249 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:29.223490 20214 raft_consensus.cc:385] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:29.231763 20129 leader_election.cc:343] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.223747 20217 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:29.239724 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad5e6c5a68d4b07b3d5830de6d5be23 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:29.241201 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad5e6c5a68d4b07b3d5830de6d5be23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.241020 20153 tablet_service.cc:1511] Processing CreateTablet for tablet d6f643e855f54a4d8def1836bbdcf39b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:29.242327 20153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f643e855f54a4d8def1836bbdcf39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.245888 20209 raft_consensus.cc:359] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:29.246789 20209 raft_consensus.cc:385] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.246541 19838 catalog_manager.cc:5697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:29.232056 20210 raft_consensus.cc:359] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.232410 20076 tablet_service.cc:1511] Processing CreateTablet for tablet d6f643e855f54a4d8def1836bbdcf39b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:29.248827 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f643e855f54a4d8def1836bbdcf39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.237135 20214 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.238293 19930 tablet_service.cc:1511] Processing CreateTablet for tablet d6f643e855f54a4d8def1836bbdcf39b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:29.239657 20217 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.231969 19999 tablet_service.cc:1511] Processing CreateTablet for tablet 8d2ab233d0664594955baf948686386b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:29.254554 20252 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.256079 20252 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.256577 20252 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.247200 20209 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.254197 20214 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.252470 20210 raft_consensus.cc:385] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.265357 19999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d2ab233d0664594955baf948686386b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.269622 20214 ts_tablet_manager.cc:1434] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.098s	user 0.000s	sys 0.009s
I20260709 12:59:29.275259 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f643e855f54a4d8def1836bbdcf39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.281684 20148 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad5e6c5a68d4b07b3d5830de6d5be23 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:29.283478 20214 tablet_bootstrap.cc:492] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:29.283962 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad5e6c5a68d4b07b3d5830de6d5be23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.296070 20252 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 12:59:29.261394 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 75941.8us
I20260709 12:59:29.298347 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.298645 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:29.299608 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:29.300191 19981 leader_election.cc:343] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:29.300889 20130 leader_election.cc:343] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.301403 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:29.303253 20249 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:29.305583 19982 leader_election.cc:343] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.261821 20210 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.306305 19925 tablet_service.cc:1511] Processing CreateTablet for tablet e4bbc03ab1f04c8ebbfad0c35770dad1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:29.311643 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4bbc03ab1f04c8ebbfad0c35770dad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.313153 20211 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:29.312816 20007 tablet_service.cc:1511] Processing CreateTablet for tablet e4bbc03ab1f04c8ebbfad0c35770dad1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:29.313783 20211 ts_tablet_manager.cc:1403] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.231s	user 0.010s	sys 0.020s
I20260709 12:59:29.314648 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4bbc03ab1f04c8ebbfad0c35770dad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.317291 20211 raft_consensus.cc:359] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.318375 20211 raft_consensus.cc:385] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.318696 20211 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.319736 20211 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.328083 20209 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:29.306310 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:29.332731 20217 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.333758 20077 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad5e6c5a68d4b07b3d5830de6d5be23 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:29.353505 20210 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.356241 20211 ts_tablet_manager.cc:1434] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.042s	user 0.013s	sys 0.003s
I20260709 12:59:29.355885 20072 tablet_service.cc:1511] Processing CreateTablet for tablet e4bbc03ab1f04c8ebbfad0c35770dad1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:29.355975 20210 ts_tablet_manager.cc:1434] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.128s	user 0.009s	sys 0.001s
I20260709 12:59:29.347908 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad5e6c5a68d4b07b3d5830de6d5be23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.363545 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4bbc03ab1f04c8ebbfad0c35770dad1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.364732 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 5acf254dbaee44d1bc844da86793ef92 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:29.368436 20211 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:29.370388 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 5acf254dbaee44d1bc844da86793ef92 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:29.370846 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5acf254dbaee44d1bc844da86793ef92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.371765 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5acf254dbaee44d1bc844da86793ef92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.371827 20210 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:29.371793 20214 tablet_bootstrap.cc:654] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.371387 20081 tablet_service.cc:1511] Processing CreateTablet for tablet ebe7c81078d24dfd8ebcfce8dd265486 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:29.380488 20081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebe7c81078d24dfd8ebcfce8dd265486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.381330 20149 tablet_service.cc:1511] Processing CreateTablet for tablet ebe7c81078d24dfd8ebcfce8dd265486 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:29.383225 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebe7c81078d24dfd8ebcfce8dd265486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.394153 19998 tablet_service.cc:1511] Processing CreateTablet for tablet 5665e7206f7d4ad58380653cdf2490c2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:29.404841 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5665e7206f7d4ad58380653cdf2490c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.388777 19929 tablet_service.cc:1511] Processing CreateTablet for tablet ebe7c81078d24dfd8ebcfce8dd265486 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:29.391459 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 5acf254dbaee44d1bc844da86793ef92 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:29.413764 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebe7c81078d24dfd8ebcfce8dd265486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.413625 20209 ts_tablet_manager.cc:1434] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.243s	user 0.014s	sys 0.005s
I20260709 12:59:29.422649 20211 tablet_bootstrap.cc:654] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.429121 20217 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.434669 20209 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:29.434852 20217 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.437371 20217 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:29.427810 20210 tablet_bootstrap.cc:654] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.447765 20209 tablet_bootstrap.cc:654] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.454916 20214 tablet_bootstrap.cc:492] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:29.455565 20214 ts_tablet_manager.cc:1403] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.172s	user 0.043s	sys 0.012s
I20260709 12:59:29.462248 20211 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:29.463043 20211 ts_tablet_manager.cc:1403] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.095s	user 0.013s	sys 0.003s
I20260709 12:59:29.466287 20211 raft_consensus.cc:359] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.466820 20211 raft_consensus.cc:385] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.467006 20211 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.467548 20211 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.504549 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5acf254dbaee44d1bc844da86793ef92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.516906 20072 tablet_service.cc:1511] Processing CreateTablet for tablet 5665e7206f7d4ad58380653cdf2490c2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:29.517609 20214 raft_consensus.cc:359] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:29.519685 20214 raft_consensus.cc:385] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.520222 20214 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.523334 20214 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:29.524801 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.527559 20211 ts_tablet_manager.cc:1434] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.064s	user 0.003s	sys 0.003s
I20260709 12:59:29.527894 20214 ts_tablet_manager.cc:1434] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.069s	user 0.012s	sys 0.001s
I20260709 12:59:29.529299 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:29.529683 20211 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:29.530421 19941 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
W20260709 12:59:29.538343 19982 leader_election.cc:343] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.539773 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:29.543157 20217 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:29.543679 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5665e7206f7d4ad58380653cdf2490c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.543803 20217 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:29.544502 20217 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.554553 20222 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.555481 20222 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.556531 20209 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:29.559877 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.563130 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 5665e7206f7d4ad58380653cdf2490c2 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:29.567346 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5665e7206f7d4ad58380653cdf2490c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.567565 20007 tablet_service.cc:1511] Processing CreateTablet for tablet 25b25157ae9c4e9d87d2ff89c2c0c4dd (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:29.569425 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b25157ae9c4e9d87d2ff89c2c0c4dd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:29.569552 20056 leader_election.cc:343] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.571054 20222 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:29.572172 20209 ts_tablet_manager.cc:1403] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.013s
I20260709 12:59:29.571903 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 12:59:29.573968 20055 leader_election.cc:343] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.574540 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:29.574308 20076 tablet_service.cc:1511] Processing CreateTablet for tablet 25b25157ae9c4e9d87d2ff89c2c0c4dd (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:29.575946 20222 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.586596 20210 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:29.587901 20210 ts_tablet_manager.cc:1403] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.226s	user 0.011s	sys 0.004s
I20260709 12:59:29.589417 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b25157ae9c4e9d87d2ff89c2c0c4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.507191 19837 catalog_manager.cc:5697] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:29.588241 19925 tablet_service.cc:1511] Processing CreateTablet for tablet 25b25157ae9c4e9d87d2ff89c2c0c4dd (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:29.600340 20148 tablet_service.cc:1511] Processing CreateTablet for tablet 4c36c61154404ebebcf309833a8bf5ea (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:29.601960 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c36c61154404ebebcf309833a8bf5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.601858 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 4c36c61154404ebebcf309833a8bf5ea (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:29.601925 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25b25157ae9c4e9d87d2ff89c2c0c4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.607605 19999 tablet_service.cc:1511] Processing CreateTablet for tablet 4c36c61154404ebebcf309833a8bf5ea (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:29.609386 19999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c36c61154404ebebcf309833a8bf5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.615043 20214 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:29.620079 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c36c61154404ebebcf309833a8bf5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.623769 20072 tablet_service.cc:1511] Processing CreateTablet for tablet fac4d79f682d4f169331b9656bef55fe (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:29.625428 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac4d79f682d4f169331b9656bef55fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.593482 20210 raft_consensus.cc:359] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:29.630998 20211 tablet_bootstrap.cc:654] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.645340 20214 tablet_bootstrap.cc:654] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.649658 20153 tablet_service.cc:1511] Processing CreateTablet for tablet fac4d79f682d4f169331b9656bef55fe (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:29.651175 20153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac4d79f682d4f169331b9656bef55fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.629511 20210 raft_consensus.cc:385] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.669222 19930 tablet_service.cc:1511] Processing CreateTablet for tablet fac4d79f682d4f169331b9656bef55fe (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:29.669721 20209 raft_consensus.cc:359] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.670643 20209 raft_consensus.cc:385] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.671027 20209 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.670745 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fac4d79f682d4f169331b9656bef55fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.659564 20210 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.676693 20210 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:29.677754 20209 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.678836 20249 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.679329 20249 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.679800 20210 ts_tablet_manager.cc:1434] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.001s
I20260709 12:59:29.681449 20210 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:29.685953 20218 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.689599 20218 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.690582 20252 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:29.688267 20129 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:29.691910 20218 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:29.693288 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.694051 20164 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:29.694585 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:29.696349 19906 leader_election.cc:343] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.697078 19907 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:29.698518 20218 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:29.703680 20244 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:29.710834 20249 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:29.706403 20218 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:29.716815 20218 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.718555 20211 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:29.719267 20211 ts_tablet_manager.cc:1403] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.190s	user 0.045s	sys 0.013s
I20260709 12:59:29.724946 20211 raft_consensus.cc:359] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.725847 20211 raft_consensus.cc:385] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.726215 20211 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.727005 20211 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:29.730690 20249 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.731405 20249 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:29.734368 20249 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:29.736696 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:29.736603 20217 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.737404 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:29.737600 20003 tablet_service.cc:1511] Processing CreateTablet for tablet e35516f88d604db2ad4a7f67d0017312 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:29.739017 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:29.739037 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e35516f88d604db2ad4a7f67d0017312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.739665 20217 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:29.723835 20222 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.741585 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:29.742166 20209 ts_tablet_manager.cc:1434] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.170s	user 0.002s	sys 0.007s
I20260709 12:59:29.742283 19940 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.745519 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 12:59:29.747186 19982 leader_election.cc:343] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.748733 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:29.741904 20222 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:29.755452 20222 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:29.756177 19940 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
W20260709 12:59:29.752759 20129 leader_election.cc:343] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.766003 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.766741 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:29.767724 20056 leader_election.cc:343] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:29.768344 19981 consensus_peers.cc:597] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:29.769596 20129 leader_election.cc:343] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:29.768500 20056 leader_election.cc:343] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.770416 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:29.772039 20222 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:29.776310 20130 leader_election.cc:343] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.777030 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:29.778678 20249 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.778555 20072 tablet_service.cc:1511] Processing CreateTablet for tablet e35516f88d604db2ad4a7f67d0017312 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:29.783308 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e35516f88d604db2ad4a7f67d0017312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.784353 20149 tablet_service.cc:1511] Processing CreateTablet for tablet e35516f88d604db2ad4a7f67d0017312 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:29.785725 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e35516f88d604db2ad4a7f67d0017312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.791242 20209 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:29.791014 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:29.795394 20252 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:29.795939 20252 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:29.797353 20210 tablet_bootstrap.cc:654] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.796950 20252 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.817793 20265 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.818528 20265 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.821738 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:29.823642 19906 leader_election.cc:343] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.835076 20214 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:29.835879 20214 ts_tablet_manager.cc:1403] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.221s	user 0.019s	sys 0.039s
I20260709 12:59:29.850735 20265 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:29.850955 20218 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:29.853555 19942 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 1: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 12:59:29.854079 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:29.854664 20218 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:29.855561 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 12:59:29.857019 19907 leader_election.cc:343] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.857554 19907 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 12:59:29.857738 19906 leader_election.cc:343] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.860325 20076 tablet_service.cc:1511] Processing CreateTablet for tablet 88eccd78839140bba038b898679a00b0 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:29.860936 20270 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.861250 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 88eccd78839140bba038b898679a00b0 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:29.864244 20214 raft_consensus.cc:359] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:29.865036 20214 raft_consensus.cc:385] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.865312 20214 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.866799 19999 tablet_service.cc:1511] Processing CreateTablet for tablet fc68b8861f1d4bf09ec026e1ba26cdf6 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:29.867005 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:29.867853 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:29.868449 20249 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.869037 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88eccd78839140bba038b898679a00b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.869225 20077 tablet_service.cc:1511] Processing CreateTablet for tablet 05d19c460b3d4d2cab96fa3f1372b25b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:29.872100 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88eccd78839140bba038b898679a00b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.875641 20148 tablet_service.cc:1511] Processing CreateTablet for tablet fc68b8861f1d4bf09ec026e1ba26cdf6 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:29.877020 20148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc68b8861f1d4bf09ec026e1ba26cdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.877578 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 05d19c460b3d4d2cab96fa3f1372b25b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:29.877640 20214 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:29.878942 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05d19c460b3d4d2cab96fa3f1372b25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.878728 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 05d19c460b3d4d2cab96fa3f1372b25b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:29.880051 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05d19c460b3d4d2cab96fa3f1372b25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.877640 19925 tablet_service.cc:1511] Processing CreateTablet for tablet fc68b8861f1d4bf09ec026e1ba26cdf6 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:29.881678 19925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc68b8861f1d4bf09ec026e1ba26cdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.883481 19924 tablet_service.cc:1511] Processing CreateTablet for tablet 88eccd78839140bba038b898679a00b0 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:29.884673 19924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88eccd78839140bba038b898679a00b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.885056 19999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc68b8861f1d4bf09ec026e1ba26cdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.896775 20210 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:29.897481 20210 ts_tablet_manager.cc:1403] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.216s	user 0.017s	sys 0.012s
I20260709 12:59:29.897373 19998 tablet_service.cc:1511] Processing CreateTablet for tablet 22d3f15573b24fd9aba34f52879de263 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:29.898731 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22d3f15573b24fd9aba34f52879de263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.895265 19838 catalog_manager.cc:5697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:29.902357 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05d19c460b3d4d2cab96fa3f1372b25b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.907162 20214 ts_tablet_manager.cc:1434] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.004s
I20260709 12:59:29.908759 20214 tablet_bootstrap.cc:492] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:29.910275 20007 tablet_service.cc:1511] Processing CreateTablet for tablet 81fd99f42a47497f9d4372157320611b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:29.912048 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fd99f42a47497f9d4372157320611b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:29.911995 20269 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:59:29.914549 20146 tablet_service.cc:1511] Processing CreateTablet for tablet 4b58ddc27cb54fd28e2324df51a5ff24 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:29.919174 20081 tablet_service.cc:1511] Processing CreateTablet for tablet 81fd99f42a47497f9d4372157320611b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:29.920583 20081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fd99f42a47497f9d4372157320611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.939399 20249 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.941033 20072 tablet_service.cc:1511] Processing CreateTablet for tablet 22d3f15573b24fd9aba34f52879de263 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:29.941357 19927 tablet_service.cc:1511] Processing CreateTablet for tablet 4b58ddc27cb54fd28e2324df51a5ff24 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:29.957427 19927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b58ddc27cb54fd28e2324df51a5ff24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.960534 19932 tablet_service.cc:1511] Processing CreateTablet for tablet 81fd99f42a47497f9d4372157320611b (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:29.962324 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fd99f42a47497f9d4372157320611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.964421 20072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22d3f15573b24fd9aba34f52879de263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.942843 20210 raft_consensus.cc:359] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:29.945544 20000 tablet_service.cc:1511] Processing CreateTablet for tablet 4b58ddc27cb54fd28e2324df51a5ff24 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:29.946286 20209 tablet_bootstrap.cc:654] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.947589 20211 ts_tablet_manager.cc:1434] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.228s	user 0.013s	sys 0.021s
I20260709 12:59:29.947788 20073 tablet_service.cc:1511] Processing CreateTablet for tablet 9d934e44cfcb452b9a7d72dc634e9e77 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:29.950186 19929 tablet_service.cc:1511] Processing CreateTablet for tablet 9d934e44cfcb452b9a7d72dc634e9e77 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:29.980018 20000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b58ddc27cb54fd28e2324df51a5ff24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.981452 20153 tablet_service.cc:1511] Processing CreateTablet for tablet 22d3f15573b24fd9aba34f52879de263 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:29.981616 20146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b58ddc27cb54fd28e2324df51a5ff24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.001140 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d934e44cfcb452b9a7d72dc634e9e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:29.976107 20210 raft_consensus.cc:385] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.981541 20218 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.015602 20218 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.019774 20080 tablet_service.cc:1511] Processing CreateTablet for tablet 258312f626c3416b90ed0d3ac81f31e1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:30.028837 19929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d934e44cfcb452b9a7d72dc634e9e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.020900 20150 tablet_service.cc:1511] Processing CreateTablet for tablet 9d934e44cfcb452b9a7d72dc634e9e77 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:30.025315 20211 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:30.026119 20153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22d3f15573b24fd9aba34f52879de263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.047262 20150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d934e44cfcb452b9a7d72dc634e9e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.004680 20210 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.048791 20210 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.051294 20210 ts_tablet_manager.cc:1434] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.153s	user 0.005s	sys 0.002s
I20260709 12:59:30.056028 20080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258312f626c3416b90ed0d3ac81f31e1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:30.063534 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:30.064350 20214 tablet_bootstrap.cc:654] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.083830 19930 tablet_service.cc:1511] Processing CreateTablet for tablet 3e1c1a13f585422ea6d2daf771874d19 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:30.085477 19930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e1c1a13f585422ea6d2daf771874d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.090689 19998 tablet_service.cc:1511] Processing CreateTablet for tablet 258312f626c3416b90ed0d3ac81f31e1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:30.092573 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:30.096287 20217 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.096897 20217 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.099848 20007 tablet_service.cc:1511] Processing CreateTablet for tablet 3e1c1a13f585422ea6d2daf771874d19 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:30.101287 20007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e1c1a13f585422ea6d2daf771874d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.104403 19942 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.109519 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258312f626c3416b90ed0d3ac81f31e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.114882 20222 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.119568 20222 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.119647 20164 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:30.124043 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:30.124246 20218 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:30.125485 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:30.126276 19981 leader_election.cc:343] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.127322 19907 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:30.128707 20210 tablet_bootstrap.cc:492] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:30.121225 20149 tablet_service.cc:1511] Processing CreateTablet for tablet 258312f626c3416b90ed0d3ac81f31e1 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:30.142737 20149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258312f626c3416b90ed0d3ac81f31e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.144948 20222 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:30.145860 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:30.146126 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:30.146682 19940 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
W20260709 12:59:30.147763 20056 leader_election.cc:343] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.148468 20055 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:30.172353 20217 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.173211 20217 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.173926 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.173818 20217 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.174803 20090 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:30.176819 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:30.178370 20250 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.178784 20250 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.179150 20250 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.160676 20211 tablet_bootstrap.cc:654] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.157927 20222 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.192075 20222 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.200685 20222 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.193478 20217 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.198781 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:30.214586 19981 leader_election.cc:343] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.200044 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:30.216640 19907 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 12:59:30.217025 19981 leader_election.cc:343] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.217721 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:30.218053 20218 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.218631 20218 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:30.219740 20218 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.218674 20252 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:30.235613 20146 tablet_service.cc:1511] Processing CreateTablet for tablet b26fb84490474f98b31c0593ddf1b013 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:30.239741 20146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b26fb84490474f98b31c0593ddf1b013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.242184 20073 tablet_service.cc:1511] Processing CreateTablet for tablet 3e1c1a13f585422ea6d2daf771874d19 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:30.243569 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e1c1a13f585422ea6d2daf771874d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.246279 20209 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.247005 20209 ts_tablet_manager.cc:1403] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.456s	user 0.024s	sys 0.001s
I20260709 12:59:30.253698 20209 raft_consensus.cc:359] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.254678 20209 raft_consensus.cc:385] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.255039 20209 raft_consensus.cc:740] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.257227 20210 tablet_bootstrap.cc:654] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.256999 19927 tablet_service.cc:1511] Processing CreateTablet for tablet b26fb84490474f98b31c0593ddf1b013 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:30.258366 19927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b26fb84490474f98b31c0593ddf1b013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.261433 20267 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:30.229802 20250 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.275657 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 12:59:30.277580 19981 leader_election.cc:343] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.277679 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:30.276273 20250 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.278343 20090 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.279181 20211 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:30.279692 20211 ts_tablet_manager.cc:1403] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.255s	user 0.013s	sys 0.013s
I20260709 12:59:30.279877 20000 tablet_service.cc:1511] Processing CreateTablet for tablet b26fb84490474f98b31c0593ddf1b013 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:30.281755 20000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b26fb84490474f98b31c0593ddf1b013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.285802 20214 tablet_bootstrap.cc:492] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:30.286360 20214 ts_tablet_manager.cc:1403] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.378s	user 0.037s	sys 0.043s
I20260709 12:59:30.288043 20090 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:30.290150 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:30.294309 20214 raft_consensus.cc:359] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.295166 20214 raft_consensus.cc:385] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.295459 20214 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.296095 20252 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.296659 20252 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:30.297503 20252 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:30.300625 19982 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Couldn't send request to peer 8b5b4e2a92b046d096f2661c54321d41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:30.303797 20073 tablet_service.cc:1511] Processing CreateTablet for tablet ea4e65a94f914c99a52cbf5467a84a13 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:30.305163 20073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4e65a94f914c99a52cbf5467a84a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.305893 20209 consensus_queue.cc:260] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.306515 20214 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.315157 20222 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.321547 20211 raft_consensus.cc:359] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.322338 20211 raft_consensus.cc:385] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.322650 20211 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.312553 19932 tablet_service.cc:1511] Processing CreateTablet for tablet ea4e65a94f914c99a52cbf5467a84a13 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:30.326416 20209 ts_tablet_manager.cc:1434] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.079s	user 0.010s	sys 0.002s
I20260709 12:59:30.328859 19932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4e65a94f914c99a52cbf5467a84a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.328353 20211 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.331671 20211 ts_tablet_manager.cc:1434] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.000s
I20260709 12:59:30.327863 20209 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:30.343832 19837 catalog_manager.cc:5697] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:30.346647 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:30.347493 19942 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.344810 20222 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:30.352268 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:30.357358 20211 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:30.356786 20209 tablet_bootstrap.cc:654] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:30.358418 20056 leader_election.cc:343] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.364915 19942 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:30.364413 19840 catalog_manager.cc:5697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:30.372025 20146 tablet_service.cc:1511] Processing CreateTablet for tablet ea4e65a94f914c99a52cbf5467a84a13 (DEFAULT_TABLE table=test-workload [id=88adc17616ec4fc48b361759d1d37e56]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:30.373349 20146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4e65a94f914c99a52cbf5467a84a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:30.376685 20211 tablet_bootstrap.cc:654] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.377902 20210 tablet_bootstrap.cc:492] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:30.378451 20210 ts_tablet_manager.cc:1403] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.250s	user 0.031s	sys 0.015s
I20260709 12:59:30.381774 20210 raft_consensus.cc:359] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.381901 20055 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:30.382575 20210 raft_consensus.cc:385] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.383008 20210 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.383430 20285 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.385056 20285 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:30.383831 20210 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.386237 20285 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.387599 20209 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.388207 20209 ts_tablet_manager.cc:1403] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.000s
I20260709 12:59:30.391903 20214 ts_tablet_manager.cc:1434] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.026s
I20260709 12:59:30.393491 20214 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:30.395066 20210 ts_tablet_manager.cc:1434] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:59:30.396490 20210 tablet_bootstrap.cc:492] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:30.396606 20211 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:30.397109 20211 ts_tablet_manager.cc:1403] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.041s	user 0.005s	sys 0.008s
I20260709 12:59:30.400174 20211 raft_consensus.cc:359] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.400971 20211 raft_consensus.cc:385] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.401254 20211 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.393206 20209 raft_consensus.cc:359] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.402055 20209 raft_consensus.cc:385] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.402387 20209 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.403268 20209 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.403865 20210 tablet_bootstrap.cc:654] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.415563 20252 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:30.418496 20214 tablet_bootstrap.cc:654] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.406160 20209 ts_tablet_manager.cc:1434] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:59:30.423225 20209 tablet_bootstrap.cc:492] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:30.406452 20211 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.422043 19837 catalog_manager.cc:5697] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:30.432405 20209 tablet_bootstrap.cc:654] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.434119 20211 ts_tablet_manager.cc:1434] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:59:30.437225 20211 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:30.446092 20285 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.446673 20285 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.448825 20285 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:30.450071 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:30.450799 20164 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:30.452014 20210 tablet_bootstrap.cc:492] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:30.452178 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.452602 20210 ts_tablet_manager.cc:1403] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.056s	user 0.008s	sys 0.010s
I20260709 12:59:30.453575 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:30.455621 20056 leader_election.cc:343] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.455883 20210 raft_consensus.cc:359] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.456610 20210 raft_consensus.cc:385] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.457101 20210 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.457839 20210 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.459892 20210 ts_tablet_manager.cc:1434] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:30.461501 20285 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.461925 20285 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.462282 20285 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.469980 20285 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.473217 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:30.474118 20017 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.474264 20210 tablet_bootstrap.cc:492] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:30.475871 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:30.476511 20164 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.483090 20209 tablet_bootstrap.cc:492] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.483641 20209 ts_tablet_manager.cc:1403] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.061s	user 0.033s	sys 0.026s
I20260709 12:59:30.484647 20164 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:30.485996 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.486902 20222 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.486647 20209 raft_consensus.cc:359] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.487349 20209 raft_consensus.cc:385] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.487638 20209 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.487778 20017 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:30.488535 20209 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.490891 20209 ts_tablet_manager.cc:1434] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:59:30.491451 20222 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:30.492511 20210 tablet_bootstrap.cc:654] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.492395 20222 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.496230 20209 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:30.504360 20209 tablet_bootstrap.cc:654] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.507603 20210 tablet_bootstrap.cc:492] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:30.508322 20210 ts_tablet_manager.cc:1403] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 12:59:30.512887 20210 raft_consensus.cc:359] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.513773 20210 raft_consensus.cc:385] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.514230 20210 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.516131 20209 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.516645 20209 ts_tablet_manager.cc:1403] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.011s
I20260709 12:59:30.515806 20210 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.518872 20210 ts_tablet_manager.cc:1434] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:59:30.521070 20210 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:30.523156 20209 raft_consensus.cc:359] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.524011 20209 raft_consensus.cc:385] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.524274 20209 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.525051 20209 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.527491 20209 ts_tablet_manager.cc:1434] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:59:30.528090 20210 tablet_bootstrap.cc:654] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.528687 20209 tablet_bootstrap.cc:492] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:30.519533 20285 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:30.533381 20211 tablet_bootstrap.cc:654] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.535565 19840 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:30.538065 20214 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:30.538626 20214 ts_tablet_manager.cc:1403] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.145s	user 0.019s	sys 0.066s
I20260709 12:59:30.542114 20214 raft_consensus.cc:359] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.543067 20214 raft_consensus.cc:385] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.543411 20214 raft_consensus.cc:740] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.544333 20214 consensus_queue.cc:260] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.544763 20209 tablet_bootstrap.cc:654] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.546767 20214 ts_tablet_manager.cc:1434] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:59:30.548089 20214 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:30.550563 20211 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:30.551318 20211 ts_tablet_manager.cc:1403] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.001s
I20260709 12:59:30.551856 20210 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:30.552359 20210 ts_tablet_manager.cc:1403] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.003s
I20260709 12:59:30.555161 20210 raft_consensus.cc:359] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.555087 20211 raft_consensus.cc:359] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.556070 20210 raft_consensus.cc:385] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.556293 20211 raft_consensus.cc:385] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.556838 20211 raft_consensus.cc:740] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.560138 20209 tablet_bootstrap.cc:492] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.562458 20209 ts_tablet_manager.cc:1403] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.006s
I20260709 12:59:30.562497 20211 consensus_queue.cc:260] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.564779 20211 ts_tablet_manager.cc:1434] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.004s
I20260709 12:59:30.565809 20211 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:30.565449 20209 raft_consensus.cc:359] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.566171 20209 raft_consensus.cc:385] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.566452 20209 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.567215 20209 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.561669 20214 tablet_bootstrap.cc:654] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.572764 20211 tablet_bootstrap.cc:654] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.575076 20209 ts_tablet_manager.cc:1434] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:59:30.576172 20209 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:30.556388 20210 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.580117 20210 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.580564 20249 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.581099 20249 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.582336 20209 tablet_bootstrap.cc:654] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.582469 20210 ts_tablet_manager.cc:1434] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 12:59:30.583109 20249 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:30.583738 20210 tablet_bootstrap.cc:492] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:30.585069 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:30.585466 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.586320 20090 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:30.588049 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.589206 20249 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.589677 20249 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.590090 20249 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.590396 20210 tablet_bootstrap.cc:654] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.595786 20214 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:30.596288 20214 ts_tablet_manager.cc:1403] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.003s
I20260709 12:59:30.597074 20209 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.597756 20209 ts_tablet_manager.cc:1403] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 12:59:30.585716 19942 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:30.599125 20214 raft_consensus.cc:359] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.599368 20249 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.604458 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:30.605080 20090 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.606226 20209 raft_consensus.cc:359] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.606901 20209 raft_consensus.cc:385] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.607173 20209 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.608594 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:30.608875 20249 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:30.599828 20214 raft_consensus.cc:385] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.609252 19942 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.609519 20214 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.610350 20214 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.612567 20214 ts_tablet_manager.cc:1434] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:59:30.613119 20090 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:30.613845 20214 tablet_bootstrap.cc:492] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:30.614347 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.615231 20249 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.615885 20249 raft_consensus.cc:697] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:30.616739 20249 consensus_queue.cc:237] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.618569 20209 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.620697 20214 tablet_bootstrap.cc:654] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.621331 20209 ts_tablet_manager.cc:1434] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:59:30.622686 20209 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:30.629518 20209 tablet_bootstrap.cc:654] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.632048 20210 tablet_bootstrap.cc:492] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:30.632648 20210 ts_tablet_manager.cc:1403] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.003s
I20260709 12:59:30.635645 20211 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:30.636548 20211 ts_tablet_manager.cc:1403] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.008s
I20260709 12:59:30.638188 20244 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.638900 20244 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.642458 20217 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.643023 20217 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.644619 20210 raft_consensus.cc:359] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.645381 20210 raft_consensus.cc:385] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.645699 20210 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.643102 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:30.647389 19942 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:30.645392 20217 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:30.648876 20218 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.645993 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:30.649526 20218 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.651504 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:30.648052 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:30.652801 20274 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:30.653426 20164 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:30.653802 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:30.654881 20165 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:30.655329 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.656762 20244 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:30.656991 20250 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.657800 20250 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.658555 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.659622 20090 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:30.656792 19907 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:30.661572 20267 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.662122 20267 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.662612 20267 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.664891 20211 raft_consensus.cc:359] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.667075 20211 raft_consensus.cc:385] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.667394 20211 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.668397 20214 tablet_bootstrap.cc:492] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:30.668253 20211 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.668886 20214 ts_tablet_manager.cc:1403] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.055s	user 0.033s	sys 0.020s
I20260709 12:59:30.670473 20211 ts_tablet_manager.cc:1434] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:59:30.674261 20211 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:30.674518 20217 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.678629 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:30.680022 20252 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.680725 20252 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.681118 20250 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:30.675184 20217 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.685076 20252 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:30.686182 20209 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.686733 20209 ts_tablet_manager.cc:1403] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.064s	user 0.023s	sys 0.020s
I20260709 12:59:30.686494 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:30.687384 20017 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:30.688234 20214 raft_consensus.cc:359] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.688395 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.689206 20214 raft_consensus.cc:385] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.685736 19840 catalog_manager.cc:5697] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:30.689515 20214 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.689702 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:30.690002 20209 raft_consensus.cc:359] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.690472 20164 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:30.690384 20214 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.691380 20090 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:30.691871 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.684315 20218 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:30.693035 20252 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.693548 20252 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.694031 20252 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.694725 20211 tablet_bootstrap.cc:654] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.673476 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:30.696388 20165 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:30.682636 20217 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.705281 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.701167 20218 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.706233 20250 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.706740 20250 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.707207 20250 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.710525 20214 ts_tablet_manager.cc:1434] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.006s
I20260709 12:59:30.648571 20210 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.690713 20209 raft_consensus.cc:385] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.715556 20209 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.716440 20209 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.711969 20218 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.719759 20267 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.721171 19941 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 12:59:30.724342 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.725728 20214 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:30.725230 20090 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:30.727202 20209 ts_tablet_manager.cc:1434] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 12:59:30.728288 19906 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:30.729501 20270 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.729885 20270 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.730237 20270 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.731834 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:30.732611 20165 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.743113 20214 tablet_bootstrap.cc:654] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.744632 20165 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:30.746009 19907 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:30.748566 20211 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:30.749229 20211 ts_tablet_manager.cc:1403] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.075s	user 0.034s	sys 0.029s
I20260709 12:59:30.752648 20211 raft_consensus.cc:359] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.753338 20211 raft_consensus.cc:385] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.753667 20211 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.755308 20312 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.756179 20267 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:30.756906 20211 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.757608 20209 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:30.759557 20211 ts_tablet_manager.cc:1434] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:30.761277 20214 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:30.762245 20218 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.739711 20270 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.771965 20214 ts_tablet_manager.cc:1403] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.000s
I20260709 12:59:30.772338 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:30.773130 20090 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.773336 20211 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:30.775111 20209 tablet_bootstrap.cc:654] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.776110 20249 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.776876 20312 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:30.776684 20249 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.777791 20214 raft_consensus.cc:359] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.777928 20312 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.778527 20214 raft_consensus.cc:385] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.778805 20214 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.779161 20249 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:30.781355 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:30.781610 20214 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.781355 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.783805 20091 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
W20260709 12:59:30.785521 20129 leader_election.cc:343] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.786312 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:30.786765 20211 tablet_bootstrap.cc:654] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.788314 20214 ts_tablet_manager.cc:1434] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:59:30.788883 20249 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.798732 20090 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:30.799003 20249 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.799559 20249 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.800515 19906 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:30.801492 20267 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.802109 20267 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:30.794854 20244 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.804260 20244 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 12:59:30.806624 19906 consensus_peers.cc:597] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:30.806766 20244 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.803237 20267 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.809136 20314 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:30.813966 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.814716 20091 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:30.814378 20249 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.816289 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.817013 20214 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:30.817559 20311 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.818063 20311 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.820329 20270 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.820392 20274 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.820942 20274 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.821295 20274 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.822113 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.822909 20091 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:30.825417 19981 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:30.829135 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:30.829916 20091 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.832000 20210 ts_tablet_manager.cc:1434] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.190s	user 0.010s	sys 0.014s
I20260709 12:59:30.834240 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:30.834457 20310 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:30.841305 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:30.841888 20017 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:30.842234 20014 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.836599 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:30.838183 20244 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.839368 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:30.840107 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:30.842303 20091 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:30.830256 20274 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.767400 20250 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.842789 20310 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.883623 20015 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:30.884033 20211 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:30.888679 20211 ts_tablet_manager.cc:1403] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.022s
I20260709 12:59:30.844584 20249 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:30.795691 20217 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.895844 20214 tablet_bootstrap.cc:654] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:30.892081 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 66731.8us
I20260709 12:59:30.814500 20252 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:30.905279 20210 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:30.867807 20244 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.891379 20310 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.894625 20267 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.915347 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:30.907810 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:30.916337 20164 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 12:59:30.916865 20129 leader_election.cc:343] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:30.918377 20274 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.919013 20274 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:30.919478 20274 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:30.920611 20222 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:30.921947 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:30.922545 20091 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.922408 20211 raft_consensus.cc:359] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.923130 20211 raft_consensus.cc:385] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.923418 20211 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.923909 20210 tablet_bootstrap.cc:654] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:30.924108 20211 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:30.925848 20056 consensus_peers.cc:597] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:30.927281 20222 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:30.927716 20311 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:30.929246 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:30.930086 20164 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:30.936532 20309 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:30.942471 20163 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.944741 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:30.945698 20089 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.911141 20267 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:30.947139 19942 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:30.948956 20209 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:30.944916 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:30.949564 20209 ts_tablet_manager.cc:1403] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.192s	user 0.011s	sys 0.009s
I20260709 12:59:30.966981 20211 ts_tablet_manager.cc:1434] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.078s	user 0.003s	sys 0.005s
I20260709 12:59:30.969283 20209 raft_consensus.cc:359] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:30.969794 20209 raft_consensus.cc:385] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:30.969954 20209 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:30.965713 20250 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:30.976619 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:30.984092 20211 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:30.989751 20014 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:30.990343 20267 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:30.992765 20244 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:30.993757 20252 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:30.994196 20274 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:30.995505 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:30.996603 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:30.997562 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:31.008912 20087 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.012668 20252 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:30.998539 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:31.000768 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.013988 19940 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:31.014084 20016 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:31.015880 20129 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:31.016863 20249 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.984208 20217 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:30.999505 20089 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:31.018976 20244 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.017335 20249 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.022773 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.042820 20249 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:31.045171 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:31.045528 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:31.029079 20244 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.050526 19940 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:31.052284 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:31.042891 20091 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:31.052933 20164 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.053496 20214 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:31.054044 20214 ts_tablet_manager.cc:1403] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.237s	user 0.018s	sys 0.000s
I20260709 12:59:31.064323 19840 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.050405 20244 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.069669 20210 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:31.070164 20210 ts_tablet_manager.cc:1403] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.028s
I20260709 12:59:31.056746 20090 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:31.072590 20214 raft_consensus.cc:359] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.075645 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:31.075941 20209 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.071013 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:31.078958 19907 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:31.079795 20087 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:31.080334 20210 raft_consensus.cc:359] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.094384 20210 raft_consensus.cc:385] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.095072 20210 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.087317 20211 tablet_bootstrap.cc:654] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.082798 20214 raft_consensus.cc:385] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.057472 20310 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.097110 20214 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.098826 20210 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:31.101787 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:31.072068 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:31.105659 20252 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:31.106304 20252 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:31.107484 20252 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.110102 20310 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:31.113018 20267 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:31.092499 20249 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.096417 20162 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.121615 20210 ts_tablet_manager.cc:1434] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 12:59:31.119410 20249 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.123286 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:31.123519 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:31.124166 20165 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:31.124432 20087 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:31.126296 20249 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:31.126863 20249 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.127218 20249 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.127524 20249 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.127801 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:31.129961 20130 leader_election.cc:343] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.136742 20015 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:31.149106 20164 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:31.148890 20333 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.156064 19981 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:31.156965 20217 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:31.157492 20217 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:31.157150 19981 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:31.158902 20217 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.160287 20309 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:31.161269 20309 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.167243 20210 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:31.168005 20214 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.168788 20209 ts_tablet_manager.cc:1434] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.219s	user 0.021s	sys 0.024s
I20260709 12:59:31.190016 20214 ts_tablet_manager.cc:1434] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.136s	user 0.010s	sys 0.003s
I20260709 12:59:31.207753 20209 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:31.218571 20267 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:59:31.224987 20211 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:31.225572 20211 ts_tablet_manager.cc:1403] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.244s	user 0.024s	sys 0.005s
I20260709 12:59:31.227406 20244 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.230621 20211 raft_consensus.cc:359] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.231745 20211 raft_consensus.cc:385] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.232168 20211 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.233069 20211 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.233888 20244 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:31.235426 20214 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:31.235303 20162 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:31.235843 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:31.236192 20322 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.247507 20211 ts_tablet_manager.cc:1434] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
W20260709 12:59:31.185474 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 118473us
I20260709 12:59:31.197199 19839 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.257428 20015 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.274487 20210 tablet_bootstrap.cc:654] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.276669 19840 catalog_manager.cc:5697] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.280279 20209 tablet_bootstrap.cc:654] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.281764 20211 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:31.292339 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:31.294189 20249 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.297360 20087 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.300441 20249 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:31.313757 20209 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:31.314217 20209 ts_tablet_manager.cc:1403] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.129s	user 0.008s	sys 0.007s
I20260709 12:59:31.310956 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:31.317162 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:31.319519 19941 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.321425 20244 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:59:31.321834 20129 leader_election.cc:343] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.322023 20244 raft_consensus.cc:697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:31.322365 20129 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:31.323097 20244 consensus_queue.cc:237] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.324313 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:31.332494 20091 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.324800 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:31.325773 20015 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:31.326742 20340 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:31.337975 20130 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:31.338943 20340 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:31.339457 20340 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:31.339717 20249 consensus_queue.cc:1048] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.341641 20334 consensus_queue.cc:1048] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.342332 20091 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:31.340448 20340 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.331579 20214 tablet_bootstrap.cc:654] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.336629 19940 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.347259 20267 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.347806 20267 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.351197 19941 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:31.354867 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.352948 20209 raft_consensus.cc:359] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.356376 20015 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:31.356788 20209 raft_consensus.cc:385] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.357134 20209 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.358090 20210 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:31.358140 20209 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.358520 19907 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:31.360169 20210 ts_tablet_manager.cc:1403] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.195s	user 0.022s	sys 0.007s
I20260709 12:59:31.360522 20218 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.360952 20218 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.361261 20218 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.362962 20210 raft_consensus.cc:359] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.363646 20210 raft_consensus.cc:385] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.364324 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:31.355376 20267 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:31.366112 20249 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:31.363905 20210 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.371054 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:31.376129 19940 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:31.378127 20211 tablet_bootstrap.cc:654] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.384639 20209 ts_tablet_manager.cc:1434] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 12:59:31.377496 20249 raft_consensus.cc:697] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:31.387676 20210 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:31.397274 19906 leader_election.cc:343] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.397634 20267 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.400415 19840 catalog_manager.cc:5697] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.391618 20249 consensus_queue.cc:237] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.402798 19840 catalog_manager.cc:5697] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.408420 20209 tablet_bootstrap.cc:492] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:31.413342 20214 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:31.413950 20214 ts_tablet_manager.cc:1403] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.179s	user 0.010s	sys 0.010s
I20260709 12:59:31.413728 20218 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.415061 20210 ts_tablet_manager.cc:1434] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 12:59:31.416916 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:31.416989 20214 raft_consensus.cc:359] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.420140 20214 raft_consensus.cc:385] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.420486 20214 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.421904 20210 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:31.426689 20214 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.417598 20014 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.442523 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:31.443339 20089 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.445631 20214 ts_tablet_manager.cc:1434] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.005s
I20260709 12:59:31.447117 20214 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:31.458441 20267 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.467037 20014 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:31.468263 19907 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:31.470319 20270 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:31.470803 20270 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:31.471616 20270 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.436502 20218 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:31.500494 20209 tablet_bootstrap.cc:654] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.510015 20211 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:31.510533 20211 ts_tablet_manager.cc:1403] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.231s	user 0.019s	sys 0.034s
I20260709 12:59:31.512981 20089 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:31.513546 20211 raft_consensus.cc:359] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.514309 20211 raft_consensus.cc:385] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.514710 20211 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.514719 20214 tablet_bootstrap.cc:654] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.515586 20211 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.519143 19836 catalog_manager.cc:5697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:31.521095 20211 ts_tablet_manager.cc:1434] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:59:31.522687 20211 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:31.524792 19940 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.526453 20210 tablet_bootstrap.cc:654] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.527983 20015 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.540524 19836 catalog_manager.cc:5697] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.543635 19836 catalog_manager.cc:5697] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.545361 20359 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.550993 20359 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.551190 20340 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.559001 20214 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:31.559437 20214 ts_tablet_manager.cc:1403] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.113s	user 0.013s	sys 0.000s
I20260709 12:59:31.562419 20211 tablet_bootstrap.cc:654] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.564410 20209 tablet_bootstrap.cc:492] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:31.565179 20209 ts_tablet_manager.cc:1403] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.157s	user 0.022s	sys 0.003s
I20260709 12:59:31.568372 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:31.568552 20359 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:31.568799 20209 raft_consensus.cc:359] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.569785 20209 raft_consensus.cc:385] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.568434 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:31.570423 20089 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:31.573047 20210 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:31.572757 19906 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:31.573556 20210 ts_tablet_manager.cc:1403] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.156s	user 0.017s	sys 0.009s
I20260709 12:59:31.575456 20209 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.576488 20209 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.576611 20214 raft_consensus.cc:359] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.576766 20210 raft_consensus.cc:359] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.577441 20214 raft_consensus.cc:385] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.577502 20210 raft_consensus.cc:385] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.577854 20210 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.577952 20332 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.577811 20214 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.579453 20209 ts_tablet_manager.cc:1434] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:59:31.579090 20210 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.581667 20210 ts_tablet_manager.cc:1434] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:31.582847 20209 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:31.582962 20210 tablet_bootstrap.cc:492] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:31.589357 20359 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:31.590572 20209 tablet_bootstrap.cc:654] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.591699 20359 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:31.592281 20359 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:31.593662 20211 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:31.594151 20211 ts_tablet_manager.cc:1403] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.009s
I20260709 12:59:31.594998 20209 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:31.595350 20209 ts_tablet_manager.cc:1403] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 12:59:31.597057 20211 raft_consensus.cc:359] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.596735 19839 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:31.597752 20211 raft_consensus.cc:385] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.598063 20211 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.598361 20210 tablet_bootstrap.cc:654] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.597983 20209 raft_consensus.cc:359] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.598755 20209 raft_consensus.cc:385] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.599059 20209 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.599002 20211 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.600039 20209 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.609035 20209 ts_tablet_manager.cc:1434] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:59:31.610564 20209 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:31.612978 20359 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.623608 19940 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:31.579689 20214 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.624257 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:31.625948 20334 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.626976 20214 ts_tablet_manager.cc:1434] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 12:59:31.627075 20334 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:31.627665 20334 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:31.628775 20334 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.630379 20334 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:31.631969 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:31.640219 20209 tablet_bootstrap.cc:654] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.645175 20310 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.660080 20214 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:31.660310 20211 ts_tablet_manager.cc:1434] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 12:59:31.657059 20359 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:31.664561 20211 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:31.666814 20163 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:31.669248 19907 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:31.674582 20310 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.685055 20359 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:31.686945 20211 tablet_bootstrap.cc:654] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.688401 20359 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:31.698920 20359 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.705015 20209 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:31.709049 20209 ts_tablet_manager.cc:1403] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.099s	user 0.018s	sys 0.031s
I20260709 12:59:31.711568 20217 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.713033 20209 raft_consensus.cc:359] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.714018 20209 raft_consensus.cc:385] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.714367 20209 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.715832 20209 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.719554 20209 ts_tablet_manager.cc:1434] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
W20260709 12:59:31.721022 20369 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:59:31.721143 20209 tablet_bootstrap.cc:492] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:31.726670 20210 tablet_bootstrap.cc:492] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:31.727243 20210 ts_tablet_manager.cc:1403] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.145s	user 0.051s	sys 0.077s
I20260709 12:59:31.727380 20214 tablet_bootstrap.cc:654] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.730388 20209 tablet_bootstrap.cc:654] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.730994 20310 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.732887 20211 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:31.733505 20211 ts_tablet_manager.cc:1403] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.069s	user 0.011s	sys 0.000s
I20260709 12:59:31.736716 20211 raft_consensus.cc:359] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.737569 20211 raft_consensus.cc:385] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.737675 20334 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.737943 20211 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.738415 20334 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.741951 20334 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:31.739135 20361 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.742611 20361 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.744349 20327 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.744771 20361 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:31.745118 20327 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.752928 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:31.752213 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:31.753682 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:31.751039 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:31.747891 20327 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:31.750046 20210 raft_consensus.cc:359] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.759250 20210 raft_consensus.cc:385] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:31.759610 20129 leader_election.cc:343] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:31.760399 20129 leader_election.cc:343] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.764642 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.765023 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.751806 20211 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.759596 20210 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.766156 19940 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
W20260709 12:59:31.769469 19907 leader_election.cc:343] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.774109 20211 ts_tablet_manager.cc:1434] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.000s
I20260709 12:59:31.774999 20211 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:31.776844 20327 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.777041 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:31.778116 20334 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.778508 20334 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.778848 20334 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.782400 20211 tablet_bootstrap.cc:654] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.791390 20214 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:31.791857 20214 ts_tablet_manager.cc:1403] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.164s	user 0.015s	sys 0.001s
I20260709 12:59:31.791965 20090 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.794766 20214 raft_consensus.cc:359] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.795145 20209 tablet_bootstrap.cc:492] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:31.795475 20214 raft_consensus.cc:385] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.795766 20209 ts_tablet_manager.cc:1403] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.075s	user 0.023s	sys 0.005s
I20260709 12:59:31.795819 20214 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.798818 20209 raft_consensus.cc:359] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.799810 20209 raft_consensus.cc:385] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.800100 20209 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.800760 20209 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.796841 20214 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.802850 20209 ts_tablet_manager.cc:1434] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:31.804278 20209 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:31.804955 20214 ts_tablet_manager.cc:1434] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:59:31.806474 20214 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:31.810364 20210 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.758069 20163 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:31.814568 20210 ts_tablet_manager.cc:1434] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.087s	user 0.000s	sys 0.009s
I20260709 12:59:31.815814 20210 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:31.816437 20359 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.816927 20359 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.817334 19907 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:31.818178 20361 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.818490 20340 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.818522 20361 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.819017 20359 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:31.819317 20361 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.819346 20340 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.822012 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.822422 20014 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:31.823755 20214 tablet_bootstrap.cc:654] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.824290 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:31.834440 20244 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.836699 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:31.837268 20210 tablet_bootstrap.cc:654] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.824994 19941 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:31.842362 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:31.822057 20340 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 12:59:31.843235 19907 leader_election.cc:343] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.844993 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:31.847508 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.847322 20361 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.837466 20163 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:31.851095 20334 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.853888 20334 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:31.853762 19839 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:31.855214 20361 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:31.864785 19907 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:31.866505 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:31.867128 20163 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.867617 20322 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.868084 20322 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.868408 20322 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.876161 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:31.876830 20089 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.861964 20244 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.878465 20244 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.878870 20244 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.878134 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:31.881173 19940 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.883394 20340 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:31.885109 20130 leader_election.cc:343] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.886310 20274 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:59:31.891881 20244 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.892793 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:31.886821 20334 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:31.892205 20089 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:31.895311 20334 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:31.899009 20334 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:31.902642 19907 leader_election.cc:343] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.907058 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:31.908517 20211 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:31.909096 20211 ts_tablet_manager.cc:1403] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.134s	user 0.016s	sys 0.035s
I20260709 12:59:31.914757 19940 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:31.901948 20322 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.917711 19942 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 1: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 12:59:31.918865 20322 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:31.917490 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:31.919833 20244 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:31.920075 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:31.920547 20244 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:31.920977 20162 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:31.921185 20244 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:31.921317 20340 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.922210 20244 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.923247 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 12:59:31.924902 19907 leader_election.cc:343] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.925379 19907 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:31.931586 20361 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:31.934686 20340 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.934855 20327 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:31.935127 20163 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:31.935333 20274 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:31.936658 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:31.935436 20327 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:31.938319 20209 tablet_bootstrap.cc:654] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:31.935330 20340 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.953377 20340 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:31.954149 20334 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:31.954778 19907 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:31.956660 20334 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:31.960035 20327 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:31.957361 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:31.966704 20211 raft_consensus.cc:359] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.967438 20211 raft_consensus.cc:385] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:31.967695 20211 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:31.968487 20214 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:31.968997 20214 ts_tablet_manager.cc:1403] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.163s	user 0.027s	sys 0.015s
W20260709 12:59:31.957319 20130 leader_election.cc:343] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.975143 20130 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:31.976027 20327 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:59:31.971661 20129 leader_election.cc:343] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.961558 20361 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:31.978084 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:31.978421 20361 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:31.978760 20017 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:31.979324 20361 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:31.986816 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:31.986819 20210 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:31.988840 20210 ts_tablet_manager.cc:1403] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.173s	user 0.015s	sys 0.008s
I20260709 12:59:31.989729 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:31.971352 20211 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:31.992570 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 12:59:31.989773 20130 leader_election.cc:343] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:31.993824 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:31.994837 20129 leader_election.cc:343] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.000566 20214 raft_consensus.cc:359] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.001308 20214 raft_consensus.cc:385] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.001645 20214 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.009932 20214 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.017572 20214 ts_tablet_manager.cc:1434] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:59:32.018690 20214 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:32.008208 20322 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:59:32.024092 19838 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.001368 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:32.029462 20214 tablet_bootstrap.cc:654] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.033699 20210 raft_consensus.cc:359] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.037287 20210 raft_consensus.cc:385] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.037750 20210 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.038656 20210 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.025221 20252 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.039301 20211 ts_tablet_manager.cc:1434] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.130s	user 0.009s	sys 0.000s
I20260709 12:59:32.030090 20244 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.044656 20210 ts_tablet_manager.cc:1434] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.003s
W20260709 12:59:32.035342 20129 leader_election.cc:343] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.024840 20017 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:32.053296 20211 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.042610 20252 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.056962 20252 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:32.059422 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:32.059728 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.060474 20087 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:32.061275 20244 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:32.062023 20244 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:32.063881 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:32.063130 20244 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.064613 19941 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:32.066828 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:32.067901 20252 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:59:32.052850 20129 leader_election.cc:343] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.073634 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:32.075285 20332 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.068481 20252 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:32.077442 20210 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.086566 20209 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:32.087397 20209 ts_tablet_manager.cc:1403] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.284s	user 0.017s	sys 0.016s
I20260709 12:59:32.079970 20252 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.099020 20210 tablet_bootstrap.cc:654] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.102016 20322 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:32.107684 20209 raft_consensus.cc:359] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.113322 20209 raft_consensus.cc:385] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.113626 20209 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.114398 20209 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.116781 20209 ts_tablet_manager.cc:1434] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:59:32.122774 19840 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.146644 20211 tablet_bootstrap.cc:654] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.168491 20163 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:32.171597 20214 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:32.172210 20214 ts_tablet_manager.cc:1403] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.154s	user 0.025s	sys 0.033s
I20260709 12:59:32.171761 20209 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:32.173381 20359 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:32.176846 20214 raft_consensus.cc:359] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.177506 20214 raft_consensus.cc:385] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.177683 20214 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.178330 20214 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.180155 20214 ts_tablet_manager.cc:1434] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:32.181195 20214 tablet_bootstrap.cc:492] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:32.188270 20087 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:32.184242 20252 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.198698 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:32.199366 19941 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:32.184027 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:32.205049 19839 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:32.215982 20252 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:32.219830 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:32.220614 20088 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.222761 19941 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:32.223346 20214 tablet_bootstrap.cc:654] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.223905 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:32.226496 20252 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:32.228397 20252 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:32.229233 20252 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.229880 20087 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:32.232648 20359 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:32.235818 20244 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.236377 20244 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.238420 20244 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.242841 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.248908 20337 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.249517 20337 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.251787 20210 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:32.252333 20210 ts_tablet_manager.cc:1403] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.201s	user 0.016s	sys 0.007s
I20260709 12:59:32.248800 20088 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:32.259795 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:32.260540 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:32.261178 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.261821 20163 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:32.262626 20210 raft_consensus.cc:359] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.263294 20210 raft_consensus.cc:385] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.263573 20210 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.265018 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:32.266084 20285 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:32.266520 20285 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:32.266863 20285 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:32.267223 20055 leader_election.cc:343] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.254799 20337 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 12:59:32.256094 20129 leader_election.cc:343] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:32.275424 20129 leader_election.cc:343] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.278508 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:32.275681 20210 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.281661 20210 ts_tablet_manager.cc:1434] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:59:32.275941 20211 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:32.282905 20211 ts_tablet_manager.cc:1403] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.235s	user 0.018s	sys 0.024s
I20260709 12:59:32.279584 20244 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.285769 20211 raft_consensus.cc:359] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.286460 20211 raft_consensus.cc:385] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.286718 20211 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.267869 20209 tablet_bootstrap.cc:654] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.287413 20211 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.290998 20210 tablet_bootstrap.cc:492] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.277874 20285 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.298848 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:32.299496 20164 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.301429 20211 ts_tablet_manager.cc:1434] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:59:32.302534 20211 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.305521 20210 tablet_bootstrap.cc:654] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.314513 20285 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.323101 19836 catalog_manager.cc:5697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.324175 20214 tablet_bootstrap.cc:492] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:32.324534 20211 tablet_bootstrap.cc:654] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.324714 20214 ts_tablet_manager.cc:1403] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.144s	user 0.035s	sys 0.055s
I20260709 12:59:32.327533 20214 raft_consensus.cc:359] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.328962 20214 raft_consensus.cc:385] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.329227 20214 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.329989 20214 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.335512 20214 ts_tablet_manager.cc:1434] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:59:32.336660 20214 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:32.337693 20211 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:32.338191 20211 ts_tablet_manager.cc:1403] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.006s
I20260709 12:59:32.338224 20209 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:32.338842 20209 ts_tablet_manager.cc:1403] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.187s	user 0.008s	sys 0.010s
I20260709 12:59:32.319540 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:32.341679 19940 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.341544 20211 raft_consensus.cc:359] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.342270 20211 raft_consensus.cc:385] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.342583 20211 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.343483 20211 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.342500 20209 raft_consensus.cc:359] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.344429 20209 raft_consensus.cc:385] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.319983 20164 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:32.346060 20209 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.347486 20209 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.348340 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:32.349469 20285 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:32.349912 20285 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:32.350754 19940 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:32.350706 20285 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.317746 20210 tablet_bootstrap.cc:492] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:32.352556 20210 ts_tablet_manager.cc:1403] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.003s
I20260709 12:59:32.355516 20210 raft_consensus.cc:359] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.345522 20310 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.359247 20210 raft_consensus.cc:385] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.359597 20210 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.359418 20310 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.360504 20015 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.360399 20210 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.361822 20310 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.345582 20211 ts_tablet_manager.cc:1434] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:32.362483 20210 ts_tablet_manager.cc:1434] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:59:32.362859 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:32.363417 20211 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.366720 20214 tablet_bootstrap.cc:654] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.368882 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.363481 20210 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.369699 20164 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 12:59:32.370935 20211 tablet_bootstrap.cc:654] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.371874 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:32.375654 20210 tablet_bootstrap.cc:654] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.376618 19840 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.390038 20359 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:32.392578 20310 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.393786 20210 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:32.394295 20210 ts_tablet_manager.cc:1403] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.006s
I20260709 12:59:32.396970 20210 raft_consensus.cc:359] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.397668 20210 raft_consensus.cc:385] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.397930 20210 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.358234 20209 ts_tablet_manager.cc:1434] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:59:32.398581 20210 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.403966 20244 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:32.407110 20210 ts_tablet_manager.cc:1434] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:59:32.408293 20210 tablet_bootstrap.cc:492] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.410840 20360 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:32.414721 20210 tablet_bootstrap.cc:654] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.424834 20209 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:32.444092 20359 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.444916 20359 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.445257 20334 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.445799 20334 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.447094 20359 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:32.446918 20244 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:32.448509 20334 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.451295 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.454942 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.455102 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:32.460862 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 12:59:32.461230 20129 leader_election.cc:343] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:32.471442 19906 leader_election.cc:343] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.475078 20211 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:32.475322 20214 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:32.475675 20211 ts_tablet_manager.cc:1403] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.112s	user 0.014s	sys 0.010s
I20260709 12:59:32.476081 20214 ts_tablet_manager.cc:1403] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.140s	user 0.021s	sys 0.011s
W20260709 12:59:32.475231 20129 leader_election.cc:343] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.479528 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:32.480551 20209 tablet_bootstrap.cc:654] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:32.481547 19907 leader_election.cc:343] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.480983 20214 raft_consensus.cc:359] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.481328 20211 raft_consensus.cc:359] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.482920 20214 raft_consensus.cc:385] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.483109 20211 raft_consensus.cc:385] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.483592 20211 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.483490 20214 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.486145 20214 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.491356 20211 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.483737 20332 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.495092 20210 tablet_bootstrap.cc:492] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:32.495610 20210 ts_tablet_manager.cc:1403] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.088s	user 0.030s	sys 0.017s
I20260709 12:59:32.498700 20210 raft_consensus.cc:359] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.499341 20214 ts_tablet_manager.cc:1434] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 12:59:32.499727 20210 raft_consensus.cc:385] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.500407 20210 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.501027 20214 tablet_bootstrap.cc:492] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:32.501395 20210 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.482395 19907 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:32.506314 20210 ts_tablet_manager.cc:1434] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:59:32.511045 20211 ts_tablet_manager.cc:1434] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:59:32.507803 20360 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.528348 20285 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.528923 20285 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.531123 20285 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.531891 20214 tablet_bootstrap.cc:654] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.517177 20217 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.535132 20211 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.538220 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.538884 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 12:59:32.541129 20055 leader_election.cc:343] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.543109 20210 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.538918 20162 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:32.545236 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:32.543516 20217 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.548208 20285 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:32.548621 20285 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:32.548967 20285 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.549487 20210 tablet_bootstrap.cc:654] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.549422 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:32.550321 19940 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 12:59:32.555354 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.556223 20162 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:32.552436 20217 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.557945 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:32.559957 20217 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.562132 20209 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:32.562657 20209 ts_tablet_manager.cc:1403] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.141s	user 0.027s	sys 0.023s
I20260709 12:59:32.565629 20209 raft_consensus.cc:359] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.566581 20209 raft_consensus.cc:385] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.566865 20209 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.572019 20285 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.567546 20209 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.577975 20209 ts_tablet_manager.cc:1434] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:59:32.578948 20209 tablet_bootstrap.cc:492] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:32.579569 20217 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.581276 20337 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.581853 20211 tablet_bootstrap.cc:654] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.581794 20337 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.582334 20310 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.582836 20310 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.583851 20337 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:32.584944 20310 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:32.585911 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.586313 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:32.586591 20164 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:32.587101 20015 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:32.588027 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.588649 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.589394 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 12:59:32.590826 19981 leader_election.cc:343] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.590202 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:32.592622 20337 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.597074 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:32.597978 20162 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.600245 20310 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:32.600783 20310 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:32.601223 20310 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.601658 20337 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.601874 20209 tablet_bootstrap.cc:654] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.606313 20211 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:32.607666 20211 ts_tablet_manager.cc:1403] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.091s	user 0.015s	sys 0.000s
I20260709 12:59:32.610613 20211 raft_consensus.cc:359] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.611346 20211 raft_consensus.cc:385] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.611673 20211 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.612589 20211 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.614956 20211 ts_tablet_manager.cc:1434] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:32.614872 20334 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:32.616397 20211 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.622807 20211 tablet_bootstrap.cc:654] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.632225 20162 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:32.633801 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:32.634923 20337 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:32.635435 20337 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:32.636297 20337 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.636881 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 12:59:32.638746 20055 leader_election.cc:343] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.640201 20285 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.645912 20214 tablet_bootstrap.cc:492] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:32.646716 20214 ts_tablet_manager.cc:1403] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.003s
I20260709 12:59:32.615561 20310 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.650352 20214 raft_consensus.cc:359] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.651039 20214 raft_consensus.cc:385] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.651289 20214 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.651906 20214 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.652878 20334 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:32.655071 20214 ts_tablet_manager.cc:1434] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:59:32.657595 20211 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:32.658232 20211 ts_tablet_manager.cc:1403] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.018s
I20260709 12:59:32.658799 20210 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:32.659402 20210 ts_tablet_manager.cc:1403] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.010s
I20260709 12:59:32.660458 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:32.661099 20162 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.661842 20211 raft_consensus.cc:359] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.662899 20211 raft_consensus.cc:385] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.663272 20211 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.663513 20209 tablet_bootstrap.cc:492] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:32.664275 20209 ts_tablet_manager.cc:1403] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.001s
I20260709 12:59:32.664186 20211 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.664992 20217 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.665501 20217 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.669003 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:32.669550 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.669721 19940 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:32.670192 20164 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:32.667686 20217 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.671310 20211 ts_tablet_manager.cc:1434] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:59:32.671223 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:32.672302 20217 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:32.672681 20217 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:32.673035 20217 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.674063 20209 raft_consensus.cc:359] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.674773 20209 raft_consensus.cc:385] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.675040 20209 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.675707 20209 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.662113 20210 raft_consensus.cc:359] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.676609 20210 raft_consensus.cc:385] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.676918 20210 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.677102 20162 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:32.677775 20210 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.658905 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:32.679991 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:32.682237 20214 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:32.677664 20310 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:32.683000 20252 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:32.683547 20252 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:32.683269 20087 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.684676 20209 ts_tablet_manager.cc:1434] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:59:32.684387 20252 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.686704 20210 ts_tablet_manager.cc:1434] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 12:59:32.687832 20210 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.690013 20214 tablet_bootstrap.cc:654] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.693447 20210 tablet_bootstrap.cc:654] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.694944 20334 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.695451 20334 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.697633 20217 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.698662 20209 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:32.699088 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.697645 20334 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:32.714926 20129 leader_election.cc:343] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.701550 20211 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.715529 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:32.715029 20217 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.717775 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:32.718497 20162 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.700809 19838 catalog_manager.cc:5697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.720126 20209 tablet_bootstrap.cc:654] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:32.720095 20130 leader_election.cc:343] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.722484 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:32.723600 20334 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.724402 20214 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:32.724879 20214 ts_tablet_manager.cc:1403] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.000s
I20260709 12:59:32.721925 20211 tablet_bootstrap.cc:654] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.739110 20322 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.739816 20322 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.742116 20322 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:32.746699 20087 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:32.746146 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.751317 20211 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:32.751472 20337 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:32.756461 20211 ts_tablet_manager.cc:1403] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.000s
I20260709 12:59:32.759456 20211 raft_consensus.cc:359] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:32.754900 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 47524.5us
I20260709 12:59:32.761721 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:32.762392 19940 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.760176 20211 raft_consensus.cc:385] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.764835 20211 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.764631 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:32.765450 20015 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:32.767254 20211 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.771193 19940 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:32.739171 20214 raft_consensus.cc:359] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.774612 20214 raft_consensus.cc:385] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.775041 20214 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.776254 20214 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.777678 20089 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:32.777909 20211 ts_tablet_manager.cc:1434] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:59:32.778088 19907 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:32.779210 20211 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.780921 20252 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.781478 20252 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.784433 20252 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:32.785807 20322 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:32.786280 20322 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:32.786638 20322 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.789280 20210 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:32.791931 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:32.794480 20162 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:32.794374 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.796028 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 12:59:32.796523 19982 leader_election.cc:343] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.801208 19840 catalog_manager.cc:5697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.789978 20210 ts_tablet_manager.cc:1403] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.102s	user 0.020s	sys 0.012s
I20260709 12:59:32.797758 20310 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:32.820077 20285 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:32.802464 20424 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:59:32.828815 20214 ts_tablet_manager.cc:1434] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.001s
I20260709 12:59:32.835505 20310 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:32.836927 20209 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:32.836767 20310 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.837487 20209 ts_tablet_manager.cc:1403] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.139s	user 0.027s	sys 0.027s
I20260709 12:59:32.837952 20214 tablet_bootstrap.cc:492] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:32.840396 20209 raft_consensus.cc:359] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.841086 20209 raft_consensus.cc:385] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.841365 20209 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.842099 20209 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.843855 20211 tablet_bootstrap.cc:654] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:32.797765 19981 leader_election.cc:343] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.844594 20209 ts_tablet_manager.cc:1434] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:32.845192 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:32.850488 20209 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:32.855274 20210 raft_consensus.cc:359] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.855851 20210 raft_consensus.cc:385] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.856200 20210 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.850447 20424 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.860170 20210 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:32.854135 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56406.3us
I20260709 12:59:32.872375 20210 ts_tablet_manager.cc:1434] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 12:59:32.873368 20210 tablet_bootstrap.cc:492] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.882222 19840 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:32.874967 20322 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.889187 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:32.889966 20015 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.890465 20209 tablet_bootstrap.cc:654] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.891077 20424 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:32.892270 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:32.896415 20214 tablet_bootstrap.cc:654] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.895516 20087 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.900103 20361 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.900163 20424 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:32.900846 20361 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.900972 20424 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:32.904659 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:32.904882 20361 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:32.905449 20164 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:32.906642 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:32.888345 20322 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:32.908396 19907 leader_election.cc:343] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.913144 20211 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:32.933314 20211 ts_tablet_manager.cc:1403] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.154s	user 0.015s	sys 0.002s
I20260709 12:59:32.920300 20015 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:32.910693 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 12:59:32.936386 19907 leader_election.cc:343] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.936249 20211 raft_consensus.cc:359] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.936962 19907 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:32.915735 20424 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:32.937922 20322 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:32.921252 20210 tablet_bootstrap.cc:654] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.908543 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:32.926065 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:32.923667 20214 tablet_bootstrap.cc:492] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:32.930562 20209 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:32.942613 20214 ts_tablet_manager.cc:1403] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.002s
I20260709 12:59:32.943255 20252 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:32.943671 20252 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:32.944028 20252 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:32.944790 20209 ts_tablet_manager.cc:1403] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.096s	user 0.020s	sys 0.012s
I20260709 12:59:32.945405 19907 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:32.945487 20214 raft_consensus.cc:359] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.952190 20214 raft_consensus.cc:385] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.951651 20209 raft_consensus.cc:359] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.952502 20322 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:32.952689 20209 raft_consensus.cc:385] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.953078 20209 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.953107 20322 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:32.937018 20211 raft_consensus.cc:385] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.953858 20209 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.954237 20211 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.952491 20214 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.954229 20322 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:32.955317 20211 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:32.955585 20214 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.956321 20209 ts_tablet_manager.cc:1434] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:59:32.907850 20087 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:32.957612 20209 tablet_bootstrap.cc:492] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
W20260709 12:59:32.958470 19981 leader_election.cc:343] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:32.963536 20209 tablet_bootstrap.cc:654] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.965742 20214 ts_tablet_manager.cc:1434] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 12:59:32.967262 20214 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:32.968366 20211 ts_tablet_manager.cc:1434] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:59:32.969559 20211 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:32.975626 20209 tablet_bootstrap.cc:492] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:32.976147 20209 ts_tablet_manager.cc:1403] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 12:59:32.979223 20214 tablet_bootstrap.cc:654] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:32.979601 20209 raft_consensus.cc:359] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.980458 20209 raft_consensus.cc:385] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.980768 20209 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.982016 20209 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.983680 20210 tablet_bootstrap.cc:492] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:32.984143 20209 ts_tablet_manager.cc:1434] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:32.985280 20209 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:32.984205 20210 ts_tablet_manager.cc:1403] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.111s	user 0.013s	sys 0.004s
I20260709 12:59:32.989464 20210 raft_consensus.cc:359] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:32.990108 20210 raft_consensus.cc:385] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:32.990365 20210 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:32.992242 20210 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.001076 20209 tablet_bootstrap.cc:654] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.004264 20210 ts_tablet_manager.cc:1434] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:59:33.006803 20211 tablet_bootstrap.cc:654] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.005353 20210 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:32.963523 20252 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.015252 20210 tablet_bootstrap.cc:654] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.018945 20252 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:33.019589 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 12:59:33.021219 19981 leader_election.cc:343] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.023259 20211 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:33.023753 20211 ts_tablet_manager.cc:1403] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.002s
I20260709 12:59:33.024304 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:33.024959 20164 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.026674 20211 raft_consensus.cc:359] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.027391 20211 raft_consensus.cc:385] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.027652 20211 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.028152 20210 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:33.023753 20214 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.028891 20210 ts_tablet_manager.cc:1403] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.004s
I20260709 12:59:33.028978 20214 ts_tablet_manager.cc:1403] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.062s	user 0.024s	sys 0.019s
I20260709 12:59:33.031158 20211 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.031865 20210 raft_consensus.cc:359] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.032699 20210 raft_consensus.cc:385] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.032968 20210 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.033071 20214 raft_consensus.cc:359] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.033773 20214 raft_consensus.cc:385] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.034161 20214 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.033802 20210 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.034997 20214 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.036717 20210 ts_tablet_manager.cc:1434] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:59:33.037153 20214 ts_tablet_manager.cc:1434] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:59:33.037744 20210 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:33.039227 20211 ts_tablet_manager.cc:1434] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:59:33.040303 20214 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.044732 19838 catalog_manager.cc:5697] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:33.047967 20211 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:33.054724 20211 tablet_bootstrap.cc:654] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.055982 20164 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:33.058422 20214 tablet_bootstrap.cc:654] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.059497 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:33.060813 20252 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.061380 20252 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:33.069319 20322 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.069114 20252 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.069931 20322 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.070554 20209 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:33.071046 20209 ts_tablet_manager.cc:1403] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.004s
I20260709 12:59:33.074081 20322 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.080752 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.082669 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.085327 20211 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:33.085860 20211 ts_tablet_manager.cc:1403] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.006s
W20260709 12:59:33.085987 19907 leader_election.cc:343] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.089076 20211 raft_consensus.cc:359] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.089759 20211 raft_consensus.cc:385] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.090019 20211 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
W20260709 12:59:33.090885 19906 leader_election.cc:343] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.092008 19906 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:33.092859 20361 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:33.094923 20210 tablet_bootstrap.cc:654] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.090651 20211 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.104283 20211 ts_tablet_manager.cc:1434] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:59:33.074119 20209 raft_consensus.cc:359] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.105573 20209 raft_consensus.cc:385] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.105964 20209 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.112129 19941 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.122241 20211 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:33.125401 20209 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.135840 20209 ts_tablet_manager.cc:1434] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.002s
I20260709 12:59:33.136013 20337 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:33.138156 20214 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.138724 20214 ts_tablet_manager.cc:1403] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.099s	user 0.037s	sys 0.030s
I20260709 12:59:33.140092 20285 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:33.141885 20214 raft_consensus.cc:359] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.142653 20214 raft_consensus.cc:385] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.142968 20214 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.143682 20214 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.146294 20214 ts_tablet_manager.cc:1434] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:33.146772 19840 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.151178 20214 tablet_bootstrap.cc:492] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.154874 20209 tablet_bootstrap.cc:492] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:33.158596 20361 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.160050 20211 tablet_bootstrap.cc:654] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.152954 20252 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:59:33.159152 20361 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.167449 20361 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:33.169588 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:33.170034 20214 tablet_bootstrap.cc:654] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.170352 20162 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:33.173554 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.174214 20088 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:33.175444 19907 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:33.182816 20361 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.183701 20361 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.184252 20361 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.189594 20210 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:33.190313 20210 ts_tablet_manager.cc:1403] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.153s	user 0.009s	sys 0.034s
I20260709 12:59:33.194731 20209 tablet_bootstrap.cc:654] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.194502 20210 raft_consensus.cc:359] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.195214 20210 raft_consensus.cc:385] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.195605 20210 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.209391 20210 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.219758 20244 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.220348 20244 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.222474 20244 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.222965 20210 ts_tablet_manager.cc:1434] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 12:59:33.230330 20210 tablet_bootstrap.cc:492] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:33.231092 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.231805 20089 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:33.233712 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.235231 20244 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.235857 20244 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.236435 20244 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.237835 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.238754 20015 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:33.241473 20210 tablet_bootstrap.cc:654] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.245289 20252 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:33.262934 20214 tablet_bootstrap.cc:492] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.263552 20214 ts_tablet_manager.cc:1403] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.113s	user 0.018s	sys 0.025s
I20260709 12:59:33.266636 20214 raft_consensus.cc:359] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.267410 20214 raft_consensus.cc:385] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.270216 20210 tablet_bootstrap.cc:492] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:33.270794 20210 ts_tablet_manager.cc:1403] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.000s
I20260709 12:59:33.267720 20214 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.273008 20214 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.277468 20214 ts_tablet_manager.cc:1434] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
W20260709 12:59:33.278167 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:33.278628 20214 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.283056 20210 raft_consensus.cc:359] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.283842 20210 raft_consensus.cc:385] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.284175 20210 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.284878 20210 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.287075 20210 ts_tablet_manager.cc:1434] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:59:33.289673 20252 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:33.295717 20424 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.296619 20424 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.299008 20424 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.304163 20210 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:33.305529 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:33.306290 19942 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:33.307713 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:33.309818 20424 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.310272 20424 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.310711 20424 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.312598 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:33.317132 19981 leader_election.cc:343] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.310227 20214 tablet_bootstrap.cc:654] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.323335 20428 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:33.325487 20424 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.327876 20424 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.328871 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:33.217242 20361 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:33.335798 19981 leader_election.cc:343] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.336114 20210 tablet_bootstrap.cc:654] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.329246 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:33.337190 19940 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.338357 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:33.338963 20088 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.339882 20211 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:33.340395 20211 ts_tablet_manager.cc:1403] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.218s	user 0.032s	sys 0.012s
I20260709 12:59:33.343547 20211 raft_consensus.cc:359] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.344240 20211 raft_consensus.cc:385] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.344527 20211 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.345733 20361 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:33.346493 20211 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.347803 20210 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:33.348366 20210 ts_tablet_manager.cc:1403] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.003s
I20260709 12:59:33.348742 20211 ts_tablet_manager.cc:1434] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:33.351418 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:33.351249 20210 raft_consensus.cc:359] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.352037 20210 raft_consensus.cc:385] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.352075 20162 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.336458 20334 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.355883 20214 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.362972 20214 ts_tablet_manager.cc:1403] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.001s
I20260709 12:59:33.354355 20088 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:33.362788 20334 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.333530 20244 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.364501 19906 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:33.365942 20322 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.367137 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:33.367822 20015 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.367717 20214 raft_consensus.cc:359] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.368636 20214 raft_consensus.cc:385] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.368979 20214 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.370616 20334 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:33.362159 20210 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.371083 20322 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:33.371399 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:33.371825 20210 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.372037 19942 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:33.373869 20210 ts_tablet_manager.cc:1434] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:59:33.374836 20210 tablet_bootstrap.cc:492] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:33.375010 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:33.372128 20322 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.376168 20214 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.380957 20340 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.381359 20340 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.381680 20340 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.383374 20211 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:33.404604 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.358301 20209 tablet_bootstrap.cc:492] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:33.406072 20209 ts_tablet_manager.cc:1403] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.256s	user 0.037s	sys 0.030s
I20260709 12:59:33.407681 20214 ts_tablet_manager.cc:1434] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:59:33.356359 19940 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:33.409968 20015 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:33.410584 20285 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.428844 20285 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.429749 20211 tablet_bootstrap.cc:654] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.412014 20244 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.430994 20285 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:33.413144 20209 raft_consensus.cc:359] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.432029 20209 raft_consensus.cc:385] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.432372 20130 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 12:59:33.416201 20129 leader_election.cc:343] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.433641 20244 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.434149 20244 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:33.434944 20244 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.423002 20162 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:33.426081 20210 tablet_bootstrap.cc:654] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.432431 20209 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.436542 20209 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.439043 20209 ts_tablet_manager.cc:1434] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 12:59:33.440270 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:33.441725 20209 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:33.441869 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:33.442469 20089 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.442421 20165 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:33.443850 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.444710 20285 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.445107 20285 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.445418 20285 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.448117 20211 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:33.448794 20211 ts_tablet_manager.cc:1403] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.066s	user 0.002s	sys 0.008s
I20260709 12:59:33.451288 20089 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:33.452158 20210 tablet_bootstrap.cc:492] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:33.452394 20428 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.452672 20210 ts_tablet_manager.cc:1403] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.003s
I20260709 12:59:33.452878 20428 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.453811 20285 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.455134 20428 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:33.455466 20210 raft_consensus.cc:359] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.456166 20210 raft_consensus.cc:385] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.456432 20210 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.457089 20210 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.458443 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:33.459084 20014 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.459287 20210 ts_tablet_manager.cc:1434] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:33.459717 20214 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.460920 20210 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:33.456996 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:33.461503 20165 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:33.462711 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.415494 20340 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.435067 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.423182 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:33.464967 20211 raft_consensus.cc:359] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.465780 20211 raft_consensus.cc:385] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.465756 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.466759 20088 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 12:59:33.466799 20340 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:33.467882 20211 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.468106 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:33.468715 19941 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.468679 20211 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.471033 20211 ts_tablet_manager.cc:1434] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:59:33.465289 20252 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.471539 20359 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:33.472014 20211 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:33.472325 20252 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:33.473254 20252 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.479295 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:33.491242 20211 tablet_bootstrap.cc:654] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:33.492684 20129 leader_election.cc:343] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.494951 20214 tablet_bootstrap.cc:654] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.501216 20014 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:33.510310 20428 raft_consensus.cc:2764] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:59:33.511197 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:33.501406 20210 tablet_bootstrap.cc:654] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.512383 20337 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.513015 20337 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:33.514091 20337 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.519065 20214 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.519598 20214 ts_tablet_manager.cc:1403] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.000s
I20260709 12:59:33.521322 19838 catalog_manager.cc:5697] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:33.508847 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:33.505743 20359 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:33.523660 20165 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.524645 20322 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.525174 20322 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.527237 20322 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.522698 20428 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:33.531162 20165 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:33.531241 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.533067 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:33.533089 19907 leader_election.cc:343] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.534385 20359 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:33.535140 19906 leader_election.cc:343] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.535495 20359 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.535981 19906 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:33.537078 20361 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:33.537834 20359 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:33.538275 20285 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:33.539650 20214 raft_consensus.cc:359] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.540364 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:33.540366 20214 raft_consensus.cc:385] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.541828 20214 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.542456 20214 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.543990 20209 tablet_bootstrap.cc:654] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.504734 19941 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:33.546499 20214 ts_tablet_manager.cc:1434] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:59:33.547564 20214 tablet_bootstrap.cc:492] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.548054 20428 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.548776 20428 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.555003 19839 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.556197 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.561066 20428 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:33.562407 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 12:59:33.564759 19982 leader_election.cc:343] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.562521 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.566956 19907 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:33.568665 20359 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:33.569754 20334 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:33.570035 19981 leader_election.cc:343] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.570581 19981 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:33.570345 20334 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.572584 20334 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:33.572952 20252 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:33.574186 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.574990 20214 tablet_bootstrap.cc:654] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.574949 20014 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:33.576258 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.577113 20334 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.577540 20334 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.577934 20334 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.582510 20252 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.583520 20252 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.589026 20252 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:33.589769 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59318.9us
I20260709 12:59:33.593043 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:33.593806 19940 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:33.594971 20334 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.595229 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.555975 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:33.596673 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:33.596844 20244 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.597486 20244 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:33.597579 20424 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.598030 20424 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.598384 20424 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.598322 20244 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.599648 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:33.600310 20014 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.600947 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:33.601655 19940 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:33.608280 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:33.608886 19942 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.609211 20334 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:33.611215 20014 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:33.612583 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.613494 20334 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.614005 20334 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:33.615078 20334 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.616204 20088 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:33.616622 19942 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:33.618189 20361 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.633651 20340 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.657704 20361 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.657953 20340 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.659894 20361 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:33.662403 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.663159 20014 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:33.664546 19907 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:33.665145 20340 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.665469 20361 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:33.665849 20361 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.666394 20361 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:33.638381 20424 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:33.639832 20466 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:59:33.640719 19839 catalog_manager.cc:5697] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.649775 19941 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.679124 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:33.679840 20165 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:33.682549 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.682593 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:33.685024 20088 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.686856 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:33.688828 19940 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.683717 20014 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:33.696300 20130 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.687951 20424 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.653651 19836 catalog_manager.cc:5697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:33.698256 20428 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:33.697499 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:33.697180 20334 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:59:33.700841 20129 leader_election.cc:343] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.701117 20334 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.701663 20334 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.703806 20210 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:33.704306 20210 ts_tablet_manager.cc:1403] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.244s	user 0.021s	sys 0.004s
I20260709 12:59:33.687472 20361 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.707091 20211 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:33.708581 20211 ts_tablet_manager.cc:1403] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.237s	user 0.010s	sys 0.021s
I20260709 12:59:33.696463 20088 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:33.709097 20209 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:33.709584 20209 ts_tablet_manager.cc:1403] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.268s	user 0.012s	sys 0.015s
I20260709 12:59:33.709825 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:33.711233 20424 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.711750 20424 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:33.705147 19940 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:33.713006 20424 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.716886 20334 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.718606 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:33.719214 20014 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:33.719969 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:33.713963 19836 catalog_manager.cc:5697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:33.716240 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:33.720552 20015 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.720890 20165 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:33.728890 20014 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:33.731472 20015 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:33.731828 19907 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:33.728950 19836 catalog_manager.cc:5697] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:33.732667 20481 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:33.733237 20481 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:33.734088 20481 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.730471 20210 raft_consensus.cc:359] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.735486 20361 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:33.738852 20334 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:33.739454 20130 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.740322 20334 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:33.741073 20334 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:33.742539 20209 raft_consensus.cc:359] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.743433 20209 raft_consensus.cc:385] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.743741 20209 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.744441 20209 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.711607 20211 raft_consensus.cc:359] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.745671 20211 raft_consensus.cc:385] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.745952 20211 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.746685 20209 ts_tablet_manager.cc:1434] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:59:33.746280 20334 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.735229 20210 raft_consensus.cc:385] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.747862 20210 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.751003 20209 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:33.753760 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:33.754930 20214 tablet_bootstrap.cc:492] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.763396 20214 ts_tablet_manager.cc:1403] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.216s	user 0.033s	sys 0.041s
I20260709 12:59:33.766222 20214 raft_consensus.cc:359] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.766911 20214 raft_consensus.cc:385] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.767198 20214 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
W20260709 12:59:33.767628 20129 leader_election.cc:343] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.766656 20165 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:33.767872 20214 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:33.759172 19839 catalog_manager.cc:5697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:33.772256 20214 ts_tablet_manager.cc:1434] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:59:33.773552 20214 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.775286 20209 tablet_bootstrap.cc:654] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.746644 20211 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.779826 20214 tablet_bootstrap.cc:654] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.785176 19840 catalog_manager.cc:5697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 2, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:33.788123 20211 ts_tablet_manager.cc:1434] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.001s
I20260709 12:59:33.789110 20211 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:33.753324 20210 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.808898 20211 tablet_bootstrap.cc:654] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.814122 20210 ts_tablet_manager.cc:1434] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.110s	user 0.005s	sys 0.001s
I20260709 12:59:33.817781 20214 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.818775 20214 ts_tablet_manager.cc:1403] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.046s	user 0.032s	sys 0.010s
I20260709 12:59:33.819244 20210 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:33.821580 20214 raft_consensus.cc:359] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.822336 20214 raft_consensus.cc:385] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.822643 20214 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.818023 20209 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:33.823635 20209 ts_tablet_manager.cc:1403] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.029s
I20260709 12:59:33.826429 20209 raft_consensus.cc:359] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.827102 20209 raft_consensus.cc:385] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.827342 20210 tablet_bootstrap.cc:654] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.828162 20211 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:33.827672 20214 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.830314 20214 ts_tablet_manager.cc:1434] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:59:33.827399 20209 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.836608 20337 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.847734 20337 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.848186 20214 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.848055 20209 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.850543 20209 ts_tablet_manager.cc:1434] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.007s
I20260709 12:59:33.850768 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:33.851562 20209 tablet_bootstrap.cc:492] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:33.851653 19941 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:33.834649 19836 catalog_manager.cc:5697] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.855242 20214 tablet_bootstrap.cc:654] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.859416 20209 tablet_bootstrap.cc:654] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.862579 20361 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:33.871884 20214 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.872639 20214 ts_tablet_manager.cc:1403] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.004s
I20260709 12:59:33.876477 20214 raft_consensus.cc:359] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.877447 20214 raft_consensus.cc:385] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.877796 20214 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.878643 20214 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.882134 20214 ts_tablet_manager.cc:1434] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:59:33.883574 20214 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:33.885160 20361 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:33.886915 20473 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.887441 20473 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:33.888541 20428 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:33.889688 20473 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:33.891583 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:33.891583 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:33.892737 19941 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
W20260709 12:59:33.893486 20056 leader_election.cc:343] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.898383 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:33.899484 20214 tablet_bootstrap.cc:654] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.899670 20473 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.900336 20473 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.900738 20473 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.905781 20210 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:33.920313 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.920933 20210 ts_tablet_manager.cc:1403] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.102s	user 0.017s	sys 0.034s
I20260709 12:59:33.922166 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:33.923950 20285 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.925598 20210 raft_consensus.cc:359] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.926282 20210 raft_consensus.cc:385] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.926549 20210 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.927299 20210 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.929059 20210 ts_tablet_manager.cc:1434] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:33.929999 20210 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:33.933818 20285 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:33.935866 20210 tablet_bootstrap.cc:654] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:33.937062 20214 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:33.937589 20214 ts_tablet_manager.cc:1403] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.054s	user 0.005s	sys 0.014s
I20260709 12:59:33.938875 20252 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:33.942420 20334 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.942942 20334 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:33.941068 20214 raft_consensus.cc:359] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.947465 20214 raft_consensus.cc:385] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:33.947748 20214 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:33.948539 20214 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:33.950484 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.951207 20015 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:33.951992 20334 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:33.953547 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:33.957055 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:33.963177 20340 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:33.963616 20340 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:33.964056 20340 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:33.966610 20214 ts_tablet_manager.cc:1434] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:59:33.966382 20390 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:33.967273 20390 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 12:59:33.969141 20129 leader_election.cc:343] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:33.970255 20390 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:33.981335 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:33.988365 20337 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:33.991707 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:33.992995 20015 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:33.828804 20211 ts_tablet_manager.cc:1403] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.008s
I20260709 12:59:34.000346 20214 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:34.002214 20334 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:33.986747 20340 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.010511 20244 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.012950 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:34.021123 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:34.022404 20390 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.026664 20340 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:34.031363 20390 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:34.034519 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:34.054607 20014 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.062249 20209 tablet_bootstrap.cc:492] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:34.062770 20209 ts_tablet_manager.cc:1403] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.211s	user 0.030s	sys 0.046s
I20260709 12:59:34.042409 20337 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.065322 20337 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.007332 20211 raft_consensus.cc:359] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.070742 20211 raft_consensus.cc:385] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.071060 20211 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.072382 20337 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:34.073498 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:34.071730 20211 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.074931 19941 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.053814 20214 tablet_bootstrap.cc:654] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.076258 20211 ts_tablet_manager.cc:1434] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.000s
I20260709 12:59:34.077383 20211 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:34.049206 20473 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.080516 20473 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:34.082410 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:34.083005 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:34.083596 19940 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.085212 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:34.087896 20088 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.088665 19941 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:34.070099 20014 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:34.087786 20217 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.090996 20217 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.093184 20217 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:34.093694 20211 tablet_bootstrap.cc:654] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.094393 20210 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:34.094916 20210 ts_tablet_manager.cc:1403] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.165s	user 0.011s	sys 0.080s
I20260709 12:59:34.095221 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:34.096056 20089 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:34.097577 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:34.067112 20209 raft_consensus.cc:359] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.097756 20210 raft_consensus.cc:359] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.103766 20210 raft_consensus.cc:385] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.104060 20210 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.099215 20217 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:34.099074 20334 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:59:34.105396 20217 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:34.105757 20217 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.106590 20214 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:34.100147 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:34.107147 20214 ts_tablet_manager.cc:1403] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.107s	user 0.010s	sys 0.001s
I20260709 12:59:34.107291 19942 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:34.109882 20214 raft_consensus.cc:359] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.110550 20214 raft_consensus.cc:385] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.110828 20214 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.102320 20462 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:59:34.112380 19940 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:34.111485 20214 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.098403 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:34.114742 20334 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:34.115217 20334 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:34.116569 20214 ts_tablet_manager.cc:1434] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:59:34.117462 20214 tablet_bootstrap.cc:492] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:34.116868 20334 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:34.102769 20056 leader_election.cc:343] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.119086 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:34.120159 20337 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:34.120688 20337 raft_consensus.cc:697] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:34.121569 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:34.121534 20337 consensus_queue.cc:237] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.122593 20473 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:34.123400 20214 tablet_bootstrap.cc:654] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.100545 20209 raft_consensus.cc:385] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.126361 20209 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.130755 20209 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.104740 20210 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.135890 20217 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.150719 20214 tablet_bootstrap.cc:492] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:34.151219 20214 ts_tablet_manager.cc:1403] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.003s
I20260709 12:59:34.152985 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:34.153609 19940 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.153892 20214 raft_consensus.cc:359] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.154597 20214 raft_consensus.cc:385] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.154865 20214 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.155480 20214 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.155545 20473 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.157694 20214 ts_tablet_manager.cc:1434] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.002s
I20260709 12:59:34.158739 20214 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:34.159227 20473 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.140204 20210 ts_tablet_manager.cc:1434] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.000s
I20260709 12:59:34.160992 20210 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:34.140422 20209 ts_tablet_manager.cc:1434] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.003s
I20260709 12:59:34.164618 20214 tablet_bootstrap.cc:654] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.165092 20209 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:34.164736 19838 catalog_manager.cc:5697] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:34.143265 19836 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:34.182840 20334 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.184464 20210 tablet_bootstrap.cc:654] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.186352 20424 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:34.190943 20209 tablet_bootstrap.cc:654] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.194698 20165 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:34.196365 20481 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.199203 20252 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.199748 20252 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.202742 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 12:59:34.204319 19981 leader_election.cc:343] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.099124 20211 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:34.206996 20211 ts_tablet_manager.cc:1403] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.130s	user 0.011s	sys 0.000s
I20260709 12:59:34.207733 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:34.208478 20091 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:34.210469 20424 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.210790 20211 raft_consensus.cc:359] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.212908 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:34.213512 19940 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:34.213940 20424 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:34.214509 20424 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:34.214931 20424 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.215122 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:34.216099 20428 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:34.223095 20014 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:34.224855 20481 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:34.226001 20217 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:34.226169 20252 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:34.231416 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:34.240262 20091 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.245908 20334 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.211771 20211 raft_consensus.cc:385] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.261258 20211 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.262745 20211 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:34.274860 20512 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.065s	user 0.003s	sys 0.000s
I20260709 12:59:34.286217 20211 ts_tablet_manager.cc:1434] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 12:59:34.287277 20211 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:34.296720 20428 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:34.297747 20428 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.301749 20209 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:34.302335 20209 ts_tablet_manager.cc:1403] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.137s	user 0.014s	sys 0.001s
I20260709 12:59:34.302457 20334 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.303042 20334 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.304898 20334 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:34.307598 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:34.307878 20091 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:34.321089 20210 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:34.344709 20210 ts_tablet_manager.cc:1403] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.184s	user 0.009s	sys 0.008s
W20260709 12:59:34.344709 20129 leader_election.cc:343] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:34.324132 20513 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:59:34.327414 20211 tablet_bootstrap.cc:654] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.323760 20209 raft_consensus.cc:359] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.351781 20209 raft_consensus.cc:385] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.352141 20209 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.353111 20209 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.355500 20209 ts_tablet_manager.cc:1434] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.001s
I20260709 12:59:34.356407 20209 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:34.347530 20210 raft_consensus.cc:359] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.359776 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:34.360245 20210 raft_consensus.cc:385] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.360576 20210 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.361020 20091 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:34.361860 20334 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:34.364161 20210 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.366330 20210 ts_tablet_manager.cc:1434] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:59:34.367250 20210 tablet_bootstrap.cc:492] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:34.374476 20424 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.377461 20211 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:34.377981 20211 ts_tablet_manager.cc:1403] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.091s	user 0.010s	sys 0.000s
I20260709 12:59:34.383412 20211 raft_consensus.cc:359] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.384356 20211 raft_consensus.cc:385] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.384689 20211 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.385994 20211 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.388584 20211 ts_tablet_manager.cc:1434] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:59:34.388525 20340 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.390029 20211 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:34.392238 20209 tablet_bootstrap.cc:654] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.395841 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:34.397181 20334 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:34.397679 20334 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:34.398110 20211 tablet_bootstrap.cc:654] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.398077 20334 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.406260 20210 tablet_bootstrap.cc:654] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.418017 20334 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.420953 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:34.421922 19941 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.414366 20514 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.437611 19941 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:34.431941 20514 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.441254 20514 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:34.442992 20285 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.444468 20285 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.446421 20285 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:34.449465 19839 catalog_manager.cc:5697] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:34.452896 20424 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:34.453545 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:34.455418 20334 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:34.456043 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:34.457134 20209 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:34.457237 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:34.457690 20209 ts_tablet_manager.cc:1403] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.101s	user 0.004s	sys 0.007s
I20260709 12:59:34.457850 20088 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.462750 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:34.463421 20424 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:34.464020 20424 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:34.464886 20424 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:34.468322 20129 leader_election.cc:343] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:34.468799 19981 leader_election.cc:343] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.479887 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:34.477368 20209 raft_consensus.cc:359] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.482050 20209 raft_consensus.cc:385] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.482546 20209 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
W20260709 12:59:34.494733 19981 leader_election.cc:343] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.497645 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:34.498845 20515 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:34.515864 20211 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:34.516435 20211 ts_tablet_manager.cc:1403] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.127s	user 0.013s	sys 0.021s
I20260709 12:59:34.519711 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:34.520550 20089 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.522161 20088 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:34.525818 20211 raft_consensus.cc:359] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.530771 20211 raft_consensus.cc:385] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.531178 20211 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.538483 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:34.543700 20211 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.526365 20209 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.546496 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:34.549271 20211 ts_tablet_manager.cc:1434] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:59:34.551757 20211 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:34.555410 20056 leader_election.cc:343] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.563475 20089 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:34.566948 20209 ts_tablet_manager.cc:1434] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.002s
I20260709 12:59:34.575042 20209 tablet_bootstrap.cc:492] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:34.573698 20359 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.576485 20359 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.578676 20359 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:34.581909 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:34.583005 20162 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:34.585953 20462 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:34.586017 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:34.591496 20056 leader_election.cc:343] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.592520 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:34.592763 20088 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:34.596247 20210 tablet_bootstrap.cc:492] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:34.598649 20210 ts_tablet_manager.cc:1403] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.232s	user 0.016s	sys 0.017s
I20260709 12:59:34.601727 19838 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:34.602319 20473 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:34.607275 19906 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:34.602978 20462 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.608542 20359 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.608927 20462 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.609681 20211 tablet_bootstrap.cc:654] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.611749 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:34.614840 20214 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:34.615358 20214 ts_tablet_manager.cc:1403] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.457s	user 0.067s	sys 0.086s
I20260709 12:59:34.617173 20015 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:34.618103 20214 raft_consensus.cc:359] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.618552 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:34.619565 20340 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:34.620057 20340 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:34.620499 20340 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.621274 20462 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:34.620227 20210 raft_consensus.cc:359] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.629040 20210 raft_consensus.cc:385] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.618815 20214 raft_consensus.cc:385] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.630256 20210 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.630767 20214 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.631500 20214 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.632087 20359 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:34.631525 20210 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.634198 20214 ts_tablet_manager.cc:1434] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.004s
I20260709 12:59:34.635309 20214 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:34.635910 20210 ts_tablet_manager.cc:1434] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:59:34.648378 20210 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:34.649158 20244 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:34.649926 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:34.654400 20210 tablet_bootstrap.cc:654] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:34.636354 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:34.656208 20162 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.639444 20340 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.640422 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:34.667203 20334 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:59:34.667289 20129 leader_election.cc:343] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.643147 20214 tablet_bootstrap.cc:654] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.672268 20361 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.672989 20361 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.678488 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:34.679129 20165 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:34.679446 20361 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:34.681304 19907 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:34.686172 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:34.686218 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:34.686883 20014 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:34.688792 19907 leader_election.cc:343] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.691275 20359 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:34.682920 20340 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:34.691906 20359 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:34.699297 20211 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:34.701619 20211 ts_tablet_manager.cc:1403] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.150s	user 0.011s	sys 0.002s
I20260709 12:59:34.699826 20473 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:34.643572 20209 tablet_bootstrap.cc:654] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.707332 20285 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:34.708489 20515 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.709040 20515 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.709201 20211 raft_consensus.cc:359] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.709553 20359 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.709913 20211 raft_consensus.cc:385] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.710265 20211 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.711329 20515 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:34.712697 20211 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.724105 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 12:59:34.727618 19982 leader_election.cc:343] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.738758 20211 ts_tablet_manager.cc:1434] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:59:34.745668 20211 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:34.746417 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:34.758368 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 87526.1us
I20260709 12:59:34.762650 20091 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:34.771793 19981 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:34.772209 20359 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.773222 20424 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:34.773602 20424 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:34.773931 20424 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.782112 20424 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.785394 20424 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:34.788945 20359 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:34.791280 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 12:59:34.803877 19907 leader_election.cc:343] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.808384 20334 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:34.809407 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:34.809311 20334 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.796326 20014 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:34.810214 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:34.812110 20164 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.820173 20209 tablet_bootstrap.cc:492] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:34.820621 20209 ts_tablet_manager.cc:1403] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.246s	user 0.019s	sys 0.015s
I20260709 12:59:34.823093 20210 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:34.823534 20210 ts_tablet_manager.cc:1403] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.176s	user 0.031s	sys 0.026s
I20260709 12:59:34.824496 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 12:59:34.826766 20129 leader_election.cc:343] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.819769 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:34.828003 20090 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.829831 20211 tablet_bootstrap.cc:654] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.834198 20244 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:34.835906 20244 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:34.836815 20244 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.844329 20090 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:34.851987 20515 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:34.852705 19981 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:34.853842 20428 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:34.854385 20428 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:34.856779 20473 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.857417 20473 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:34.859620 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:34.868732 20209 raft_consensus.cc:359] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.869612 20209 raft_consensus.cc:385] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.869947 20209 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.871273 20209 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.859711 20473 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:34.873236 20209 ts_tablet_manager.cc:1434] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 12:59:34.875243 20163 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:34.856559 20210 raft_consensus.cc:359] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.885673 20210 raft_consensus.cc:385] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.877254 20209 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:34.886121 20210 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.888036 20210 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:34.893342 20214 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:34.893858 20214 ts_tablet_manager.cc:1403] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.259s	user 0.021s	sys 0.028s
I20260709 12:59:34.855319 20428 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:34.899224 20214 raft_consensus.cc:359] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.901851 20214 raft_consensus.cc:385] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:34.902221 20214 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:34.909965 20210 ts_tablet_manager.cc:1434] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.086s	user 0.008s	sys 0.000s
I20260709 12:59:34.912803 20210 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:34.906985 20214 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.893958 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:34.887727 20164 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:34.906935 20481 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:34.933897 20214 ts_tablet_manager.cc:1434] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.003s
I20260709 12:59:34.915221 20515 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:59:34.940923 20214 tablet_bootstrap.cc:492] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:34.922315 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:34.943424 19907 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:34.927631 20481 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:34.950234 20359 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:59:34.945724 20055 leader_election.cc:343] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.958356 20211 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:34.959753 20211 ts_tablet_manager.cc:1403] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.214s	user 0.030s	sys 0.055s
I20260709 12:59:34.972409 20359 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:34.973483 20359 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:34.974507 20056 leader_election.cc:343] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:34.974961 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:34.976315 20543 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:34.981297 20214 tablet_bootstrap.cc:654] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:34.967733 19836 catalog_manager.cc:5697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:34.993191 20210 tablet_bootstrap.cc:654] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.013857 20543 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.014496 20543 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:35.020538 20543 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:35.021870 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:35.022698 20015 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:35.024121 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:35.025007 20543 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:35.025413 20543 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:35.025771 20543 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.027024 20481 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:35.029659 20209 tablet_bootstrap.cc:654] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.034556 20543 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:35.039882 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:35.039896 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:35.040844 19940 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.040854 20014 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.043332 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:35.045873 20091 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 12:59:35.015499 20541 log.cc:927] Time spent T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 12:59:35.048837 19940 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:35.009490 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:35.060420 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:35.061740 20473 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:35.061859 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:35.062068 19907 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:35.062528 20473 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:35.063764 20473 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:35.071332 19839 catalog_manager.cc:5697] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:35.075678 20551 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.077s	user 0.003s	sys 0.000s
I20260709 12:59:35.062578 19941 raft_consensus.cc:2376] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1. Re-sending same reply.
I20260709 12:59:35.079479 20514 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:35.082170 20014 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:35.083676 20211 raft_consensus.cc:359] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.085202 20481 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:35.084072 20543 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:35.091733 20211 raft_consensus.cc:385] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.095757 20211 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.102325 20211 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:35.112818 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 147117us
I20260709 12:59:35.115332 20211 ts_tablet_manager.cc:1434] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.155s	user 0.009s	sys 0.000s
I20260709 12:59:35.116775 20211 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:35.123788 20211 tablet_bootstrap.cc:654] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.129172 20549 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.112s
I20260709 12:59:35.147599 20210 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:35.148145 20210 ts_tablet_manager.cc:1403] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.236s	user 0.028s	sys 0.008s
I20260709 12:59:35.152205 20209 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:35.153288 20209 ts_tablet_manager.cc:1403] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.276s	user 0.012s	sys 0.010s
I20260709 12:59:35.152673 20514 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.154656 20514 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.151075 20210 raft_consensus.cc:359] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.157975 20210 raft_consensus.cc:385] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.143877 20163 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:35.159997 20210 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.169420 20210 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.180552 20214 tablet_bootstrap.cc:492] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:35.194142 20214 ts_tablet_manager.cc:1403] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.253s	user 0.010s	sys 0.022s
I20260709 12:59:35.198072 20214 raft_consensus.cc:359] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.199075 20214 raft_consensus.cc:385] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.199412 20214 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.191349 20209 raft_consensus.cc:359] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.200119 20209 raft_consensus.cc:385] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.200551 20209 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.200181 19839 catalog_manager.cc:5697] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:35.201960 20209 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.192878 20514 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:35.205051 20209 ts_tablet_manager.cc:1434] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 12:59:35.206141 20209 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:35.207441 20214 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.209635 20214 ts_tablet_manager.cc:1434] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:59:35.211565 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:35.212330 19941 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:35.213284 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:35.219002 20340 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.222468 20340 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.225324 20340 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:35.225982 20161 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:35.212096 20210 ts_tablet_manager.cc:1434] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 12:59:35.229900 20210 tablet_bootstrap.cc:492] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:35.235121 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:35.219008 20209 tablet_bootstrap.cc:654] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.236336 20514 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:35.236761 20514 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:35.238596 20514 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.242652 20210 tablet_bootstrap.cc:654] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.256815 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:35.263446 20214 tablet_bootstrap.cc:492] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:35.263882 20514 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.264241 20340 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:35.265215 20017 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:35.268276 20209 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:35.268930 20209 ts_tablet_manager.cc:1403] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.003s
I20260709 12:59:35.269236 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:35.269944 19940 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:35.271224 20214 tablet_bootstrap.cc:654] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.272189 20209 raft_consensus.cc:359] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.272925 20209 raft_consensus.cc:385] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.273193 20514 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:35.276923 19836 catalog_manager.cc:5697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:35.284456 20209 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.277316 19839 catalog_manager.cc:5697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:35.286293 20209 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.281105 20210 tablet_bootstrap.cc:492] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:35.287667 20210 ts_tablet_manager.cc:1403] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.004s
I20260709 12:59:35.290802 20210 raft_consensus.cc:359] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.291488 20210 raft_consensus.cc:385] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.291745 20210 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.296926 20209 ts_tablet_manager.cc:1434] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 12:59:35.298382 20209 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:35.292464 20210 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.312705 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:35.313426 20161 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.317263 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:35.319635 20210 ts_tablet_manager.cc:1434] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:59:35.322036 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:35.322553 19940 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.329110 20210 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:35.334949 19940 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:35.336216 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:35.321105 20340 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:59:35.342680 20515 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:35.358314 20515 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:35.358616 20211 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:35.359179 20211 ts_tablet_manager.cc:1403] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.243s	user 0.040s	sys 0.051s
I20260709 12:59:35.359391 20515 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.350658 20549 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.378535 20214 tablet_bootstrap.cc:492] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:35.379142 20214 ts_tablet_manager.cc:1403] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.116s	user 0.003s	sys 0.011s
I20260709 12:59:35.380398 20211 raft_consensus.cc:359] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.383618 20211 raft_consensus.cc:385] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.383885 20211 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.384557 20211 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.385221 20210 tablet_bootstrap.cc:654] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.387387 20089 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.391630 20209 tablet_bootstrap.cc:654] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.409871 20161 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:35.416085 20550 raft_consensus.cc:2764] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:59:35.416970 20014 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.377463 20549 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.429466 20549 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:35.432658 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:35.437019 20211 ts_tablet_manager.cc:1434] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.078s	user 0.010s	sys 0.013s
I20260709 12:59:35.441300 20550 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:35.442142 20424 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:35.457049 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 12:59:35.460181 19981 leader_election.cc:343] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:35.465605 20214 raft_consensus.cc:359] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.466699 20214 raft_consensus.cc:385] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.467060 20214 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
W20260709 12:59:35.468422 20560 log.cc:927] Time spent T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:59:35.477110 20534 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:35.504793 20552 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:35.506322 20473 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:35.512809 19982 leader_election.cc:343] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:35.513432 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:35.467861 20214 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.518489 20211 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:35.515232 20359 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:59:35.526463 20214 ts_tablet_manager.cc:1434] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.147s	user 0.009s	sys 0.000s
I20260709 12:59:35.528021 20214 tablet_bootstrap.cc:492] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:35.533113 20211 tablet_bootstrap.cc:654] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.524678 20514 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:35.539088 20209 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:35.539644 20209 ts_tablet_manager.cc:1403] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.242s	user 0.026s	sys 0.092s
I20260709 12:59:35.542595 20209 raft_consensus.cc:359] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.543356 20209 raft_consensus.cc:385] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.543700 20209 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.548040 20209 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.556206 20209 ts_tablet_manager.cc:1434] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:59:35.556089 20424 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:35.564956 20209 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:35.609504 20473 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.610147 20473 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.612705 20473 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:35.624400 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:35.625164 20015 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:35.626166 20211 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:35.626694 20211 ts_tablet_manager.cc:1403] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.113s	user 0.018s	sys 0.009s
I20260709 12:59:35.615664 20340 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.629647 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:35.631234 20214 tablet_bootstrap.cc:654] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.631803 20340 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:35.633126 20244 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.633625 20244 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.634092 20340 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:35.634258 20210 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:35.636222 20210 ts_tablet_manager.cc:1403] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.316s	user 0.019s	sys 0.016s
I20260709 12:59:35.639292 20209 tablet_bootstrap.cc:654] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.639488 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:35.638935 20210 raft_consensus.cc:359] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:35.640805 20210 raft_consensus.cc:385] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.641224 20210 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.642858 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:35.635619 20244 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:35.645671 20210 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:35.649099 20129 leader_election.cc:343] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:35.651087 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:35.652673 20337 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:35.654635 20210 ts_tablet_manager.cc:1434] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 12:59:35.656574 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 12:59:35.663991 20129 leader_election.cc:343] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:35.655678 20210 tablet_bootstrap.cc:492] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:35.650043 20211 raft_consensus.cc:359] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:35.666647 20211 raft_consensus.cc:385] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.667150 20211 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.668339 20211 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:35.645292 20473 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:59:35.691149 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 46539us
I20260709 12:59:35.691754 20211 ts_tablet_manager.cc:1434] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.001s
W20260709 12:59:35.693295 20056 leader_election.cc:343] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:35.651856 20090 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:35.690718 20473 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:35.741926 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:35.755542 20214 tablet_bootstrap.cc:492] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:35.773550 20214 ts_tablet_manager.cc:1403] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.246s	user 0.017s	sys 0.006s
I20260709 12:59:35.732487 20481 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.739174 20473 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:35.783509 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 106087us
I20260709 12:59:35.805122 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:35.807088 20334 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:35.807459 20334 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:35.807763 20334 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.843403 20211 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:35.854890 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:59:35.867599 19907 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 124482us
I20260709 12:59:35.784447 20481 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.863324 20334 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.880609 20334 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:35.888425 20210 tablet_bootstrap.cc:654] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.895234 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:35.896000 20088 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:35.896876 20211 tablet_bootstrap.cc:654] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.897827 20209 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:35.898329 20209 ts_tablet_manager.cc:1403] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.334s	user 0.016s	sys 0.046s
I20260709 12:59:35.900347 20214 raft_consensus.cc:359] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.901046 20214 raft_consensus.cc:385] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.901333 20214 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.901978 20214 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:35.901118 20209 raft_consensus.cc:359] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.903815 20209 raft_consensus.cc:385] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:35.904214 20209 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:35.905289 20209 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:35.907617 20209 ts_tablet_manager.cc:1434] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
W20260709 12:59:35.910349 20576 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.179s	user 0.002s	sys 0.000s
I20260709 12:59:35.910560 20209 tablet_bootstrap.cc:492] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:35.922242 20209 tablet_bootstrap.cc:654] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.932883 20214 ts_tablet_manager.cc:1434] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.159s	user 0.003s	sys 0.005s
I20260709 12:59:35.934424 20214 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:35.946293 20214 tablet_bootstrap.cc:654] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:35.938647 20285 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:35.988011 20088 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:35.988948 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:36.000102 19839 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:36.010969 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 12:59:35.986443 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 150510us
I20260709 12:59:35.961012 20285 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 12:59:36.026072 20129 leader_election.cc:343] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.028612 20462 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:36.034058 20462 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:36.035437 20462 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.040330 20481 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 12:59:36.078709 20130 leader_election.cc:343] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.102368 20214 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:36.103092 20214 ts_tablet_manager.cc:1403] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.169s	user 0.031s	sys 0.033s
I20260709 12:59:36.105556 20592 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.106146 20592 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.107640 20285 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:36.099498 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:36.111508 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 12:59:36.109220 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68330.9us
W20260709 12:59:36.119204 19981 leader_election.cc:343] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.128422 20210 tablet_bootstrap.cc:492] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:36.125092 20214 raft_consensus.cc:359] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.128961 20210 ts_tablet_manager.cc:1403] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.473s	user 0.017s	sys 0.011s
I20260709 12:59:36.129197 20214 raft_consensus.cc:385] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.130867 20211 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:36.131392 20211 ts_tablet_manager.cc:1403] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.288s	user 0.007s	sys 0.022s
I20260709 12:59:36.131645 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:36.129631 20214 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.135584 20214 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.114635 20244 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:36.142400 20592 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:36.101994 20549 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:59:36.148628 20209 tablet_bootstrap.cc:492] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:36.149151 20209 ts_tablet_manager.cc:1403] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.241s	user 0.017s	sys 0.013s
I20260709 12:59:36.152763 20590 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.155011 20590 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.157910 20590 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:36.173159 20214 ts_tablet_manager.cc:1434] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 12:59:36.178683 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:36.186336 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:36.186124 20211 raft_consensus.cc:359] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.186975 20211 raft_consensus.cc:385] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.189982 20210 raft_consensus.cc:359] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.190786 20210 raft_consensus.cc:385] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.191095 20210 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.192977 20211 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.193836 20211 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.195621 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:36.196578 20090 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:36.200867 20211 ts_tablet_manager.cc:1434] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.069s	user 0.003s	sys 0.004s
I20260709 12:59:36.204082 20473 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.183670 20209 raft_consensus.cc:359] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.207330 20209 raft_consensus.cc:385] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.207279 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:36.207832 20209 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.208166 20158 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:36.208580 20209 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 12:59:36.219368 19906 leader_election.cc:343] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.222584 20209 ts_tablet_manager.cc:1434] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 12:59:36.223642 20209 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
W20260709 12:59:36.224692 20056 leader_election.cc:343] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.226406 19907 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:36.227303 20552 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:36.227841 20552 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:36.228200 20552 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.219391 20214 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:36.191864 20210 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.238097 20012 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:36.239356 20210 ts_tablet_manager.cc:1434] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.110s	user 0.005s	sys 0.000s
I20260709 12:59:36.243458 20211 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:36.252274 20210 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
W20260709 12:59:36.252864 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 156446us
W20260709 12:59:36.254359 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 129055us
I20260709 12:59:36.264237 20473 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:36.266276 20211 tablet_bootstrap.cc:654] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.272150 20244 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.272738 20244 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.274878 20244 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:36.277220 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:36.282527 20090 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 1: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 12:59:36.278867 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:36.332616 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:36.334406 20547 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:59:36.332767 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:36.336023 20016 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:36.337914 20214 tablet_bootstrap.cc:654] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.352293 20552 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.354624 20552 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:36.355188 20552 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.364110 20210 tablet_bootstrap.cc:654] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:36.356029 19981 leader_election.cc:343] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.367105 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:36.375860 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:36.380932 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:36.383661 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:36.384454 20160 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.374943 20592 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:36.376845 20543 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:36.389201 20543 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:36.389695 20543 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.391768 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:36.392449 20090 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.392481 20211 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:36.393257 20211 ts_tablet_manager.cc:1403] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.150s	user 0.014s	sys 0.007s
I20260709 12:59:36.383738 20165 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:36.405176 19907 leader_election.cc:343] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.405715 19907 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:36.407223 20590 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:36.355613 20552 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.417856 20211 raft_consensus.cc:359] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.418838 20211 raft_consensus.cc:385] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.419184 20211 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.420214 20211 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.438578 20211 ts_tablet_manager.cc:1434] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.002s
I20260709 12:59:36.439685 20211 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:36.461851 20209 tablet_bootstrap.cc:654] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.463601 20211 tablet_bootstrap.cc:654] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.458681 20160 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:36.440433 20165 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
W20260709 12:59:36.468938 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 90705.5us
I20260709 12:59:36.439884 20552 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:36.444114 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:36.441244 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:36.449893 20090 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:36.451103 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 12:59:36.528654 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 58843.7us
I20260709 12:59:36.532482 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:36.541261 19907 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:36.464627 20130 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:36.535831 20010 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.543589 20285 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:36.552225 20085 raft_consensus.cc:2393] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:36.563804 20594 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:36.561239 20462 raft_consensus.cc:2764] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:59:36.580044 20594 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:36.580658 20209 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:36.581163 20209 ts_tablet_manager.cc:1403] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.358s	user 0.024s	sys 0.011s
I20260709 12:59:36.581096 20594 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.591428 20214 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:36.592056 20214 ts_tablet_manager.cc:1403] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.373s	user 0.042s	sys 0.045s
I20260709 12:59:36.596920 20214 raft_consensus.cc:359] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.597676 20214 raft_consensus.cc:385] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.597898 20210 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:36.595336 20209 raft_consensus.cc:359] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.598107 19906 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:36.598460 20210 ts_tablet_manager.cc:1403] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.346s	user 0.027s	sys 0.014s
I20260709 12:59:36.598587 20209 raft_consensus.cc:385] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.598963 20209 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.597946 20214 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.602083 20209 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:36.569185 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 108538us
I20260709 12:59:36.605544 20209 ts_tablet_manager.cc:1434] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:59:36.609131 20209 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:36.610432 20543 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.608512 20552 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.642844 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:36.643489 20012 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.641587 20214 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.631958 20285 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:36.662973 20214 ts_tablet_manager.cc:1434] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.004s
I20260709 12:59:36.663497 20543 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:36.675457 20012 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:36.686342 20285 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.702921 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:36.713614 20010 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:36.707248 20543 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:36.715250 20543 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:36.716116 20543 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:36.715646 20210 raft_consensus.cc:359] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.719259 20210 raft_consensus.cc:385] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.719581 20210 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.720284 20210 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.702277 20550 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:59:36.732089 20214 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:36.737247 20210 ts_tablet_manager.cc:1434] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.138s	user 0.007s	sys 0.000s
I20260709 12:59:36.738417 20210 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:36.745409 20552 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:36.750568 20210 tablet_bootstrap.cc:654] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.754110 19836 catalog_manager.cc:5697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:36.754532 20214 tablet_bootstrap.cc:654] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.763536 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 12:59:36.776135 19907 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 98395us
I20260709 12:59:36.816498 20359 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.826565 20359 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:36.828293 20056 leader_election.cc:343] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.830875 20536 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:36.832708 20210 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:36.833226 20210 ts_tablet_manager.cc:1403] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.095s	user 0.013s	sys 0.003s
I20260709 12:59:36.836561 20211 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:36.837097 20211 ts_tablet_manager.cc:1403] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.398s	user 0.008s	sys 0.023s
I20260709 12:59:36.838901 20214 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:36.841790 20214 ts_tablet_manager.cc:1403] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.007s
I20260709 12:59:36.846115 20214 raft_consensus.cc:359] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.846881 20214 raft_consensus.cc:385] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.847148 20214 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.848124 20214 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:36.881254 19907 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 51944.2us
I20260709 12:59:36.881582 20209 tablet_bootstrap.cc:654] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.883030 20214 ts_tablet_manager.cc:1434] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.041s	user 0.012s	sys 0.012s
I20260709 12:59:36.879165 20210 raft_consensus.cc:359] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.883829 20210 raft_consensus.cc:385] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.886616 20214 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:36.884172 20210 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.887867 20210 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.877964 20619 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.898928 20210 ts_tablet_manager.cc:1434] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 12:59:36.899716 20210 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:36.904022 20619 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.906756 20603 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.915458 20603 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.916087 20514 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.916606 20514 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.917680 20603 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:36.918610 20514 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:36.922236 20211 raft_consensus.cc:359] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:36.924863 20211 raft_consensus.cc:385] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:36.925134 20211 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:36.925299 19939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:36.925995 19939 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:36.930964 20619 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:36.931886 19838 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:36.933586 19838 catalog_manager.cc:5697] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:36.934448 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:36.935114 19938 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:36.925799 20211 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:36.939993 20211 ts_tablet_manager.cc:1434] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.000s
I20260709 12:59:36.940871 20211 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:36.941612 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:36.942919 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:36.946805 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:36.947427 20015 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:36.949918 20214 tablet_bootstrap.cc:654] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.951234 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:36.969185 20016 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:36.969376 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:36.954898 20619 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.970074 19938 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:36.970703 20619 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.961871 20340 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:36.971892 20340 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:36.955289 20084 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:36.973301 20619 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 12:59:36.964396 19981 leader_election.cc:343] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.974499 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:36.974745 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:36.975426 20158 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:36.975581 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:36.976425 20428 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:36.976800 20428 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:36.976917 20340 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:36.977178 20055 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:36.977166 20428 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.975522 20549 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:36.978791 20549 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:36.979177 20549 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.982319 20619 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:36.982767 20619 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:36.983098 20619 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.984287 19939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 12:59:36.985556 20055 leader_election.cc:343] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:36.988955 20428 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:36.990104 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:36.993214 19939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:36.994294 20285 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:36.994761 20285 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:36.995155 20285 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.951731 19907 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:36.990470 19839 catalog_manager.cc:5697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:36.998668 20584 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:36.999042 20584 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:36.999367 20584 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:36.999806 20210 tablet_bootstrap.cc:654] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.001511 20359 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:36.998099 20211 tablet_bootstrap.cc:654] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.010039 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 12:59:37.012342 19907 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60848.7us
I20260709 12:59:37.013406 19940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:37.012236 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:37.014055 19940 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.015856 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:37.016391 19906 leader_election.cc:343] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:37.025959 20214 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:37.026702 20214 ts_tablet_manager.cc:1403] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.140s	user 0.008s	sys 0.003s
I20260709 12:59:37.029093 20209 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:37.029614 20209 ts_tablet_manager.cc:1403] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.421s	user 0.016s	sys 0.000s
I20260709 12:59:37.029560 20214 raft_consensus.cc:359] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.030444 20214 raft_consensus.cc:385] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.030742 20214 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.032634 20209 raft_consensus.cc:359] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.033545 20209 raft_consensus.cc:385] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.033865 20209 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.031411 20214 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.038187 20214 ts_tablet_manager.cc:1434] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:59:37.039197 20214 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:37.011114 20285 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:37.041698 20209 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.043275 20086 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 1: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 12:59:37.045588 20549 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.047876 20549 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:37.048094 20214 tablet_bootstrap.cc:654] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:36.991010 20428 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:37.058032 20129 leader_election.cc:343] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:37.101850 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 43905.1us
I20260709 12:59:37.065115 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:37.065471 20209 ts_tablet_manager.cc:1434] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:59:37.105707 20209 tablet_bootstrap.cc:492] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:37.072911 19940 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:37.073109 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:37.010524 20584 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.111317 19938 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 12:59:37.111089 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:37.114131 20584 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:37.080693 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:37.115976 20158 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.082274 20285 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:37.090859 20623 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:37.117095 20623 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.119474 20623 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:37.120133 20550 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:37.120638 20550 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.122635 20550 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:37.094791 20210 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:37.095136 19939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:37.049150 20619 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.123448 20210 ts_tablet_manager.cc:1403] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.224s	user 0.015s	sys 0.004s
I20260709 12:59:37.126145 20084 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 12:59:37.126274 20210 raft_consensus.cc:359] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.127017 20210 raft_consensus.cc:385] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.127262 20210 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.127899 20210 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.130069 20619 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:37.130117 20210 ts_tablet_manager.cc:1434] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:37.136411 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:37.137156 20209 tablet_bootstrap.cc:654] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.137372 20015 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:37.137619 20210 tablet_bootstrap.cc:492] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:37.154174 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:37.155002 20161 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:37.156920 20210 tablet_bootstrap.cc:654] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.139736 20211 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:37.140923 20214 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:37.158730 20214 ts_tablet_manager.cc:1403] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.120s	user 0.008s	sys 0.011s
W20260709 12:59:37.161783 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:59:37.157065 19981 leader_election.cc:343] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:37.162580 20214 raft_consensus.cc:359] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.163380 20214 raft_consensus.cc:385] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.164278 20211 ts_tablet_manager.cc:1403] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.224s	user 0.021s	sys 0.056s
I20260709 12:59:37.165109 19907 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:37.164975 20623 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:37.166023 20584 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:37.166416 20584 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:37.166808 20584 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.163699 20214 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.167289 20211 raft_consensus.cc:359] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.171629 20211 raft_consensus.cc:385] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.172027 20211 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.168608 20214 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.173979 20214 ts_tablet_manager.cc:1434] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.004s
I20260709 12:59:37.175026 20214 tablet_bootstrap.cc:492] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:37.175163 20209 tablet_bootstrap.cc:492] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:37.175793 20209 ts_tablet_manager.cc:1403] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.002s
I20260709 12:59:37.175539 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:37.176712 20211 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.177673 19938 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:37.179051 20211 ts_tablet_manager.cc:1434] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:59:37.181757 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:37.182066 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:37.182569 20015 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:37.190739 20211 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:37.178762 20209 raft_consensus.cc:359] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.193490 20209 raft_consensus.cc:385] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.193883 20209 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.194717 20209 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 12:59:37.195758 20130 leader_election.cc:343] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:37.197994 20209 ts_tablet_manager.cc:1434] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.002s
I20260709 12:59:37.197830 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:37.198513 20016 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 0.
I20260709 12:59:37.198921 20209 tablet_bootstrap.cc:492] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:37.200273 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:37.200868 20085 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.201962 20214 tablet_bootstrap.cc:654] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.205408 20619 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:37.208653 20211 tablet_bootstrap.cc:654] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.209945 20210 tablet_bootstrap.cc:492] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:37.210916 20210 ts_tablet_manager.cc:1403] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.004s
I20260709 12:59:37.214824 20210 raft_consensus.cc:359] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.215518 20210 raft_consensus.cc:385] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.215796 20210 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.216622 20210 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.219027 20210 ts_tablet_manager.cc:1434] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:59:37.219995 20210 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:37.220142 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 12:59:37.222594 20055 leader_election.cc:343] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:37.223430 20056 consensus_peers.cc:597] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Couldn't send request to peer 8b5b4e2a92b046d096f2661c54321d41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:37.225144 20209 tablet_bootstrap.cc:654] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.229094 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:37.230376 20462 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:37.230485 20549 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:37.230886 20462 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:37.231325 20549 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:37.231658 20462 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.232596 20549 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.239655 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:37.243605 20428 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:37.243654 20619 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:37.246088 20210 tablet_bootstrap.cc:654] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.210096 20085 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:37.249984 20285 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:37.250464 20285 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.258237 20285 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:37.267061 19906 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:37.281363 20211 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:37.281970 20211 ts_tablet_manager.cc:1403] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.091s	user 0.010s	sys 0.005s
I20260709 12:59:37.285204 20623 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:37.285943 20623 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:37.287143 20623 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.309350 19941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:37.312647 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:37.327637 20158 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:37.328061 19938 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:37.275856 20543 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
I20260709 12:59:37.330509 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:37.333496 20285 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:37.334127 20285 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:37.335076 20285 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:37.311828 20361 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
W20260709 12:59:37.345598 20055 leader_election.cc:343] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:37.350104 20211 raft_consensus.cc:359] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:37.351415 20211 raft_consensus.cc:385] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.351792 20211 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.330667 20536 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:37.320102 20584 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.362062 20211 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:37.392601 20214 tablet_bootstrap.cc:492] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:37.393195 20214 ts_tablet_manager.cc:1403] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.218s	user 0.048s	sys 0.032s
W20260709 12:59:37.401842 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 55483us
I20260709 12:59:37.389000 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:37.407895 20009 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:37.410851 20211 ts_tablet_manager.cc:1434] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.005s
I20260709 12:59:37.412365 20211 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:37.409265 20646 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:59:37.416833 20462 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.442927 20462 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:37.444597 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:37.434494 20209 tablet_bootstrap.cc:492] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
W20260709 12:59:37.456298 20130 leader_election.cc:343] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:37.448233 20214 raft_consensus.cc:359] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.460019 20209 ts_tablet_manager.cc:1403] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.261s	user 0.050s	sys 0.012s
I20260709 12:59:37.460467 20214 raft_consensus.cc:385] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.461876 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:37.463297 20649 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:37.463686 20649 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:37.464061 20649 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.464097 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:37.464797 20161 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.460906 20214 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.474510 20214 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.480444 20214 ts_tablet_manager.cc:1434] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 12:59:37.450412 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:37.450810 20584 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:37.488001 19942 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.490396 20211 tablet_bootstrap.cc:654] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.402839 20055 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:37.481766 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:37.498371 20014 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.507241 20014 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:37.508653 19907 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:37.480011 20649 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.539309 20536 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:37.512317 20623 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:37.513759 20543 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:59:37.546115 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 143379us
I20260709 12:59:37.554423 20210 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:37.555055 20210 ts_tablet_manager.cc:1403] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.335s	user 0.007s	sys 0.026s
I20260709 12:59:37.559610 20210 raft_consensus.cc:359] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.560468 20210 raft_consensus.cc:385] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.560781 20210 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.564673 20210 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.553032 20623 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:37.574787 20210 ts_tablet_manager.cc:1434] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:59:37.572403 20359 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:37.578105 20359 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:37.547955 20536 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.585145 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:37.585801 20015 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:37.586973 20130 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:37.586688 19836 catalog_manager.cc:5697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:37.587852 20340 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:37.588217 20340 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:37.588536 20340 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.590725 19942 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:37.592893 20536 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:37.594090 20359 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:37.597177 20623 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.597901 19838 catalog_manager.cc:5697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:37.599363 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:37.609256 20210 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Bootstrap starting.
I20260709 12:59:37.609839 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:37.610710 20158 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 12:59:37.612790 20550 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:37.614082 20550 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:37.614176 20649 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:37.615276 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:37.614985 20550 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.617172 20087 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 0.
I20260709 12:59:37.610365 20209 raft_consensus.cc:359] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.622006 20209 raft_consensus.cc:385] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.622797 20209 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.627470 19939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:37.632256 20209 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.601274 20340 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.641507 20599 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:37.640203 20161 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
W20260709 12:59:37.668761 20055 leader_election.cc:343] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:37.688787 20209 ts_tablet_manager.cc:1434] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.228s	user 0.008s	sys 0.000s
I20260709 12:59:37.697201 20209 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
I20260709 12:59:37.664063 20599 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.773626 20210 tablet_bootstrap.cc:654] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.779578 20340 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:37.789247 20654 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:59:37.789638 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:37.790326 20016 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.755422 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:37.797827 20214 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Bootstrap starting.
I20260709 12:59:37.799719 20211 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:37.801654 20211 ts_tablet_manager.cc:1403] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.390s	user 0.012s	sys 0.007s
I20260709 12:59:37.803735 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:37.807633 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:37.813776 20214 tablet_bootstrap.cc:654] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.817346 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:37.818349 20599 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:37.845259 20285 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:37.846130 19906 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:37.837901 20009 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.845021 20599 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:59:37.833264 20086 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.854374 20584 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.900720 20016 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:37.888005 19942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:37.907292 19942 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 1: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:37.919126 20214 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: No bootstrap required, opened a new log
I20260709 12:59:37.920346 20214 ts_tablet_manager.cc:1403] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Time spent bootstrapping tablet: real 0.123s	user 0.016s	sys 0.002s
I20260709 12:59:37.925395 20209 tablet_bootstrap.cc:654] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:37.940716 20130 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:37.927627 20211 raft_consensus.cc:359] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.943087 20211 raft_consensus.cc:385] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.943464 20211 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.944931 20211 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.953686 20537 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:37.954246 20537 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.956086 20086 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:37.956475 20537 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:37.959143 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:37.962503 20211 ts_tablet_manager.cc:1434] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.136s	user 0.007s	sys 0.001s
I20260709 12:59:37.947007 20536 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:37.968209 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:37.973495 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:37.978570 20214 raft_consensus.cc:359] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:37.979589 20214 raft_consensus.cc:385] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.984788 20210 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: No bootstrap required, opened a new log
I20260709 12:59:37.979948 20549 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.987015 20210 ts_tablet_manager.cc:1403] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent bootstrapping tablet: real 0.378s	user 0.035s	sys 0.051s
I20260709 12:59:37.979920 20214 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.987951 20214 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:37.984671 20009 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:37.985456 20165 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:37.989928 20210 raft_consensus.cc:359] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.990619 20210 raft_consensus.cc:385] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:37.990876 20210 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Initialized, Role: FOLLOWER
I20260709 12:59:37.991482 20210 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:37.993500 20210 ts_tablet_manager.cc:1434] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:37.994213 20549 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:37.996963 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:37.997507 20536 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:37.998528 20655 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:37.999179 20655 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:37.975770 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:38.000087 20655 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:37.998458 20536 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:37.946949 20361 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:59:38.010140 20214 ts_tablet_manager.cc:1434] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.002s
I20260709 12:59:38.017688 20584 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:38.043694 20653 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:38.045035 20340 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:38.045069 20653 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:38.047636 20653 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:38.058877 20537 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:38.065315 20547 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:38.076622 20211 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:38.070362 20669 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.223s	user 0.003s	sys 0.000s
I20260709 12:59:38.098378 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 12:59:38.118686 20056 leader_election.cc:343] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:38.122561 20209 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:38.123131 20209 ts_tablet_manager.cc:1403] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.426s	user 0.025s	sys 0.008s
I20260709 12:59:38.077649 20361 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 12:59:38.124047 20661 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:38.142050 20661 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.160444 20209 raft_consensus.cc:359] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.161224 20209 raft_consensus.cc:385] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:38.161484 20209 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:38.164273 20661 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:38.177824 19840 catalog_manager.cc:5697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:38.182760 20211 tablet_bootstrap.cc:654] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:38.275841 20209 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.292052 20209 ts_tablet_manager.cc:1434] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.168s	user 0.007s	sys 0.000s
I20260709 12:59:38.343043 20211 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:38.343587 20211 ts_tablet_manager.cc:1403] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.267s	user 0.007s	sys 0.007s
I20260709 12:59:38.346565 20211 raft_consensus.cc:359] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.347251 20211 raft_consensus.cc:385] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:38.347519 20211 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:38.348248 20211 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.350126 20211 ts_tablet_manager.cc:1434] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:38.352887 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:38.353639 20163 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:38.360580 20211 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:38.360618 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:38.362388 20599 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:38.362807 20599 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:38.363198 20599 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:38.367687 20211 tablet_bootstrap.cc:654] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:38.371482 20599 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.378250 20211 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: No bootstrap required, opened a new log
I20260709 12:59:38.378824 20211 ts_tablet_manager.cc:1403] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 12:59:38.380658 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:38.381295 20163 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:38.393448 20163 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:38.394925 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:38.396121 20549 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:38.396978 20549 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:38.398123 20549 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.396202 20599 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:38.400899 20211 raft_consensus.cc:359] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:38.401665 20211 raft_consensus.cc:385] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:38.401962 20211 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:38.402752 20211 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:38.409272 20211 ts_tablet_manager.cc:1434] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 12:59:38.438925 20584 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:38.439569 20584 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:38.441682 20584 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:38.443281 20537 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:38.456352 20537 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:38.484552 20537 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:38.499698 20652 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:38.530174 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:38.558846 20695 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:38.559434 20695 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:38.565851 20695 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:38.567700 19937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:38.568444 19937 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 0.
I20260709 12:59:38.574025 19934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:38.574843 19934 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:38.581483 19939 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:38.582273 19939 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 0.
I20260709 12:59:38.582978 19934 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:38.585497 19939 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:38.592685 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:38.593932 20543 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:38.594270 20543 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:38.594590 20543 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:38.602098 20543 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:38.605429 20543 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:38.605726 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:38.608246 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 12:59:38.610528 20056 leader_election.cc:343] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:38.617149 20649 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:38.619647 19935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:38.620330 19935 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:38.634706 19935 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:38.641626 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:38.643252 20543 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:38.643774 20543 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:38.644652 20543 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:38.650063 19840 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60996 (request call id 89) took 1222 ms. Trace:
W20260709 12:59:38.653227 19888 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:38.650511 19840 rpcz_store.cc:276] 0709 12:59:37.431384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:37.431550 (+   166us) service_pool.cc:224] Handling call
0709 12:59:38.649987 (+1218437us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:59:38.225971 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:59:38.226556 (+   585us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:59:38.226599 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:59:38.226619 (+    20us) tablet.cc:662] Decoding operations
0709 12:59:38.227033 (+   414us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:59:38.227103 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 12:59:38.227134 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:59:38.227358 (+   224us) tablet.cc:701] Row locks acquired
0709 12:59:38.227372 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:59:38.227549 (+   177us) write_op.cc:270] Start()
0709 12:59:38.227669 (+   120us) write_op.cc:276] Timestamp: P: 1783601978227498 usec, L: 0
0709 12:59:38.227689 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:59:38.228384 (+   695us) log.cc:844] Serialized 537 byte log entry
0709 12:59:38.230468 (+  2084us) op_driver.cc:464] REPLICATION: finished
0709 12:59:38.294209 (+ 63741us) write_op.cc:301] APPLY: starting
0709 12:59:38.299642 (+  5433us) tablet.cc:1367] starting BulkCheckPresence
0709 12:59:38.301944 (+  2302us) tablet.cc:1370] finished BulkCheckPresence
0709 12:59:38.309487 (+  7543us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:59:38.311375 (+  1888us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:59:38.314568 (+  3193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:59:38.314604 (+    36us) write_op.cc:312] APPLY: finished
0709 12:59:38.314837 (+   233us) log.cc:844] Serialized 22 byte log entry
0709 12:59:38.315026 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:38.315107 (+    81us) write_op.cc:454] Released schema lock
0709 12:59:38.315150 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:59:38.316218 (+  1068us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:38.316247 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66600,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":48535,"apply.run_cpu_time_us":2594,"apply.run_wall_time_us":22695,"num_ops":1,"prepare.queue_time_us":811,"prepare.run_cpu_time_us":2718,"prepare.run_wall_time_us":2719,"replication_time_us":2646}]]}
I20260709 12:59:38.726832 20617 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:38.735463 20209 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Bootstrap starting.
W20260709 12:59:38.741021 19907 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67135.6us
I20260709 12:59:38.797094 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:38.798338 20165 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 1: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 12:59:38.806514 20359 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:38.818571 19907 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:38.821457 20594 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:38.831437 20209 tablet_bootstrap.cc:654] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:38.842767 20695 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:38.850908 20209 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: No bootstrap required, opened a new log
I20260709 12:59:38.851471 20209 ts_tablet_manager.cc:1403] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.003s
I20260709 12:59:38.868889 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:38.870110 20209 raft_consensus.cc:359] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:38.870870 20209 raft_consensus.cc:385] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:38.871135 20209 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Initialized, Role: FOLLOWER
I20260709 12:59:38.877491 19838 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
W20260709 12:59:38.882594 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 221165us
I20260709 12:59:38.871774 20209 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:38.886529 20209 ts_tablet_manager.cc:1434] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:59:38.892092 20549 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:38.920138 19907 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:38.921029 20694 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:38.946751 19839 catalog_manager.cc:5697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:38.956389 20547 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:39.010694 19837 catalog_manager.cc:5697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:39.013286 19839 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 81) took 1146 ms. Trace:
I20260709 12:59:39.013725 19839 rpcz_store.cc:276] 0709 12:59:37.867329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:37.868206 (+   877us) service_pool.cc:224] Handling call
0709 12:59:38.384642 (+516436us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b54008b1fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 12:59:39.013228 (+628586us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:59:38.954491 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:59:38.955067 (+   576us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:59:38.955109 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:59:38.955130 (+    21us) tablet.cc:662] Decoding operations
0709 12:59:38.955492 (+   362us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:59:38.955549 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:59:38.955581 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:59:38.955776 (+   195us) tablet.cc:701] Row locks acquired
0709 12:59:38.955793 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:59:38.955938 (+   145us) write_op.cc:270] Start()
0709 12:59:38.956046 (+   108us) write_op.cc:276] Timestamp: P: 1783601978955875 usec, L: 0
0709 12:59:38.956068 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:59:38.956699 (+   631us) log.cc:844] Serialized 537 byte log entry
0709 12:59:38.961473 (+  4774us) op_driver.cc:464] REPLICATION: finished
0709 12:59:38.962415 (+   942us) write_op.cc:301] APPLY: starting
0709 12:59:38.962490 (+    75us) tablet.cc:1367] starting BulkCheckPresence
0709 12:59:38.962602 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 12:59:38.962736 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:59:38.963031 (+   295us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:59:38.963243 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:59:38.963308 (+    65us) write_op.cc:312] APPLY: finished
0709 12:59:38.963526 (+   218us) log.cc:844] Serialized 22 byte log entry
0709 12:59:38.963762 (+   236us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:59:38.963870 (+   108us) write_op.cc:454] Released schema lock
0709 12:59:38.964704 (+   834us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":284164,"reported_tablets":12,"spinlock_wait_cycles":74624896,"child_traces":[["op",{"apply.queue_time_us":745,"apply.run_cpu_time_us":1894,"apply.run_wall_time_us":2633,"num_ops":1,"prepare.queue_time_us":2712,"prepare.run_cpu_time_us":2483,"prepare.run_wall_time_us":2484,"replication_time_us":5289}]]}
I20260709 12:59:39.045502 19838 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 1 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 12:59:39.047513 19838 catalog_manager.cc:5697] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:39.071898 19835 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 0 to 1, leader changed from <none> to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 1 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 12:59:39.232592 20161 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:39.244401 20652 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:39.264703 20653 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:39.550221 20112 debug-util.cc:398] Leaking SignalData structure 0x7b0800257aa0 after lost signal to thread 12412
W20260709 12:59:39.551570 20112 debug-util.cc:398] Leaking SignalData structure 0x7b0800194d40 after lost signal to thread 19874
W20260709 12:59:39.552987 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4720 after lost signal to thread 19966
W20260709 12:59:39.554191 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08001703a0 after lost signal to thread 20041
W20260709 12:59:39.555358 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6f40 after lost signal to thread 20115
W20260709 12:59:39.556347 20112 debug-util.cc:398] Leaking SignalData structure 0x7b0800126a40 after lost signal to thread 20189
W20260709 12:59:39.556653 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08001d03e0 after lost signal to thread 20210
W20260709 12:59:39.769050 19907 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55260.1us
I20260709 12:59:40.742381 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
W20260709 12:59:40.756134 20055 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:59:40.764524 20055 consensus_peers.cc:597] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:40.791838 19981 consensus_peers.cc:597] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:40.796278 19981 consensus_peers.cc:597] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:40.806595 19981 consensus_peers.cc:597] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.816484 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:40.817845 12409 tablet_replica.cc:333] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.819402 12409 raft_consensus.cc:2243] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.820843 12409 raft_consensus.cc:2272] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.824363 12409 tablet_replica.cc:333] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
W20260709 12:59:40.824672 19981 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:40.826751 20055 consensus_peers.cc:597] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.827176 12409 raft_consensus.cc:2243] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:59:40.827397 19981 consensus_peers.cc:597] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.827992 12409 raft_consensus.cc:2272] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.830508 20129 consensus_peers.cc:597] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.831508 12409 tablet_replica.cc:333] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.832523 12409 raft_consensus.cc:2243] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.833101 12409 raft_consensus.cc:2272] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.836474 12409 tablet_replica.cc:333] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.837695 12409 raft_consensus.cc:2243] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.838392 12409 raft_consensus.cc:2272] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.841730 12409 tablet_replica.cc:333] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.842767 12409 raft_consensus.cc:2243] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.844096 12409 raft_consensus.cc:2272] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.847437 12409 tablet_replica.cc:333] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.848268 12409 raft_consensus.cc:2243] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.849241 12409 raft_consensus.cc:2272] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.852265 12409 tablet_replica.cc:333] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.853415 12409 raft_consensus.cc:2243] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.854025 12409 raft_consensus.cc:2272] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.857491 12409 tablet_replica.cc:333] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.858484 12409 raft_consensus.cc:2243] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.859200 12409 raft_consensus.cc:2272] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.862962 12409 tablet_replica.cc:333] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.863977 12409 raft_consensus.cc:2243] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.865296 12409 raft_consensus.cc:2272] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.870342 12409 tablet_replica.cc:333] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.871501 12409 raft_consensus.cc:2243] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.872306 12409 raft_consensus.cc:2272] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.873723 20129 consensus_peers.cc:597] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.876622 12409 tablet_replica.cc:333] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.878445 12409 raft_consensus.cc:2243] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.879179 12409 raft_consensus.cc:2272] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.883285 12409 tablet_replica.cc:333] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.884234 12409 raft_consensus.cc:2243] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.885156 12409 raft_consensus.cc:2272] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.888904 12409 tablet_replica.cc:333] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.889902 12409 raft_consensus.cc:2243] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.891100 12409 raft_consensus.cc:2272] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.892020 19981 consensus_peers.cc:597] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.894641 12409 tablet_replica.cc:333] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.895857 12409 raft_consensus.cc:2243] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.896591 12409 raft_consensus.cc:2272] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.900882 12409 tablet_replica.cc:333] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.901800 12409 raft_consensus.cc:2243] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.902630 12409 raft_consensus.cc:2272] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.905009 20129 consensus_peers.cc:597] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.907569 12409 tablet_replica.cc:333] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.908684 12409 raft_consensus.cc:2243] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.909294 12409 raft_consensus.cc:2272] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.912624 12409 tablet_replica.cc:333] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.913491 12409 raft_consensus.cc:2243] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.914223 12409 raft_consensus.cc:2272] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.915512 20129 consensus_peers.cc:597] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.917713 12409 tablet_replica.cc:333] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
W20260709 12:59:40.918049 20129 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.918814 12409 raft_consensus.cc:2243] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.919605 12409 raft_consensus.cc:2272] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.920841 20055 consensus_peers.cc:597] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.923349 12409 tablet_replica.cc:333] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.924486 12409 raft_consensus.cc:2243] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.925947 12409 raft_consensus.cc:2272] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.929253 12409 tablet_replica.cc:333] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.930119 12409 raft_consensus.cc:2243] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.930784 12409 raft_consensus.cc:2272] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.934067 12409 tablet_replica.cc:333] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.935068 12409 raft_consensus.cc:2243] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.935694 12409 raft_consensus.cc:2272] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.936589 19981 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.939221 12409 tablet_replica.cc:333] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.940225 12409 raft_consensus.cc:2243] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.941437 12409 raft_consensus.cc:2272] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.945382 12409 tablet_replica.cc:333] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.946278 12409 raft_consensus.cc:2243] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.946915 12409 raft_consensus.cc:2272] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:40.949433 19981 consensus_peers.cc:597] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.950757 12409 tablet_replica.cc:333] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.961580 12409 raft_consensus.cc:2243] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.962323 12409 raft_consensus.cc:2272] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.970101 12409 tablet_replica.cc:333] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.970957 12409 raft_consensus.cc:2243] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.971671 12409 raft_consensus.cc:2272] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.975266 12409 tablet_replica.cc:333] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
W20260709 12:59:40.976136 20129 consensus_peers.cc:597] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.976184 12409 raft_consensus.cc:2243] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:59:40.977173 20129 consensus_peers.cc:597] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:40.977386 12409 raft_consensus.cc:2272] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.981233 12409 tablet_replica.cc:333] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.982353 12409 raft_consensus.cc:2243] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.983639 12409 raft_consensus.cc:2272] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.987201 12409 tablet_replica.cc:333] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.989346 12409 raft_consensus.cc:2243] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:40.989997 12409 raft_consensus.cc:2272] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:40.993544 12409 tablet_replica.cc:333] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:40.994451 12409 raft_consensus.cc:2243] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:40.997756 12409 raft_consensus.cc:2272] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.001345 12409 tablet_replica.cc:333] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
W20260709 12:59:41.001878 19981 consensus_peers.cc:597] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.001976 12409 raft_consensus.cc:2243] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.002912 12409 raft_consensus.cc:2272] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.006286 12409 tablet_replica.cc:333] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.007160 12409 raft_consensus.cc:2243] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.007898 12409 raft_consensus.cc:2272] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.010809 12409 tablet_replica.cc:333] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.011432 12409 raft_consensus.cc:2243] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:41.012506 12409 raft_consensus.cc:2272] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.015308 12409 tablet_replica.cc:333] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.016162 12409 raft_consensus.cc:2243] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:41.019557 12409 raft_consensus.cc:2272] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.022823 12409 tablet_replica.cc:333] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.023486 12409 raft_consensus.cc:2243] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.024328 12409 raft_consensus.cc:2272] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.027094 12409 tablet_replica.cc:333] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.027673 12409 raft_consensus.cc:2243] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:41.028704 12409 raft_consensus.cc:2272] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.033082 12409 tablet_replica.cc:333] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.033958 12409 raft_consensus.cc:2243] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:41.035149 12409 raft_consensus.cc:2272] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.038700 12409 tablet_replica.cc:333] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.039474 12409 raft_consensus.cc:2243] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.040319 12409 raft_consensus.cc:2272] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.043354 12409 tablet_replica.cc:333] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
W20260709 12:59:41.043968 19981 consensus_peers.cc:597] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.044152 12409 raft_consensus.cc:2243] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.046505 12409 raft_consensus.cc:2272] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.049726 12409 tablet_replica.cc:333] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.050498 12409 raft_consensus.cc:2243] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.051155 12409 raft_consensus.cc:2272] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:41.053719 19981 consensus_peers.cc:597] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.054788 12409 tablet_replica.cc:333] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.055593 12409 raft_consensus.cc:2243] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:41.056870 12409 raft_consensus.cc:2272] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:41.059816 20129 consensus_peers.cc:597] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.060201 12409 tablet_replica.cc:333] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.061092 12409 raft_consensus.cc:2243] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.061654 12409 raft_consensus.cc:2272] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.064775 12409 tablet_replica.cc:333] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.065852 12409 raft_consensus.cc:2243] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.066828 12409 raft_consensus.cc:2272] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.070202 12409 tablet_replica.cc:333] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.070931 12409 raft_consensus.cc:2243] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.071538 12409 raft_consensus.cc:2272] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.074640 12409 tablet_replica.cc:333] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.075557 12409 raft_consensus.cc:2243] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.076290 12409 raft_consensus.cc:2272] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:41.078329 20129 consensus_peers.cc:597] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.079190 12409 tablet_replica.cc:333] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: stopping tablet replica
I20260709 12:59:41.080220 12409 raft_consensus.cc:2243] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.080797 12409 raft_consensus.cc:2272] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:41.162118 20055 consensus_peers.cc:597] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:41.164132 20129 consensus_peers.cc:597] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:41.166533 20055 consensus_peers.cc:597] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:41.168349 20055 consensus_peers.cc:597] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:41.182888 19981 consensus_peers.cc:597] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:41.185850 20055 consensus_peers.cc:597] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.191131 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
W20260709 12:59:41.230866 19981 consensus_peers.cc:597] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:41.272735 20055 consensus_peers.cc:597] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:41.316756 20129 consensus_peers.cc:597] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:41.388592 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:41.407065 20733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:41.440711 20734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:41.473297 20736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:41.476583 12409 server_base.cc:1061] running on GCE node
I20260709 12:59:41.482565 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:41.482918 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:41.483150 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783601981483131 us; error 0 us; skew 500 ppm
I20260709 12:59:41.483973 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:41.518024 12409 webserver.cc:533] Webserver started at http://127.12.30.65:38037/ using document root <none> and password file <none>
I20260709 12:59:41.518798 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:41.519055 12409 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:59:41.525497 12409 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:59:41.556188 20741 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:41.557703 12409 fs_manager.cc:730] Time spent opening block manager: real 0.025s	user 0.011s	sys 0.012s
I20260709 12:59:41.558578 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "e5eb27db496b47129df3e689e546d94a"
format_stamp: "Formatted at 2026-07-09 12:59:27 on dist-test-slave-mlt7"
I20260709 12:59:41.561443 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:41.598340 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:41.599794 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:41.601590 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:41.607055 20748 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:59:41.902401 20732 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:41.903115 20732 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:41.905951 20732 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:41.914273 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:41.920176 19981 leader_election.cc:336] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:41.920727 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:41.921515 20732 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:41.947288 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:59:41.947779 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.342s	user 0.008s	sys 0.000s
I20260709 12:59:41.948172 12409 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:59:41.953380 20721 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:41.953997 20721 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:41.956212 20721 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:41.956980 20732 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:41.957595 20732 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:41.958411 20748 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:41.961261 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:41.962509 20732 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:41.965615 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:41.970952 20723 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:41.971632 20723 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:41.974691 20723 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:41.975741 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 12:59:41.983630 20129 leader_election.cc:336] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:41.984257 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 12:59:41.984357 19981 leader_election.cc:336] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:41.985157 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:41.985659 20721 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:59:41.988431 20732 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:41.998667 20721 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:41.999413 20721 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.008118 20748 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.008742 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.009232 20748 tablet_bootstrap.cc:492] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:42.012610 20748 ts_tablet_manager.cc:1403] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.006s
I20260709 12:59:42.015655 20652 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.016234 20748 raft_consensus.cc:359] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.016474 20652 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.017158 20748 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:42.018486 20748 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.009617 20721 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 12:59:42.010205 19981 leader_election.cc:336] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.025382 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.032665 20723 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:59:42.034540 20055 leader_election.cc:336] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.036021 20732 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.036594 20652 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.036713 20732 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.039256 20732 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:42.040091 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:42.045003 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:42.047379 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.049110 20652 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:42.052397 20652 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.054623 20652 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:42.055981 20129 leader_election.cc:336] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.056736 20129 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 12:59:42.063695 20055 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.065179 20721 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:42.069206 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:42.080477 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.081355 20652 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 12:59:42.084515 19981 leader_election.cc:336] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.085074 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.085902 20649 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:42.087234 20748 ts_tablet_manager.cc:1434] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.074s	user 0.024s	sys 0.029s
I20260709 12:59:42.088581 20748 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:42.090884 20732 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:42.092567 20721 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.093231 20721 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.093816 20652 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.094365 20652 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.097078 20721 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.098487 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:42.103381 20055 leader_election.cc:336] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.107061 20652 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.108647 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.109740 20721 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.110349 20721 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.111076 20748 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.112248 20748 tablet_bootstrap.cc:492] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:42.113171 20748 ts_tablet_manager.cc:1403] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.008s
I20260709 12:59:42.114075 20721 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.114511 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.115235 20009 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:42.116128 20748 raft_consensus.cc:359] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.116791 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.118049 20721 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.118412 20721 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.118777 20721 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.117123 20748 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:42.122109 20748 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.111056 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.125835 20748 ts_tablet_manager.cc:1434] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:59:42.126783 20748 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:42.126845 20652 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:59:42.123962 20129 leader_election.cc:336] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.127557 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.127640 20721 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.128527 20704 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:59:42.131652 20721 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 12:59:42.133656 20129 leader_election.cc:336] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.134603 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:42.135429 20009 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:42.137868 20129 leader_election.cc:336] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.145385 20009 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:42.154387 20721 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.155195 20721 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.157546 20721 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.164351 20721 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.164912 20721 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.167054 20721 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.167124 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:42.170200 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:42.170692 20129 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.170817 20086 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:42.173272 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.173998 20704 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:42.175048 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.175752 20704 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.175951 20748 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.176141 20704 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.176612 20704 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.176666 20652 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.177007 20748 tablet_bootstrap.cc:492] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:42.177312 20652 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.177771 20748 ts_tablet_manager.cc:1403] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.007s
I20260709 12:59:42.182276 20652 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 12:59:42.184345 20055 leader_election.cc:336] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.186180 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.187969 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.189741 20748 raft_consensus.cc:359] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.190462 20748 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:42.191150 20748 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.188828 20704 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.193130 20748 ts_tablet_manager.cc:1434] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:59:42.193140 20652 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:42.193943 20748 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:42.194677 20129 leader_election.cc:336] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.194900 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.196090 20721 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.197086 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:42.197804 20086 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.193796 20704 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.199265 20720 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.199820 20720 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.207505 20720 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.210117 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:42.210803 20087 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:42.213270 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.215574 20753 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.216205 20753 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.216581 20753 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.215148 20721 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:42.218974 20721 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:42.220956 20129 leader_election.cc:336] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.229182 20129 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.229243 20086 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:42.243526 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.244380 20704 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.245064 20704 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:42.245949 20704 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.251457 20748 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.251906 20723 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.253109 20652 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.253229 20723 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.253834 20652 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.256135 20652 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.258165 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.259001 20009 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:42.260457 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:42.263562 20649 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.264322 20649 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.262820 19840 catalog_manager.cc:5697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:42.264696 20649 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:42.270571 20055 leader_election.cc:336] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.278656 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:42.279294 20723 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.279645 20160 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:42.283368 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.284610 20723 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.285177 20723 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.285596 20723 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.286475 20748 tablet_bootstrap.cc:492] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:42.287963 20748 ts_tablet_manager.cc:1403] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.094s	user 0.029s	sys 0.004s
I20260709 12:59:42.290270 20649 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.283353 20753 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:42.292284 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 48843.4us
I20260709 12:59:42.294749 20753 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 12:59:42.295182 19981 leader_election.cc:336] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.298913 20649 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.296823 20748 raft_consensus.cc:359] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.299253 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:42.301288 20009 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:42.303385 20055 leader_election.cc:336] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.313295 20704 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.314043 20704 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.314865 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:42.315604 20086 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.317651 20009 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 12:59:42.317797 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.300729 20748 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:42.321461 20704 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.311362 20723 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.322389 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.322501 20748 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.324728 20748 ts_tablet_manager.cc:1434] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:59:42.325631 20649 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.326217 20649 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:42.327275 20704 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.327942 20704 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.332652 20086 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:42.335632 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.342656 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:42.342929 20013 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
W20260709 12:59:42.343104 20129 leader_election.cc:336] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.343329 20160 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.338632 20704 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.343580 20129 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.339618 20649 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:42.344810 20129 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.347579 20704 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:59:42.349543 19981 leader_election.cc:336] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.342185 20723 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.325691 20748 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:42.351472 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.352963 20704 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.353559 20704 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.353946 20704 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.372694 19839 catalog_manager.cc:5697] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:42.374220 20160 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:42.375730 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.376760 20732 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.377287 20732 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:42.377511 20547 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.378067 20547 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.378297 20732 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.380363 20547 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:42.382833 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:42.383472 20090 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:42.372977 20704 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.390882 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:42.391534 20013 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:42.393484 19981 leader_election.cc:336] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.394395 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.396201 20723 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.396318 20652 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.396644 20723 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.397101 20723 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.397017 20652 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.399295 20652 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.399948 20730 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.400581 20730 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.401052 20013 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:42.403280 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.403358 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:42.404232 20730 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.405844 20721 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.406548 20721 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:42.404754 20704 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.404862 20160 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
W20260709 12:59:42.382233 20129 leader_election.cc:336] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.411058 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:42.407536 20721 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.411540 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.413677 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.416167 20730 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:59:42.416775 20730 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.412142 20013 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:42.420665 20723 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.417164 20730 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:59:42.438226 20055 leader_election.cc:336] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.439145 20055 leader_election.cc:336] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.413767 20753 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.447098 20056 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 12:59:42.447801 20129 leader_election.cc:336] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.448305 20649 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.448727 20649 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.449148 20649 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.449407 20763 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.449900 20763 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.463006 19839 catalog_manager.cc:5697] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:42.470414 19840 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 2 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:42.478561 20649 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.492071 20649 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:42.494899 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:42.495628 20013 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.499562 20748 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.503104 20013 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 12:59:42.512560 20748 tablet_bootstrap.cc:492] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:42.513664 20748 ts_tablet_manager.cc:1403] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.188s	user 0.017s	sys 0.006s
I20260709 12:59:42.519002 20723 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:42.520602 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:42.521270 20086 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.527258 20086 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:42.533442 20730 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.532454 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:42.536098 20730 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.536350 20652 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.539765 20652 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.541915 20652 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.543046 20763 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.536948 20765 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.543824 20765 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:42.544917 20765 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.505489 20753 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:42.537205 20771 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.535756 20056 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:42.556324 20730 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.556921 20730 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:42.557878 20730 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.560719 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:42.561585 20160 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:42.563736 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:42.564232 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:42.564462 20160 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 12:59:42.564862 20161 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:42.567329 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.550197 20771 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.568150 20649 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.568946 20649 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.569326 20649 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.570083 20771 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.536217 19835 catalog_manager.cc:5697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:42.572301 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.538650 20748 raft_consensus.cc:359] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.580228 20748 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:42.581280 20748 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.585179 20748 ts_tablet_manager.cc:1434] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.001s
I20260709 12:59:42.587437 20732 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:59:42.587662 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:42.588058 20732 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.588450 20732 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:42.614604 20748 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:42.614749 20547 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.602300 20753 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.618178 20547 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.622228 20649 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.619445 20161 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 12:59:42.616137 20164 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:42.630332 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.631670 20773 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:42.632536 20773 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:42.633894 20773 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:42.647058 20649 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.652456 20547 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:42.653196 20732 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.655830 20732 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 12:59:42.664170 20055 leader_election.cc:336] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.664798 20055 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.665613 20055 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.668920 20055 leader_election.cc:336] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.683522 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 93641.8us
W20260709 12:59:42.688524 20129 consensus_peers.cc:597] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:42.696271 19981 leader_election.cc:336] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.701200 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 70989.7us
W20260709 12:59:42.707988 19981 leader_election.cc:336] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:42.718912 19981 leader_election.cc:336] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.722155 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:42.723171 20160 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.732518 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:42.733711 20090 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:42.735015 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:42.735833 20164 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:42.745528 20164 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 12:59:42.748885 19835 catalog_manager.cc:5697] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 2 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:42.752348 19981 leader_election.cc:336] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.753257 20160 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
W20260709 12:59:42.757371 19981 leader_election.cc:336] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.772785 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.772286 19839 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:42.773583 20773 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.774489 20773 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:42.770982 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.775471 20773 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:42.780097 20129 consensus_peers.cc:597] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.783445 20779 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.783987 20779 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.784349 20779 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.801622 20779 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.806660 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.809424 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.810638 20783 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:42.811185 20783 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:42.807129 19839 catalog_manager.cc:5697] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 2 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:42.812541 20055 consensus_peers.cc:597] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.812201 20783 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.817704 20011 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:42.830114 20753 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.830438 20017 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:59:42.830376 20129 consensus_peers.cc:597] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.837388 20753 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.846712 20765 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:42.847363 20765 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:42.848063 20765 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.858368 20013 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:42.860766 20780 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:42.864004 20086 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:42.866721 20782 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:42.869268 20765 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:42.881069 20748 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.884615 20748 tablet_bootstrap.cc:492] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:42.886458 20748 ts_tablet_manager.cc:1403] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.301s	user 0.029s	sys 0.003s
I20260709 12:59:42.889698 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:42.890631 20164 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.897713 19839 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 2 to 3, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:42.909783 19981 leader_election.cc:336] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 12:59:42.904827 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:42.918251 20086 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:42.918640 19839 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:42.889325 20765 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:42.909761 20748 raft_consensus.cc:359] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:42.924598 20748 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:42.915361 20779 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:42.903241 20164 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:42.926362 20748 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:42.931325 20790 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:59:42.934276 20748 ts_tablet_manager.cc:1434] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.001s
I20260709 12:59:42.929418 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:42.936924 20748 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:42.930738 20086 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:42.943826 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 12:59:42.945079 20783 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.948621 12409 ts_tablet_manager.cc:600] Registering tablets (38/45 complete)
I20260709 12:59:42.941145 19835 catalog_manager.cc:5697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 3, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
W20260709 12:59:42.988719 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44958.1us
I20260709 12:59:42.993427 20777 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:42.988143 20783 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:43.004330 20777 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:43.004065 20783 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:43.005287 20777 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:43.031560 19981 leader_election.cc:336] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 12:59:43.065886 20758 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:59:43.066905 20055 consensus_peers.cc:597] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:43.070114 20164 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:43.075773 20765 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:59:43.108670 20011 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:43.118578 12409 ts_tablet_manager.cc:616] Registered 45 tablets
W20260709 12:59:43.118922 19981 consensus_peers.cc:597] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:43.119057 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 1.171s	user 0.344s	sys 0.000s
I20260709 12:59:43.128464 20652 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:59:43.151042 20086 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:59:43.153973 19981 consensus_peers.cc:597] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:43.149137 20055 consensus_peers.cc:597] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.158319 20161 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:59:43.158842 19981 consensus_peers.cc:597] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.161468 20730 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.186195 20748 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:43.162882 20777 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:59:43.197510 20748 tablet_bootstrap.cc:492] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:43.199055 20748 ts_tablet_manager.cc:1403] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.264s	user 0.035s	sys 0.043s
I20260709 12:59:43.202051 20748 raft_consensus.cc:359] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:43.203444 20748 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:43.207204 20748 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:43.210156 20748 ts_tablet_manager.cc:1434] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:59:43.211591 20748 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:43.214279 20129 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.220072 20129 consensus_peers.cc:597] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:43.229164 19835 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 2 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:43.234494 20086 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:43.242295 20781 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:43.240453 19835 catalog_manager.cc:5697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 2, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 2 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:43.245030 19981 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.279853 20164 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:43.297904 20652 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:43.309684 20055 consensus_peers.cc:597] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:43.320360 20160 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:59:43.327997 20723 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:59:43.355268 19981 consensus_peers.cc:597] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:43.367748 20055 consensus_peers.cc:597] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:43.390852 20055 consensus_peers.cc:597] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.404999 20055 consensus_peers.cc:597] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.405097 20748 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:59:43.408984 19981 consensus_peers.cc:597] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.409632 19981 consensus_peers.cc:597] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.412057 20748 tablet_bootstrap.cc:492] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:43.416496 20748 ts_tablet_manager.cc:1403] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.206s	user 0.011s	sys 0.067s
W20260709 12:59:43.420185 20055 consensus_peers.cc:597] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.423333 20748 raft_consensus.cc:359] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.424219 20748 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:43.429931 20748 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.432802 20748 ts_tablet_manager.cc:1434] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:59:43.433720 20748 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:43.449749 19981 consensus_peers.cc:597] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:43.450232 19981 consensus_peers.cc:597] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:43.469897 20129 consensus_peers.cc:597] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.491822 20748 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:43.493229 20748 tablet_bootstrap.cc:492] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:43.494067 20748 ts_tablet_manager.cc:1403] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.061s	user 0.028s	sys 0.007s
I20260709 12:59:43.497056 20748 raft_consensus.cc:359] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.497903 20748 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:43.498978 20748 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.502393 20090 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:43.504597 20748 ts_tablet_manager.cc:1434] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:59:43.505738 20748 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:43.509233 20160 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:43.510564 20723 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:43.512703 20723 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:43.538956 20129 consensus_peers.cc:597] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.549583 20750 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:43.550464 20129 consensus_peers.cc:597] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.551659 20055 consensus_peers.cc:597] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.550420 20750 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:43.559747 19981 consensus_peers.cc:597] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.578601 20055 consensus_peers.cc:597] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.579592 20750 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 12:59:43.583664 19981 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.642374 20129 consensus_peers.cc:597] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.644718 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 12:59:43.644685 20129 consensus_peers.cc:597] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.703596 20750 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:43.704221 20750 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:43.705823 19981 consensus_peers.cc:597] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.708060 20750 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:43.713694 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:43.749560 20748 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:43.750905 20748 tablet_bootstrap.cc:492] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:43.758855 20748 ts_tablet_manager.cc:1403] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.253s	user 0.038s	sys 0.053s
I20260709 12:59:43.761966 20748 raft_consensus.cc:359] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.762753 20748 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
W20260709 12:59:43.765386 19981 consensus_peers.cc:597] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.766110 19981 consensus_peers.cc:597] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.769439 19981 consensus_peers.cc:597] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.770030 19981 consensus_peers.cc:597] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.769606 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:43.771162 20748 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.773803 20748 ts_tablet_manager.cc:1434] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:59:43.773561 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:43.775626 20748 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:43.776054 20744 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:43.778100 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 12:59:43.779501 20055 consensus_peers.cc:597] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.780198 20750 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:43.781262 20750 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:59:43.806254 20129 consensus_peers.cc:597] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.814206 20748 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:43.817446 20748 tablet_bootstrap.cc:492] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:43.818159 20748 ts_tablet_manager.cc:1403] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.005s
W20260709 12:59:43.819151 20129 consensus_peers.cc:597] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.821225 20748 raft_consensus.cc:359] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.822046 20748 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:43.825978 20750 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:43.826565 20750 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 12:59:43.827857 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 49897.8us
I20260709 12:59:43.830197 20750 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:43.830596 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:43.825433 20748 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.836859 20748 ts_tablet_manager.cc:1434] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:59:43.838725 20748 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:43.843327 20129 consensus_peers.cc:597] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.862100 19981 consensus_peers.cc:597] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.888240 20129 consensus_peers.cc:597] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:43.908226 20748 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:43.910411 20748 tablet_bootstrap.cc:492] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:43.911546 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:43.911852 20748 ts_tablet_manager.cc:1403] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.074s	user 0.029s	sys 0.020s
I20260709 12:59:43.914004 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:43.915226 20750 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:43.915831 20748 raft_consensus.cc:359] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:43.916913 20748 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:43.917873 20748 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:43.921042 20748 ts_tablet_manager.cc:1434] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:43.922214 20748 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:43.934449 20750 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:43.935312 20750 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:43.940227 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:43.942500 20750 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:43.945187 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:43.948741 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:43.951184 20750 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:59:43.974963 20129 consensus_peers.cc:597] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:43.982180 19981 consensus_peers.cc:597] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:59:44.000788 20055 consensus_peers.cc:597] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:44.006688 20748 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.007773 20748 tablet_bootstrap.cc:492] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.008455 20748 ts_tablet_manager.cc:1403] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.087s	user 0.032s	sys 0.014s
I20260709 12:59:44.011359 20748 raft_consensus.cc:359] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.012151 20748 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.012828 20748 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.015341 20748 ts_tablet_manager.cc:1434] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:44.016431 20748 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.018095 20750 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:44.021350 20750 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:44.025556 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:44.044936 20750 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:44.048492 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:44.055650 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:44.056691 20750 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:44.107065 20750 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:44.107788 20750 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.111954 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:44.119381 20750 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:44.125849 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:44.133431 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 12:59:44.135141 20750 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:44.184976 20748 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.186918 20748 tablet_bootstrap.cc:492] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.187657 20748 ts_tablet_manager.cc:1403] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.172s	user 0.020s	sys 0.004s
I20260709 12:59:44.190729 20748 raft_consensus.cc:359] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.191761 20748 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.193157 20748 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.196650 20748 ts_tablet_manager.cc:1434] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:44.198177 20748 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.231076 20748 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.232425 20748 tablet_bootstrap.cc:492] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.233296 20748 ts_tablet_manager.cc:1403] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.008s
I20260709 12:59:44.237733 20748 raft_consensus.cc:359] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:44.238667 20748 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.240624 20748 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:44.243547 20748 ts_tablet_manager.cc:1434] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 12:59:44.244692 20748 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.273418 20748 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.274791 20748 tablet_bootstrap.cc:492] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.275933 20748 ts_tablet_manager.cc:1403] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.032s	user 0.021s	sys 0.003s
I20260709 12:59:44.278858 20748 raft_consensus.cc:359] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.279838 20748 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.280958 20748 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.284042 20748 ts_tablet_manager.cc:1434] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:44.284865 20748 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.315349 20748 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.317088 20748 tablet_bootstrap.cc:492] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.318065 20748 ts_tablet_manager.cc:1403] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.007s
I20260709 12:59:44.321836 20748 raft_consensus.cc:359] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:44.322832 20748 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.323894 20748 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:44.327659 20748 ts_tablet_manager.cc:1434] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:59:44.328732 20748 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.370790 20748 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.392966 20748 tablet_bootstrap.cc:492] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.404019 20748 ts_tablet_manager.cc:1403] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.076s	user 0.021s	sys 0.001s
I20260709 12:59:44.405484 20888 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:38607 every 8 connection(s)
I20260709 12:59:44.407519 20748 raft_consensus.cc:359] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.408409 20748 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.409322 20748 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.411818 20748 ts_tablet_manager.cc:1434] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:44.412228 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:38607
I20260709 12:59:44.412909 20748 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.504087 20858 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:44.513885 20748 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.515193 20748 tablet_bootstrap.cc:492] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.530800 20748 ts_tablet_manager.cc:1403] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.118s	user 0.022s	sys 0.017s
I20260709 12:59:44.534179 20748 raft_consensus.cc:359] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.536211 20748 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.541356 20748 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.550719 20748 ts_tablet_manager.cc:1434] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:59:44.551756 20748 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.563700 20856 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:44.597796 20859 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:44.620198 20890 heartbeater.cc:344] Connected to a master server at 127.12.30.126:46523
I20260709 12:59:44.620738 20890 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:44.621734 20890 heartbeater.cc:507] Master 127.12.30.126:46523 requested a full tablet report, sending...
I20260709 12:59:44.644848 20748 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.645942 20748 tablet_bootstrap.cc:492] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.646667 20748 ts_tablet_manager.cc:1403] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.095s	user 0.008s	sys 0.007s
I20260709 12:59:44.663501 20748 raft_consensus.cc:359] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.664505 20748 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.665076 20748 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.666663 20748 ts_tablet_manager.cc:1434] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:59:44.667421 20748 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.687661 19839 ts_manager.cc:194] Re-registered known tserver with Master: e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:44.729930 20748 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.735865 20748 tablet_bootstrap.cc:492] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.736658 20748 ts_tablet_manager.cc:1403] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.069s	user 0.023s	sys 0.005s
I20260709 12:59:44.739459 20748 raft_consensus.cc:359] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:44.740193 20748 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.740902 20748 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:44.742971 20748 ts_tablet_manager.cc:1434] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:44.743847 20748 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.748904 19839 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35376
I20260709 12:59:44.755513 20890 heartbeater.cc:499] Master 127.12.30.126:46523 was elected leader, sending a full tablet report...
I20260709 12:59:44.761129 20856 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:44.780037 20859 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:44.806099 20748 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.806861 20748 tablet_bootstrap.cc:492] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.807349 20748 ts_tablet_manager.cc:1403] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.064s	user 0.029s	sys 0.010s
I20260709 12:59:44.809929 20748 raft_consensus.cc:359] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.810575 20748 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.811199 20748 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.812896 20748 ts_tablet_manager.cc:1434] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:59:44.813539 20748 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.834064 20859 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:44.835000 20748 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.836087 20748 tablet_bootstrap.cc:492] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.836849 20748 ts_tablet_manager.cc:1403] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.006s
I20260709 12:59:44.839701 20748 raft_consensus.cc:359] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.840464 20748 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.841223 20748 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.842967 20748 ts_tablet_manager.cc:1434] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:59:44.843837 20748 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.939227 20748 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.940546 20748 tablet_bootstrap.cc:492] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.941217 20748 ts_tablet_manager.cc:1403] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.098s	user 0.024s	sys 0.022s
I20260709 12:59:44.946169 20748 raft_consensus.cc:359] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.946976 20748 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.953092 20748 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:44.956110 20748 ts_tablet_manager.cc:1434] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:59:44.957901 20748 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:44.985821 20748 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:44.987419 20748 tablet_bootstrap.cc:492] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:44.988256 20748 ts_tablet_manager.cc:1403] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.008s
I20260709 12:59:44.991011 20748 raft_consensus.cc:359] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.991734 20748 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:44.992559 20748 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:44.994725 20748 ts_tablet_manager.cc:1434] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:44.995534 20748 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.019644 20748 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.021013 20748 tablet_bootstrap.cc:492] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.021987 20748 ts_tablet_manager.cc:1403] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.006s
I20260709 12:59:45.025033 20748 raft_consensus.cc:359] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.025812 20748 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.026671 20748 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.032544 20748 ts_tablet_manager.cc:1434] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:59:45.033500 20748 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.054250 20748 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.055410 20748 tablet_bootstrap.cc:492] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.056160 20748 ts_tablet_manager.cc:1403] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.009s
I20260709 12:59:45.058976 20748 raft_consensus.cc:359] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.059724 20748 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.060626 20748 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.062963 20748 ts_tablet_manager.cc:1434] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 12:59:45.069420 20748 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.097671 20748 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.098750 20748 tablet_bootstrap.cc:492] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.099476 20748 ts_tablet_manager.cc:1403] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.001s
I20260709 12:59:45.102236 20748 raft_consensus.cc:359] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.102955 20748 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.103664 20748 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.105330 20748 ts_tablet_manager.cc:1434] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:59:45.106180 20748 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.174196 20859 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:59:45.185377 20129 consensus_peers.cc:597] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:45.192938 20856 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:45.190514 20748 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.217319 20748 tablet_bootstrap.cc:492] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.218076 20748 ts_tablet_manager.cc:1403] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.112s	user 0.037s	sys 0.016s
I20260709 12:59:45.222497 20748 raft_consensus.cc:359] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.223220 20748 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.224027 20748 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.226279 20748 ts_tablet_manager.cc:1434] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:45.227696 20748 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.247592 20748 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.248598 20748 tablet_bootstrap.cc:492] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.249305 20748 ts_tablet_manager.cc:1403] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.002s
I20260709 12:59:45.251946 20748 raft_consensus.cc:359] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.252607 20748 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.253412 20748 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.255478 20748 ts_tablet_manager.cc:1434] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:45.256261 20748 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.272727 20748 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.273679 20748 tablet_bootstrap.cc:492] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.274372 20748 ts_tablet_manager.cc:1403] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:59:45.276626 20748 raft_consensus.cc:359] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.277158 20748 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.277734 20748 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.279655 20748 ts_tablet_manager.cc:1434] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:45.281533 20748 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.321310 20748 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.322463 20748 tablet_bootstrap.cc:492] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.323241 20748 ts_tablet_manager.cc:1403] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.006s
I20260709 12:59:45.326311 20748 raft_consensus.cc:359] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.327136 20748 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.328092 20748 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.330539 20748 ts_tablet_manager.cc:1434] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:59:45.334481 20748 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.352471 20748 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.353605 20748 tablet_bootstrap.cc:492] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.354334 20748 ts_tablet_manager.cc:1403] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:59:45.356885 20748 raft_consensus.cc:359] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.357559 20748 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.358325 20748 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.360677 20748 ts_tablet_manager.cc:1434] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:45.361347 20858 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:45.361546 20748 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.382846 20748 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.383999 20748 tablet_bootstrap.cc:492] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.384792 20748 ts_tablet_manager.cc:1403] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 12:59:45.387563 20748 raft_consensus.cc:359] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.388352 20748 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
W20260709 12:59:45.388768 20055 consensus_peers.cc:597] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:59:45.389040 20748 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.391439 20748 ts_tablet_manager.cc:1434] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:45.392292 20748 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.410949 20748 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.413159 20748 tablet_bootstrap.cc:492] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.414048 20748 ts_tablet_manager.cc:1403] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.011s
I20260709 12:59:45.416584 20748 raft_consensus.cc:359] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.417270 20748 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.418010 20748 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.423374 20748 ts_tablet_manager.cc:1434] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:45.426071 20748 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.449900 20748 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.450832 20748 tablet_bootstrap.cc:492] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.451488 20748 ts_tablet_manager.cc:1403] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.000s
I20260709 12:59:45.454252 20748 raft_consensus.cc:359] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.455206 20748 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.455989 20748 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.457739 20748 ts_tablet_manager.cc:1434] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.009s	sys 0.000s
I20260709 12:59:45.468896 20748 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.519559 20858 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:59:45.528930 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44582.4us
I20260709 12:59:45.582540 20859 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:45.584210 20748 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.585641 20748 tablet_bootstrap.cc:492] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.587298 20748 ts_tablet_manager.cc:1403] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.119s	user 0.032s	sys 0.014s
I20260709 12:59:45.591073 20748 raft_consensus.cc:359] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.601398 20748 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.602376 20748 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.607357 20748 ts_tablet_manager.cc:1434] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.001s
I20260709 12:59:45.610235 20748 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.659087 20748 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.661772 20748 tablet_bootstrap.cc:492] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.662570 20748 ts_tablet_manager.cc:1403] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.053s	user 0.022s	sys 0.015s
I20260709 12:59:45.665092 20748 raft_consensus.cc:359] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.665787 20748 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.666442 20748 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.668421 20748 ts_tablet_manager.cc:1434] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:45.669265 20748 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.696146 20748 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.697185 20748 tablet_bootstrap.cc:492] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.697923 20748 ts_tablet_manager.cc:1403] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.008s
W20260709 12:59:45.702032 20129 consensus_peers.cc:597] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:59:45.705701 20859 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:45.707772 20748 raft_consensus.cc:359] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.708537 20748 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.710399 20748 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.712611 20748 ts_tablet_manager.cc:1434] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:59:45.716116 20748 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
W20260709 12:59:45.738569 19981 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:59:45.826889 20748 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.828202 20748 tablet_bootstrap.cc:492] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.830549 20748 ts_tablet_manager.cc:1403] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.115s	user 0.020s	sys 0.013s
I20260709 12:59:45.833539 20748 raft_consensus.cc:359] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.834271 20748 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.834972 20748 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:45.837157 20748 ts_tablet_manager.cc:1434] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:59:45.838065 20748 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.861887 20859 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:45.900180 20748 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.901274 20748 tablet_bootstrap.cc:492] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.901957 20748 ts_tablet_manager.cc:1403] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.064s	user 0.028s	sys 0.020s
I20260709 12:59:45.904717 20748 raft_consensus.cc:359] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.905452 20748 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.906286 20748 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:45.908983 20748 ts_tablet_manager.cc:1434] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:59:45.910215 20748 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.934989 20748 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.936095 20748 tablet_bootstrap.cc:492] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.941031 20748 ts_tablet_manager.cc:1403] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.006s
I20260709 12:59:45.943058 20748 raft_consensus.cc:359] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.943589 20748 raft_consensus.cc:740] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.944232 20748 consensus_queue.cc:260] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.946676 20748 ts_tablet_manager.cc:1434] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:59:45.947672 20748 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:45.973973 20748 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:45.974920 20748 tablet_bootstrap.cc:492] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:45.975608 20748 ts_tablet_manager.cc:1403] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 12:59:45.978144 20748 raft_consensus.cc:359] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.978811 20748 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:45.979491 20748 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:45.981564 20748 ts_tablet_manager.cc:1434] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:59:45.982360 20748 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:46.021592 20748 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.022719 20748 tablet_bootstrap.cc:492] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:46.023542 20748 ts_tablet_manager.cc:1403] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.000s
I20260709 12:59:46.026405 20748 raft_consensus.cc:359] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:46.027184 20748 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:46.028014 20748 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:46.030494 20748 ts_tablet_manager.cc:1434] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:46.032094 20748 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap starting.
I20260709 12:59:46.065871 20748 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.067346 20748 tablet_bootstrap.cc:492] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Bootstrap complete.
I20260709 12:59:46.069371 20748 ts_tablet_manager.cc:1403] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.000s
I20260709 12:59:46.072817 20748 raft_consensus.cc:359] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:46.073645 20748 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Initialized, Role: FOLLOWER
I20260709 12:59:46.074553 20748 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:46.077078 20748 ts_tablet_manager.cc:1434] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
W20260709 12:59:46.827950 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 58658.9us
W20260709 12:59:47.038659 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 100673us
W20260709 12:59:47.845382 20186 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1280 after lost signal to thread 12412
W20260709 12:59:47.846827 20186 debug-util.cc:398] Leaking SignalData structure 0x7b0800282f80 after lost signal to thread 19874
W20260709 12:59:47.848331 20186 debug-util.cc:398] Leaking SignalData structure 0x7b08003234c0 after lost signal to thread 20041
W20260709 12:59:47.849658 20186 debug-util.cc:398] Leaking SignalData structure 0x7b0800370680 after lost signal to thread 20115
W20260709 12:59:47.853108 20186 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2120 after lost signal to thread 20189
W20260709 12:59:47.854699 20186 debug-util.cc:398] Leaking SignalData structure 0x7b080025e8e0 after lost signal to thread 20888
W20260709 12:59:48.238451 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 76817.9us
W20260709 12:59:50.505568 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 92622.9us
I20260709 12:59:54.494109 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:54.571489 20160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 12:59:54.572594 20160 raft_consensus.cc:606] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:54.590041 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 12:59:54.591265 20017 raft_consensus.cc:606] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:54.594673 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746d924541574d6086a6ae086476a1e7"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 12:59:54.595234 20017 raft_consensus.cc:606] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:54.598485 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 12:59:54.599012 20017 raft_consensus.cc:606] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:54.601642 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 12:59:54.602170 20017 raft_consensus.cc:606] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:54.620253 20087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 12:59:54.620945 20087 raft_consensus.cc:606] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:54.622586 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:59:54.626096 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:54.628230 20547 raft_consensus.cc:993] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.628901 20973 raft_consensus.cc:1081] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.631197 20856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 12:59:54.631825 20856 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.632267 20856 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.647811 20856 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:54.653559 20856 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:54.657449 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:54.659056 20160 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.659287 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:54.660573 20017 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.661064 20017 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:54.662158 20017 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:54.663774 20017 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.676326 20160 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:54.678453 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:54.705037 20983 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.708120 20017 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:54.714884 20983 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:54.716805 20983 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:54.740083 19835 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:54.910025 20978 raft_consensus.cc:993] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.911432 20948 raft_consensus.cc:1081] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.914681 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44216
I20260709 12:59:54.915337 20853 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.915680 20853 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:54.925244 20853 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:54.927425 20853 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:54.928462 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:54.928994 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:54.929319 20011 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:54.929663 20090 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:54.930104 20090 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:54.931260 20090 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:54.932586 20090 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:54.944167 20011 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 12:59:54.945809 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:54.946862 20983 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:54.947420 20983 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:54.948486 20983 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:54.949115 20090 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 12:59:54.960067 20547 raft_consensus.cc:993] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.960604 20547 raft_consensus.cc:1081] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.964380 20858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 12:59:54.965066 20858 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.965421 20858 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.972040 19835 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 2 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 3 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:54.975471 20858 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:54.979274 20858 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:54.980927 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:54.981597 20090 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.986047 20962 raft_consensus.cc:993] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.986603 20962 raft_consensus.cc:1081] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:54.988417 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:54.989230 20011 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.989262 20858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 12:59:54.998752 20011 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:54.999550 20858 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.991930 20090 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.000020 20011 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:55.000047 20858 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.001579 20011 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.008809 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:55.009505 20858 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.009860 20983 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.011198 20011 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.012385 20983 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:55.013615 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:55.013569 20983 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:55.014273 20090 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.014365 20858 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:55.015755 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:55.016430 20165 raft_consensus.cc:3055] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.016711 20165 raft_consensus.cc:740] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:55.018044 20165 consensus_queue.cc:260] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.019469 20165 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.023468 20090 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.026109 20743 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:55.027498 20984 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.029258 20984 raft_consensus.cc:697] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:55.030261 20984 consensus_queue.cc:237] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.031862 20165 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.041618 19835 catalog_manager.cc:5697] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.047744 20973 raft_consensus.cc:993] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.048467 20990 raft_consensus.cc:1081] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.050145 20858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "746d924541574d6086a6ae086476a1e7"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 12:59:55.050632 20858 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.050921 20858 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.057612 20858 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:55.059651 20858 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:55.061174 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:55.061728 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:55.062362 20090 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.061890 20011 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.063288 20011 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:55.066011 20973 raft_consensus.cc:993] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.064814 20011 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:55.066565 20973 raft_consensus.cc:1081] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.067353 20011 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.066390 19835 catalog_manager.cc:5697] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:55.068593 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 12:59:55.069197 20853 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.069720 20853 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:55.075112 20090 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.079372 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:55.080960 20984 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.081535 20984 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:55.082605 20984 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:55.084069 20853 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:55.086262 20853 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:55.087675 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:55.087826 20011 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.088060 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:55.088461 20165 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:55.088696 20017 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:55.089046 20017 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:55.089931 20017 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:55.091773 20017 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:55.098284 20165 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 12:59:55.103708 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:55.105355 20984 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:55.105752 20017 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 12:59:55.106168 20984 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:55.107528 19835 catalog_manager.cc:5697] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.109261 20984 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:55.132238 19839 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 2 to 3, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 3 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.275697 20011 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.278327 20984 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:55.296602 20165 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.301118 20984 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:59:55.379053 20011 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:55.380636 20984 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.391296 20087 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:55.393628 20984 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.418905 20090 raft_consensus.cc:1275] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.420595 20984 consensus_queue.cc:1048] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.437768 20160 raft_consensus.cc:1275] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.440119 20983 consensus_queue.cc:1048] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.538370 20087 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.539855 20983 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.572623 20011 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.574731 20984 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.604219 20011 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.610409 20984 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.612090 20011 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:55.622097 20983 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:55.627701 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:55.629853 20087 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.637576 21008 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:55.653088 20160 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:55.655033 20984 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:55.721782 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:55.771137 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 12:59:55.772187 20011 raft_consensus.cc:606] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:55.776373 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 12:59:55.777220 20011 raft_consensus.cc:606] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:55.780500 20165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 12:59:55.781322 20165 raft_consensus.cc:606] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:55.785211 20160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:55614
I20260709 12:59:55.785815 20160 raft_consensus.cc:606] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:55.788422 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 12:59:55.788949 20011 raft_consensus.cc:606] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:55.790096 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:55.791857 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:55.814469 20990 raft_consensus.cc:993] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.815070 20973 raft_consensus.cc:1081] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.816617 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 12:59:55.817165 20853 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.817487 20853 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.823753 20853 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.826977 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:55.828088 20011 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.828598 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:55.829439 20011 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:55.829777 20160 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.827279 20853 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:55.831156 20011 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.832278 20011 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.837658 20160 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.839128 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:55.839946 20011 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.840216 20984 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.840795 20984 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:55.841810 20984 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.853526 19835 catalog_manager.cc:5697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:55.888833 20977 raft_consensus.cc:993] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 12:59:55.889331 20977 raft_consensus.cc:1081] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 12:59:55.891300 20090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36126
I20260709 12:59:55.892007 20090 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.892338 20090 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.897926 20090 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:55.900286 20090 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:55.902379 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:55.903340 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:55.903946 20160 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.904258 20160 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:55.904358 20011 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.905182 20160 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:55.906653 20160 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.913759 20011 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 12:59:55.915334 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:55.915531 20160 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 12:59:55.916216 20978 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.916842 20978 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:55.917969 20978 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:55.930718 19835 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.941661 20774 raft_consensus.cc:993] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.942162 20977 raft_consensus.cc:1081] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:55.943960 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 12:59:55.944507 20853 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.944871 20853 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.952942 20853 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.955202 20853 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:55.956807 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:55.957473 20165 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.957746 20165 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:55.958518 20165 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.959661 20165 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.962062 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:55.962772 20090 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.981727 20090 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:55.985236 20743 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:55.986322 20993 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.986824 20993 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:55.987794 20993 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:55.985450 20165 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:56.015599 19838 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:56.105572 20547 raft_consensus.cc:993] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:56.106042 20547 raft_consensus.cc:1081] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:56.108048 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 12:59:56.108554 20853 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.108837 20853 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.115681 20853 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:56.118054 20853 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:56.119163 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:56.119081 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:56.119948 20090 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.120020 20013 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.120440 20013 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:56.121254 20013 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:56.122645 20013 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.127822 20090 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:56.129295 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:56.130398 20993 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.130409 20013 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:56.131862 20993 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:56.133033 20993 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:56.145066 19838 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.168080 20973 raft_consensus.cc:993] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:56.168627 20547 raft_consensus.cc:1081] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:56.170341 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 12:59:56.171128 20853 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.171478 20853 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:56.176986 20853 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:56.178660 20853 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:56.179600 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:56.180063 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:56.180341 20165 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:56.180900 20013 raft_consensus.cc:3055] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:56.182433 20013 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:56.185560 20013 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:56.186916 20013 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:56.190052 20165 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 12:59:56.191289 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:56.192239 20993 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:56.193446 20993 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:56.194453 20993 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:56.195214 20013 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 12:59:56.208393 19838 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 2 to 3, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 3 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.268930 20165 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.270836 20993 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.282331 20013 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.283877 20993 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.364825 20158 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.366711 20948 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.385795 20011 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.387526 20948 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.392328 20090 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.394017 20993 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.410810 20165 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.412319 20984 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.519316 20013 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.521037 20984 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.533427 20090 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.535851 21008 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:56.581837 20013 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:56.583123 21008 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:56.592278 20158 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:56.594219 20984 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:56.792904 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:56.841856 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:56.901747 20090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:58322
I20260709 12:59:56.902668 20090 raft_consensus.cc:606] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:56.906111 20158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:55614
I20260709 12:59:56.906844 20158 raft_consensus.cc:606] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 12:59:56.910192 20158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 12:59:56.910882 20158 raft_consensus.cc:606] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:56.913790 20158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d2ab233d0664594955baf948686386b"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 12:59:56.914381 20158 raft_consensus.cc:606] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 12:59:56.916077 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:56.918834 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:57.012957 20774 raft_consensus.cc:993] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:57.023689 20977 raft_consensus.cc:1081] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:57.027335 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d2ab233d0664594955baf948686386b"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 12:59:57.028106 20853 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.028481 20853 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.036599 20853 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:57.038456 20853 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:57.039595 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:57.040042 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:57.040318 20013 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.040799 20158 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.041213 20158 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:57.042230 20158 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:57.047840 20158 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.050892 20013 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:57.052906 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:57.054095 20984 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.054670 20984 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:57.055635 20984 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:57.064265 20158 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:57.079654 19838 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.158360 21031 raft_consensus.cc:993] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: : Instructing follower 48da76e971b84219a6ba80d1ba83b7eb to start an election
I20260709 12:59:57.159034 21031 raft_consensus.cc:1081] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Signalling peer 48da76e971b84219a6ba80d1ba83b7eb to start an election
I20260709 12:59:57.161255 20011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:34542
I20260709 12:59:57.162092 20011 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.162686 20011 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.176182 20011 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:57.179942 20011 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:57.181859 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:57.182742 20158 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.184803 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 12:59:57.185534 20087 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.185851 20087 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:57.187055 20087 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:57.189112 20087 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.191756 20158 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:57.193481 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:57.194336 21045 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.194736 20977 raft_consensus.cc:993] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:57.194823 21045 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 12:59:57.195446 20977 raft_consensus.cc:1081] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 12:59:57.196998 21045 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:57.199280 20858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 12:59:57.199995 20858 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.200366 20858 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.205873 20087 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:57.214644 20858 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:57.216686 20858 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:57.218001 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:57.217514 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:57.218894 20158 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.219072 20011 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.219533 20158 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:57.220709 20158 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:57.222302 20158 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.221554 19838 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 2, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 2 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.230289 20011 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:57.231837 20158 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:57.231946 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:57.233337 20993 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.233898 20993 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 12:59:57.234923 20993 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:57.245131 19838 catalog_manager.cc:5697] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.359360 20977 raft_consensus.cc:993] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 12:59:57.360157 21040 raft_consensus.cc:1081] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 12:59:57.362210 20087 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36126
I20260709 12:59:57.362944 20087 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.363296 20087 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.370844 20087 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:57.372848 20087 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:57.373883 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:57.374101 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 12:59:57.374783 20011 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.374815 20158 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.375375 20158 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:57.376410 20158 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:57.377912 20158 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.385305 20011 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 12:59:57.386714 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 12:59:57.387576 20158 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 12:59:57.387677 21031 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.388420 21031 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 12:59:57.389371 21031 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:57.401506 19838 catalog_manager.cc:5697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.545586 20158 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.547501 20993 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:58.011781 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:58.593683 21063 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:58.594415 21063 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:58.598850 21063 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:58.600769 21065 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:58.601362 21065 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:58.604808 21065 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:58.664237 21065 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:58.665105 21065 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:58.669042 21065 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:58.673614 21065 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:58.674444 21065 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:58.676412 21064 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:58.677348 21064 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:58.687410 21053 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:58.688331 21053 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:58.691972 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:58.694197 21053 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:58.694487 21065 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:58.694974 21053 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:58.696650 21053 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:58.699369 21064 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:58.712007 21053 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:58.760808 21064 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:58.761731 21064 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:58.764216 21053 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:58.765074 21053 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:58.780663 21008 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:58.781791 21008 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:58.816486 21063 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:58.817515 21065 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:58.818349 21065 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:58.821123 21065 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:58.817566 21063 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:58.834859 21040 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:58.986966 21040 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:58.989266 21063 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:58.989358 21040 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:58.990090 21040 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:58.990618 21040 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:58.993160 21040 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:58.882652 21053 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:58.900153 20977 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:58.994473 20977 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:58.998034 20977 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.001885 21065 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.002694 21065 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.005606 21065 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:58.947063 21062 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:59.012262 21068 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.048520 21062 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.045238 20977 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.049679 20977 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:58.980144 21064 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.051100 21062 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.048635 21068 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.052378 21064 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.052990 21064 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.055065 21064 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.056071 20977 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.056130 21064 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.057606 21064 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.059861 21064 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.053515 21068 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.061964 21067 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.062480 21067 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.064139 21068 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 12:59:59.064788 19982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.015s	sys 0.010s
I20260709 12:59:59.064878 21067 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.065351 20977 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.065939 20977 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:59.067435 19982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.016s	sys 0.011s
I20260709 12:59:59.068095 20977 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.064816 21068 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.069849 21070 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.049883 21066 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.071360 21066 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.070763 21070 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.073745 21066 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.074851 21070 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.077319 21068 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.079999 21068 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.080082 21066 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.081111 21068 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.081308 21066 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.084663 21068 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.085405 21062 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.085939 21062 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.089564 21053 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.090150 21053 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.092701 21040 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.093381 21040 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.095417 21008 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.113045 21066 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.113662 21062 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.113629 21053 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.116427 21040 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 12:59:59.120630 20054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.312s	user 0.011s	sys 0.032s
W20260709 12:59:59.121019 20054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.313s	user 0.011s	sys 0.032s
W20260709 12:59:59.122007 20055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.016s	sys 0.014s
W20260709 12:59:59.122364 20055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.016s	sys 0.014s
I20260709 12:59:59.130411 21070 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.132690 21071 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.133912 21071 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.134111 21070 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.136767 21071 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.138043 21070 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.140841 21071 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.141466 21071 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.142603 21074 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.145278 21074 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.146077 21075 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.146731 21075 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.148069 21074 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.148725 21074 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:59.149044 21075 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.149178 21074 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.151105 21072 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.143996 21071 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.153350 21074 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.157770 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.157968 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.159145 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.159282 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.159833 20010 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:59.160468 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.160759 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.161401 20010 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:59.161646 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.162328 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.162911 20010 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 12:59:59.163857 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.164048 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.164566 20015 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 12:59:59.165388 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:59.170536 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.171286 20008 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:59.151656 21072 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.173573 21072 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:59.173760 19980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.010s	sys 0.009s
I20260709 12:59:59.174180 21072 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:59:59.174296 19980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.011s	sys 0.009s
I20260709 12:59:59.175609 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.176424 21070 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.176821 20130 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.177204 21070 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.177875 21062 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:59:59.178251 21062 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.178694 21062 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.179602 21070 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.185000 21070 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.186300 21070 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.196303 21070 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:59.230615 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53837.9us
I20260709 12:59:59.235126 21063 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:59:59.235577 21063 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.236143 21063 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.245687 21063 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.255750 21031 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.277498 21071 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.279480 21074 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.280069 21070 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:59.280800 21075 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.289083 21074 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.301126 21075 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.303442 21075 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.280807 21076 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.283092 20130 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.284433 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.285965 20013 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:59.288923 21031 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.306417 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 61) took 1671 ms. Trace:
I20260709 12:59:59.306895 20013 rpcz_store.cc:276] 0709 12:59:57.634508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:59.285100 (+1650592us) service_pool.cc:224] Handling call
0709 12:59:59.285585 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:59:59.306118 (+ 20533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:59:59.306358 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:59:59.290916 21072 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.291422 20977 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.307857 20977 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.292166 21068 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.309365 21068 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.310665 20977 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.304695 21070 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.311360 20977 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.292785 21067 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.312395 21067 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.313249 21070 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.314890 21067 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.315420 21067 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.311601 21068 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.315871 21031 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.303836 21074 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.316491 21068 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.306035 21076 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.317111 21068 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.319453 21076 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.291652 21071 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.320152 21068 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.320443 21077 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.320791 21068 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:59:59.307826 21072 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.321242 21068 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.321103 21077 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.321693 21068 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:59.323074 21072 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.323487 21077 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.298094 21063 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.312194 20977 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.287410 21062 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.324263 21067 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.330619 21067 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.326035 20977 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.325843 21071 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.352052 21062 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 12:59:59.356338 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73304.7us
I20260709 12:59:59.359182 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.359270 21068 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.360164 21062 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:59:59.360601 21062 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.361521 21068 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.361915 21062 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:59.369606 21062 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.361256 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.375077 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.375459 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.375885 20008 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:59.376945 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:59.377028 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.377503 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:59.378229 20013 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.378428 21066 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.377046 21073 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.379228 21073 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.380137 21065 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.380357 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.380780 21065 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.381978 21073 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.382599 21073 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.382999 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:59.383737 20014 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.381870 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.387467 20012 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:59.377533 20009 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:59:59.392473 19981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.481s	user 0.005s	sys 0.024s
W20260709 12:59:59.394289 19981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.483s	user 0.006s	sys 0.024s
I20260709 12:59:59.396574 20009 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 12:59:59.397728 20013 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 12:59:59.398965 20165 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:59.399070 21081 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.399556 21081 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.399938 21081 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.402522 21062 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.403329 20014 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 12:59:59.408537 20130 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.409925 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.410483 21063 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:59:59.410678 21072 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.410988 21063 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.411397 21063 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.411240 21072 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.413667 21072 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.413846 21040 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:59:59.414374 21040 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.414655 20130 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.415803 21053 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:59.415943 20130 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.416399 21053 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:59.417222 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.417531 21053 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.418010 21068 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:59.419037 21068 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:59.419739 20011 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:59.379164 21066 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.420425 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 60) took 1786 ms. Trace:
I20260709 12:59:59.421104 21067 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:59.421676 21067 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:59.422052 21066 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.423084 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:59.422773 21067 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.423686 20015 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:59.383665 21008 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.427702 21008 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.420217 21068 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 12:59:59.434056 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08002caba0 after lost signal to thread 12412
W20260709 12:59:59.435313 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08002111e0 after lost signal to thread 19874
W20260709 12:59:59.436679 20038 debug-util.cc:398] Leaking SignalData structure 0x7b080037b8e0 after lost signal to thread 20041
W20260709 12:59:59.438273 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08003cafe0 after lost signal to thread 20115
I20260709 12:59:59.439114 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:59.439596 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08002cdda0 after lost signal to thread 20189
I20260709 12:59:59.383158 21065 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 12:59:59.440057 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08002b55e0 after lost signal to thread 20190
I20260709 12:59:59.440011 20014 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
W20260709 12:59:59.441570 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08001198e0 after lost signal to thread 20888
I20260709 12:59:59.442852 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.440338 20977 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:59.444375 20977 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.446933 20977 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.420821 20011 rpcz_store.cc:276] 0709 12:59:57.634165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:59.418957 (+1784792us) service_pool.cc:224] Handling call
0709 12:59:59.419282 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:59:59.420155 (+   873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:59:59.420367 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:59:59.414775 21040 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.430428 21008 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.460556 19838 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 151) took 1894 ms. Trace:
I20260709 12:59:59.460975 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.465914 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.467907 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.468655 20161 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:59.469287 21064 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.469297 21069 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.469913 21064 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.470191 21069 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.472172 21064 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.461146 19838 rpcz_store.cc:276] 0709 12:59:57.565933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:57.566108 (+   175us) service_pool.cc:224] Handling call
0709 12:59:59.460329 (+1894221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":215975,"reported_tablets":1}
I20260709 12:59:59.472636 21064 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.472972 21071 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:59.473412 21064 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.473568 21071 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.475438 21064 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.475672 21071 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.476248 20984 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.476797 20984 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.478909 20984 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.479367 20984 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.480391 21069 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.483418 21064 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.484018 21064 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.486380 21064 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.487296 21065 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.488180 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.489272 21065 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.490861 20015 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 12:59:59.491798 21008 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.492456 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.493422 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.493522 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.494966 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.495728 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.496511 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.499084 21062 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:59.500452 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.501447 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.502511 21063 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.503324 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.503883 21081 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 12:59:59.507611 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59473.6us
I20260709 12:59:59.516098 20856 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:59.383178 21073 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.491426 21065 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.531229 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.531066 21062 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.537449 21054 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.538663 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.539147 20978 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.539755 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.540843 21076 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.563630 21076 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.550242 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.553853 20130 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.555852 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.568792 20014 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:59.561175 20016 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 12:59:59.541738 20013 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 12:59:59.564603 21079 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.479789 20984 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.566234 20161 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:59.566109 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.567965 20853 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:59.574994 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.576421 21087 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.577175 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.577445 21078 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.577291 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.577296 21057 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.554116 20850 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:59.580917 21079 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.619870 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.556463 20852 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:59.617930 21084 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.621194 20158 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 12:59:59.613808 21065 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:59:59.621704 21079 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.622278 21079 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.622753 21079 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.527033 21008 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.623394 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.624071 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.562891 20978 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.624811 20165 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:59.625167 21008 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.617470 20984 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.581423 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.626319 20851 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 12:59:59.626474 20978 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.627022 20978 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.617879 20162 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:59.533129 20857 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 12:59:59.584465 21073 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.628361 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.582934 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.628476 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.629105 20855 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:59.571341 21031 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:59.630391 21079 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.630499 21031 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.585752 21061 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.632234 21061 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.621098 21084 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.586122 20013 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 12:59:59.584640 21063 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.569088 20164 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:59.635217 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.636343 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.636905 20164 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 12:59:59.577209 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.637989 20164 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 12:59:59.639261 20164 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:59.579185 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.640251 20159 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:59.641145 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.641737 20159 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:59.537569 21058 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.644173 21058 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.644663 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.641505 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.646227 20165 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:59.588511 20160 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 2: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:59.549446 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.579892 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.616394 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.616197 21087 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.571995 21054 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.656705 20859 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 12:59:59.617097 21057 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.617767 20012 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 12:59:59.613291 21067 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:59.618623 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.618722 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 12:59:59.623322 21076 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.624138 20014 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 12:59:59.582697 21080 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 12:59:59.633617 21084 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.627451 20978 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.607618 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.629132 20162 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:59.643671 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.588485 20856 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:59.720988 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.719506 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.662773 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.650151 21078 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.804419 21080 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.804929 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:59.728297 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.775964 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.806676 21078 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.807128 21080 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.713868 21057 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.726938 21087 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.660233 21029 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.712378 21061 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.654335 21075 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.801147 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.663236 21090 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.744963 21058 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.746021 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.810690 21090 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.714167 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.813382 21090 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.701974 21073 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.149s
I20260709 12:59:59.751330 21062 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.659489 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.733796 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.736085 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.732657 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.816289 20157 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 12:59:59.737036 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.737274 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:59.713482 21088 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.707538 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.818081 21088 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 12:59:59.818949 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 111457us
I20260709 12:59:59.819615 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.808336 21067 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 12:59:59.820691 20984 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.794215 20159 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 12:59:59.821291 20984 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.821722 20984 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.811043 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.823016 20852 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:59.731488 21081 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 12:59:59.804000 20010 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.724867 21054 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.729368 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.736132 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.765013 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.830324 20984 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 12:59:59.830504 21067 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.660305 21089 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.660498 21079 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 12:59:59.832350 21079 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 12:59:59.768435 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:59.661713 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.834378 20850 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 12:59:59.795264 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.834686 21079 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 12:59:59.714921 21031 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.732115 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.836074 21063 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.836481 21063 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.836869 21063 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.838089 20087 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:59.807503 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 12:59:59.838901 21088 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 12:59:59.655364 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 12:59:59.810075 21075 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 12:59:59.840015 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 130151us
I20260709 12:59:59.840360 21075 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.840747 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.841518 21081 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:59:59.797367 20978 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.842031 21081 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.802156 20162 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 12:59:59.842480 21081 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.844627 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.654978 21070 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.188s
I20260709 12:59:59.844653 20164 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 12:59:59.848803 21089 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.849730 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:59.849874 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.850508 20087 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 12:59:59.851480 21089 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 12:59:59.852507 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.852828 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.808653 21029 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 12:59:59.853602 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.855696 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.707651 21040 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 12:59:59.859175 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:59.863044 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 12:59:59.868216 20853 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 12:59:59.868909 20015 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 12:59:59.858613 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.857828 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.857842 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.872810 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.874517 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:59:59.878638 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 232641us
I20260709 12:59:59.881256 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.882225 20977 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:59:59.882725 20977 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.883133 20977 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:59.882242 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.885264 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 12:59:59.879856 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.903573 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 12:59:59.911772 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52653.2us
I20260709 12:59:59.925064 20161 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 12:59:59.818105 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 12:59:59.925828 20744 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.950541 20853 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:59.945271 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 12:59:59.870306 21077 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:59.869006 21073 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.894435 19982 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 12:59:59.959798 21078 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 12:59:59.950541 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47532.5us
I20260709 12:59:59.893918 21072 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.889016 21062 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.913028 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 12:59:59.891125 21008 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.893046 21068 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.925078 21053 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:59:59.946148 21074 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:59.960601 21069 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:59.993413 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.994283 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.030326 20978 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 12:59:59.994313 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.994992 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.995806 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.996292 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 12:59:59.925166 21064 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:59:59.999035 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 104676us
I20260709 12:59:59.889086 21065 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 12:59:59.891053 21063 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:00.023844 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 110930us
I20260709 13:00:00.031234 20085 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:00.038513 21064 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.001701 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:00.041420 21064 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.041486 20978 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.029754 21077 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.042505 21077 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.022581 20163 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:00.042884 21077 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.025452 21008 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:00.043753 21008 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.029470 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.003043 21067 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:00.002991 21072 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 12:59:59.997577 21073 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:00.003206 21062 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.026571 20158 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:00.037227 20086 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:00.043792 20978 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.022890 21068 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.025873 21053 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:00.075980 21053 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.076098 21094 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:00.077641 20056 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:00.041572 20084 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:00.060835 21073 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.082312 21092 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:00.082772 21092 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:00.083181 21092 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.030736 21069 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.084136 21069 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.084515 21029 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.086684 21063 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.029081 21080 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:00.088404 21080 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:00.089658 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.090813 21080 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.091369 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.091665 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.058429 21031 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:00.092969 21031 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.093912 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.094575 20083 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:00.095074 21031 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.095225 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.096088 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.096572 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.098541 21040 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:00.098814 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.099157 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.074892 21091 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:00:00.075407 21068 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.026500 21078 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.092449 20984 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.099484 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.001828 21071 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:00.069983 21062 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.072678 20978 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:00.077329 21094 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.059293 21072 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.114588 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.116456 21008 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.116456 21069 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.103950 21077 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:00.045959 21089 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:00.122681 20978 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.136106 20978 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:00.136956 20978 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:00.137586 21078 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:00.138077 20978 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.099960 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.120992 21071 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:00.140172 21071 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.135015 21089 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:00.142093 21084 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:00.128957 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08003a9a20 after lost signal to thread 12412
W20260709 13:00:00.144831 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08001266a0 after lost signal to thread 19874
I20260709 13:00:00.145764 21089 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:00.129019 21008 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:00.132576 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 187393us
I20260709 13:00:00.150312 21077 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:00.121374 21073 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.129204 20853 raft_consensus.cc:1240] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.155573 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:00.146134 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08003fb680 after lost signal to thread 20041
W20260709 13:00:00.157102 20112 debug-util.cc:398] Leaking SignalData structure 0x7b0800355160 after lost signal to thread 20054
W20260709 13:00:00.158393 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08001f3fe0 after lost signal to thread 20115
I20260709 13:00:00.140257 20984 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:00.129172 21069 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.154151 21071 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:00.160326 20984 consensus_queue.cc:1243] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:00:00.161142 20112 debug-util.cc:398] Leaking SignalData structure 0x7b0800224680 after lost signal to thread 20189
W20260709 13:00:00.161839 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08003caa00 after lost signal to thread 20190
I20260709 13:00:00.162145 21071 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:00.163592 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08003a24e0 after lost signal to thread 20888
I20260709 13:00:00.083767 21081 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.168735 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.169556 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.170486 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.170811 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.163774 21069 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.172282 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.131800 21094 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.172550 21069 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:00.153997 21053 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.163373 21075 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:00.183080 21072 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.185665 21053 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:00.165961 21064 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.178020 20977 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.154033 21084 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:00.175282 21069 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.186738 21075 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.186738 21081 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.193240 21068 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.196584 21073 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:00.197535 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 195031us
I20260709 13:00:00.199568 20851 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:00.200605 21068 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.200760 21064 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.191699 21092 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:00.202502 21072 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.195729 21062 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.204481 20977 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.205333 21069 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.206620 21092 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:00.206853 21062 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.208246 21066 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.208649 21066 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:00.209496 21066 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.211045 21066 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.041441 20010 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:00.049938 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:00.274475 20016 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:00.276790 20082 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:00.280167 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.280866 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.283095 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.284591 20091 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:00.299938 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.300230 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.305047 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.306331 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.307574 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:00.308064 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.310712 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.311260 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.308099 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:00.313802 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:00.326495 21087 consensus_queue.cc:1059] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.328423 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.329663 20082 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:00.198803 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:00.297183 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:00.337375 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.342175 21102 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:00.343885 21102 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:00.360720 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 283626us
I20260709 13:00:00.365692 21065 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:00.367132 21065 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.371182 21065 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:00.370105 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:00.374955 21089 raft_consensus.cc:3055] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.377032 21089 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:00.379551 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 87819.8us
I20260709 13:00:00.379362 21089 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:00.388826 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 341303us
I20260709 13:00:00.390851 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:00.396404 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 197590us
I20260709 13:00:00.396975 21105 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:00.397710 21105 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:00.398007 21105 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.404345 21087 consensus_queue.cc:1059] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.404631 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:00.405612 21075 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.406133 21075 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.406635 21075 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.406005 21087 raft_consensus.cc:3055] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.407866 21077 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.412328 21077 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.412797 21077 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.414347 21105 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.410590 21089 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.418598 21105 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.419813 21065 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.420476 21065 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.412101 21087 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:00.421033 21065 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.422482 21087 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.424233 21087 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.429371 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.429061 19838 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 3, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.438683 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.440429 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:00.440819 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57056.3us
I20260709 13:00:00.442885 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:00.444166 21072 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:00.444526 21077 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:00.407217 20744 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:00.432478 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.449343 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:00.432966 21080 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:00.450603 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 43685.8us
I20260709 13:00:00.434238 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.434588 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.451062 21069 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:00.435101 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.451870 21080 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:00.451995 21069 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.435510 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.452621 21069 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.372195 21102 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.436929 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.395604 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:00.452373 21080 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:00.450963 21070 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:00.454627 19838 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:00.455433 21080 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.457821 21080 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.460036 21070 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.460634 21070 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.458806 19838 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 4, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:00.451452 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 51807.5us
I20260709 13:00:00.465728 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:00.466468 21065 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.463941 19838 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 4, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.471101 20085 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:00.474099 21065 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:00.484251 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 89146.5us
I20260709 13:00:00.484422 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.484443 21101 consensus_queue.cc:1059] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:00.485873 21101 consensus_queue.cc:1243] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:00.486091 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:00.486471 21064 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.486706 21101 raft_consensus.cc:3055] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:00.487099 21064 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.487390 21106 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:00.487632 21064 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.488245 21106 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:00.488344 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.488938 21106 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.489492 20984 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:00.447844 21077 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:00.490514 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.820s (SENT) [suppressed 306 similar messages]
I20260709 13:00:00.490803 20984 consensus_queue.cc:1059] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:00.493207 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.493677 20984 consensus_queue.cc:1243] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:00.494095 20085 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:00.494024 21069 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.494822 21073 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.495141 21071 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.495465 21073 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:00.496165 21098 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:00.496593 21098 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:00.497272 21073 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.498116 21098 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.499029 21073 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.500524 21098 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.498014 21069 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.487385 21101 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:00.549649 21101 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.552738 21101 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:00.544970 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.557507 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.571583 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:00.535836 21075 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:00.577620 21075 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:00.583607 20082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "8b5b4e2a92b046d096f2661c54321d41"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:00:00.585031 20082 raft_consensus.cc:606] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Received request to transfer leadership to TS 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:00.585929 20857 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:00.588169 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 97853.1us
I20260709 13:00:00.570312 21071 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:00.598946 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:00.600116 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.605931 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.606185 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.607331 20012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:00.595707 21089 consensus_queue.cc:1059] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:00:00.608767 20012 raft_consensus.cc:606] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:00.608420 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.609746 21089 raft_consensus.cc:3055] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.611080 21089 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:00.617395 21084 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:00.622785 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.623104 21089 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.624504 21089 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.626216 21084 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:00.627090 21084 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.617383 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:00.629810 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.637501 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.638420 20163 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.640363 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.641484 20161 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 2: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:00.652473 20130 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:00.653273 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:00.663816 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.615082 21070 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:00.674444 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:00.671478 21071 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:00.677112 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:00.690028 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:00.691110 21072 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:00.691891 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.692333 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:00.698244 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 62770.7us
I20260709 13:00:00.712177 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.718000 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.718497 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.684315 21071 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:00.699985 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.711848 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.724785 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.716141 21072 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:00.735378 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.733359 20086 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:00.741554 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.741914 21072 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.742501 20085 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:00.743221 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.710114 21106 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:00.744393 21071 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.745486 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.761730 20084 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:00.762419 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.763109 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.763814 20089 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:00.762693 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.764613 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.765905 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.767175 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.767526 20854 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:00.766072 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.770996 20853 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:00.772183 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:00.772339 21066 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:00:00.773669 20012 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:00.775491 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:00.772218 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.779400 21070 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:00.783833 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.789518 20858 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.785867 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:00.785769 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.793339 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.799288 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:00:00.825052 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:00.825351 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:00.820115 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 70559.4us
I20260709 13:00:00.804080 21106 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:00.838927 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.839237 20853 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.841485 21084 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.842871 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.843351 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.854648 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:00.844480 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.845160 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.842106 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.849393 20852 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.856112 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.858096 20091 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:00.858476 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.859933 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.864394 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.864921 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.866050 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.868463 20857 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.799872 20163 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:00.859699 20010 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.875437 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.803130 21064 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.881603 21075 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.882220 21084 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:00.858932 20851 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
W20260709 13:00:00.886322 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:00.888264 20085 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:00.891222 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.896142 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:00.897038 20089 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:00.899554 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:00.905989 20086 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:00.915225 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.909303 20083 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:00.918426 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.918877 20852 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.919378 21070 raft_consensus.cc:993] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:00.921648 20089 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.921012 21070 raft_consensus.cc:1081] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:00.926131 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:00.923074 21087 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:00:00.927395 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:00.927855 20852 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:00.928795 21087 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.929411 21087 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:00.930439 21080 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:00.930371 21087 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.931835 21080 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.934774 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.932291 21080 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:00.937340 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:00.936950 21080 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:00.942289 21069 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:00.942639 21112 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:00:00.943679 21075 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:00.944309 21075 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:00.947471 21075 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:00.947708 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:00.949211 20016 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:00.949626 20012 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.950830 21069 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:00.951650 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.952941 20156 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:00.958316 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:00.966192 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:00.966958 20157 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.967998 20161 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:00.972954 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:00.974778 20156 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:00.974594 21080 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.982676 19982 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:00.984021 21106 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:00.984598 21106 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:00.985674 21106 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:00.941088 21072 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:00:00.934676 21087 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.965106 21070 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.992738 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:00.996694 20156 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:01.001520 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:00.993898 20010 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:00:01.002970 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 80218.9us
I20260709 13:00:01.003791 21101 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:01.003993 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:01.011439 21077 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:01.020761 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.021333 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.022773 20161 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:01.023036 21113 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:00:01.023597 20012 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:01.024317 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.025453 20164 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.026192 20156 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.026885 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.031533 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.032439 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.005119 20157 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:01.055619 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.056957 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.055624 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:01.066344 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 63701.8us
I20260709 13:00:01.084932 21118 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:01.089790 21118 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:01.084995 21077 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.092427 21077 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:01.092679 21120 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.089047 21122 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:01.094501 21122 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:01.093044 21077 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.102771 21064 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:01.104601 21070 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.104512 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.071324 21072 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:00:01.121291 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:01.121495 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.122499 20012 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:01.122620 20156 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:01.123317 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.137871 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.091988 21118 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:01.139458 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.139755 21118 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:01.141104 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.097607 21122 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:01.144371 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.144826 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.147092 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:01.158989 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 79514.5us
W20260709 13:00:01.171504 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 164645us
W20260709 13:00:01.180751 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 120032us
I20260709 13:00:01.127559 20008 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:01.181272 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:01.183396 21084 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.185940 20161 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.187031 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.190999 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.191375 21062 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:01.196704 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.197501 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.198164 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.198822 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.198643 21117 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.200438 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.205787 21117 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:01.202617 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.204171 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.201864 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.207024 21117 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:01.208277 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.218879 21071 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.219306 21071 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:01.219748 21071 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.219480 20129 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:01.225322 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 173594us
I20260709 13:00:01.234444 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.247789 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.249336 21072 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:01.250404 21069 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:01.248992 21077 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:01.251981 21069 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:01.258862 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:01.260679 21077 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:01.269220 21069 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:01.270251 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:01.272559 21069 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.264101 21072 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:01.285543 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.288900 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:01.294092 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.299654 21118 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.300523 21118 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:01.301054 21118 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:01.303134 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.311393 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.314672 21075 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.314976 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:01.315682 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:01.296440 21072 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:01.320916 21118 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:01.324258 20016 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:01.328369 21124 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:01.340554 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:01.341915 20855 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.343075 21080 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:01.344365 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:01.344156 21080 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:01.345765 21087 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:01.346468 21087 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:01.347105 21080 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:01.348062 19835 catalog_manager.cc:5697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:01.344575 20130 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.363368 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.371207 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:01.350466 21124 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:01.377004 21124 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.380021 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:01.382925 20858 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:01.369155 19835 catalog_manager.cc:5697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 2, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 2 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:01.357502 21118 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:01.401008 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.380098 21116 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:01.365432 21071 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:01.397866 21120 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:01.412724 20085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
W20260709 13:00:01.429361 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50644.6us
W20260709 13:00:01.430220 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.430322 20085 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:01.430984 21135 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.431082 20085 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.431257 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:01.425403 21062 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:01.429625 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:01.441686 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44802.1us
I20260709 13:00:01.442902 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.445323 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:01.446095 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:01.447235 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:01.425263 21106 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:01.448673 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.458525 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:01.459339 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.454217 21120 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:01.461871 21087 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:01.461449 20014 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:01.458606 21106 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.511507 21075 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:01.511852 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:01.512945 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 142663us
I20260709 13:00:01.513197 21106 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.513435 20009 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:01.513633 21106 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:01.514897 21106 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:01.517441 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.518384 21077 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.536669 21089 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:01.537828 21089 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:01.541682 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.542798 21135 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.554338 21120 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:01.583882 21123 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.594246 21089 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:01.597519 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:01.612633 21089 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:01.613606 21089 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:01.555411 21124 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:01.637928 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 96542.1us
I20260709 13:00:01.638252 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:01.636718 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 119375us
I20260709 13:00:01.642235 20085 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:01.653479 19838 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 152) took 1280 ms. Trace:
I20260709 13:00:01.670845 20162 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:01.674135 20085 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:01.650161 21106 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.677217 20009 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.678548 20014 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.678733 20012 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.680009 20008 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.681402 20016 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:01.684073 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.663998 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.685910 20743 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:01.655228 21064 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:01.688439 21064 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:01.690323 21098 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.691064 21064 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:01.687076 21101 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.693646 21073 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:01.694435 21073 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:01.696781 21071 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:01.696609 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:01.700150 21124 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:01.673108 21122 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:00:01.704774 21122 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.705339 21122 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:01.708092 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:01.709100 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.669453 19838 rpcz_store.cc:276] 0709 13:00:00.363109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:00.363669 (+   560us) service_pool.cc:224] Handling call
0709 13:00:01.643391 (+1279722us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:00.683850 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:00.748905 (+ 65055us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:00.748967 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:00:00.749005 (+    38us) tablet.cc:662] Decoding operations
0709 13:00:00.768897 (+ 19892us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:00.769027 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:00.769055 (+    28us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:00:00.792544 (+ 23489us) tablet.cc:701] Row locks acquired
0709 13:00:00.792603 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:00:00.792784 (+   181us) write_op.cc:270] Start()
0709 13:00:00.793009 (+   225us) write_op.cc:276] Timestamp: P: 1783602000792720 usec, L: 0
0709 13:00:00.793048 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:00:00.794115 (+  1067us) log.cc:844] Serialized 1693 byte log entry
0709 13:00:01.182974 (+388859us) op_driver.cc:464] REPLICATION: finished
0709 13:00:01.299729 (+116755us) write_op.cc:301] APPLY: starting
0709 13:00:01.299831 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:01.300022 (+   191us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:01.300242 (+   220us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:01.301256 (+  1014us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:01.302030 (+   774us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:00:01.302099 (+    69us) write_op.cc:312] APPLY: finished
0709 13:00:01.302624 (+   525us) log.cc:844] Serialized 40 byte log entry
0709 13:00:01.365688 (+ 63064us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:01.372085 (+  6397us) write_op.cc:454] Released schema lock
0709 13:00:01.393458 (+ 21373us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:01.439223 (+ 45765us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:01.450919 (+ 11696us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":116173,"apply.run_cpu_time_us":4684,"apply.run_wall_time_us":123379,"num_ops":4,"prepare.queue_time_us":142037,"prepare.run_cpu_time_us":16659,"prepare.run_wall_time_us":213950,"raft.queue_time_us":136335,"raft.run_cpu_time_us":15324,"raft.run_wall_time_us":81704,"replication_time_us":389367,"thread_start_us":312568,"threads_started":4,"wal-append.queue_time_us":110288}]]}
I20260709 13:00:01.712554 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.714289 20743 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.695446 21073 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:01.709846 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.716053 21073 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.716634 21098 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.716782 21069 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:01.716378 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:01.725646 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 64725.8us
I20260709 13:00:01.727751 21137 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.730685 21080 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:01.740674 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74134.4us
I20260709 13:00:01.748582 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:01.748813 21091 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.749344 21091 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:01.750056 20162 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:01.750625 21091 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:01.752771 20743 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.754645 20743 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.755774 21098 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.756335 21098 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:01.756692 21089 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:01.757359 21098 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:01.759395 21064 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.765825 21124 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:01.768179 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.770996 21069 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:01.771580 21069 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:01.771638 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.779661 21064 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.780361 21064 raft_consensus.cc:697] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:01.781492 21064 consensus_queue.cc:237] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:01.772054 21069 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.707238 21122 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:01.807149 21069 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:01.815335 21122 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.821377 21141 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:01.821870 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 94715.6us
I20260709 13:00:01.822131 21141 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:01.823416 19835 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 3, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:01.825577 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.821513 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:01.828830 21141 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:01.829881 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 56099.2us
I20260709 13:00:01.833689 21141 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:01.834810 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.835099 20008 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:01.836519 20017 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.836913 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.836541 20009 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:01.825687 19835 catalog_manager.cc:5697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 3, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:01.842612 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:01.841799 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:01.842958 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.843796 20012 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:01.844986 20015 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.845337 20008 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:01.852793 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:01.853716 20015 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:01.856127 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:01.863102 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:01.864441 20085 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:01.864733 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:01.866385 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:01.868624 20089 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.869786 20856 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:01.870613 20089 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:01.870977 20089 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:01.871798 20089 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:01.872982 20089 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.877310 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:01.878093 20014 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:01.878898 21069 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:01.880290 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:01.882166 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:01.885910 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:01.887230 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.891338 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:01.891769 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:01.893208 20083 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:01.893882 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.896288 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:01.897488 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 165238us
W20260709 13:00:01.905762 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 175866us
I20260709 13:00:01.921517 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.916286 20015 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:01.909957 19838 catalog_manager.cc:5697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:01.928021 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.929983 20084 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.938345 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:01.938558 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53332us
I20260709 13:00:01.939175 21071 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.939966 21071 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.940861 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.930219 20082 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.944106 21101 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.944947 21101 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:01.945988 21101 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:01.946347 21101 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:01.944133 21073 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.947475 21101 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:01.916975 20083 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:01.949101 21101 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:01.942344 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.931097 20089 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:00:01.953222 20089 pending_rounds.cc:85] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 1
I20260709 13:00:01.953516 20089 pending_rounds.cc:107] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:00:01.953806 20089 raft_consensus.cc:2889] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:01.957782 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:01.959627 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:01.960786 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.962412 21122 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.962675 21091 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.963725 20083 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:01.964955 20083 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.969595 21080 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.970687 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:01.971753 21123 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.973013 21071 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:01.974072 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.974267 21118 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.974851 21118 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:01.975821 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:01.986756 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.979710 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:01.994962 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.013743 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:01.924959 21124 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:01.930256 21106 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.957399 21117 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:01.976583 21145 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.979167 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:02.026288 21073 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:02.045447 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 69732.2us
I20260709 13:00:02.048264 21157 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:02.016741 21080 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 13:00:02.055111 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 75479us
I20260709 13:00:02.055856 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.017096 21069 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:00:02.022828 20084 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:02.039628 21142 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.049443 21073 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:02.016494 21137 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
W20260709 13:00:02.058879 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 64048.9us
I20260709 13:00:02.044883 21123 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.034588 21089 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:01.976300 21118 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:02.062563 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.085023 21089 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:01.979992 21071 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:00:02.068181 20016 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:02.054872 21141 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:02.078765 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.079176 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:02.090561 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76904.4us
W20260709 13:00:02.095372 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44950us
I20260709 13:00:02.064857 21157 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:02.051139 21135 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
W20260709 13:00:02.101996 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.076524 21073 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:02.110006 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.151630 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:02.169009 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.165294 21141 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:02.076170 21142 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:02.206768 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 135387us
W20260709 13:00:02.217471 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 116062us
I20260709 13:00:02.197012 21087 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:02.218910 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 109024us
I20260709 13:00:02.219647 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.233085 20083 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:02.233579 21140 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.223567 21091 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.214383 20010 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:02.222131 21141 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:02.239472 21141 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:02.243242 21145 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.245559 21157 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:02.246309 21145 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:02.248330 21072 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:02.248912 21072 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:02.249125 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.221745 21087 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:02.251819 21135 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.272246 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.258631 21137 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:02.324965 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.267567 20082 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:02.251554 21072 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:02.350473 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 198923us
W20260709 13:00:02.354483 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86751.1us
I20260709 13:00:02.355717 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:02.331188 21113 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:00:02.324450 21137 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:02.360739 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.352344 21123 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.393661 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.374758 21077 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:02.397298 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 172887us
I20260709 13:00:02.398382 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:02.404055 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.405752 21163 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.409271 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.412498 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.415185 21172 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.410863 21077 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:02.417119 21120 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:02.422485 21077 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:02.423786 21077 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:02.424226 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:02.424988 20016 raft_consensus.cc:2376] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2. Re-sending same reply.
I20260709 13:00:02.387981 21087 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:02.435585 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:02.438102 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.401501 21098 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:02.391166 21084 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:00:02.458690 21169 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:02.470672 21169 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:02.472031 21080 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.472798 21091 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.473292 21091 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:02.424110 21075 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:02.467686 21064 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:02.471712 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:02.488884 21091 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:02.490182 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 77776.2us
I20260709 13:00:02.464299 21137 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:02.494876 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.491087 21137 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:02.482015 21101 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.004s
I20260709 13:00:02.505033 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:02.505484 21118 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.505846 21091 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.478559 21069 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:02.479341 21064 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:02.517985 21091 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:02.521564 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.512869 21169 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:02.522157 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:02.522516 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.499795 21087 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.523294 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:02.527643 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.524016 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.533958 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.534770 20089 raft_consensus.cc:2376] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2. Re-sending same reply.
I20260709 13:00:02.536355 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.532514 20087 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:02.491326 21075 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:02.475055 21098 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:02.548122 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.541553 20008 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.536355 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:02.540570 20129 leader_election.cc:336] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.620s (SENT)
I20260709 13:00:02.550793 20129 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:02.552214 20089 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 13:00:02.557514 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.551625 20085 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:02.555430 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.560824 20090 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:02.562292 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.558763 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:02.565237 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.566217 20089 raft_consensus.cc:2376] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2. Re-sending same reply.
I20260709 13:00:02.561424 20091 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:02.568295 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:02.569293 21075 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:02.532361 21073 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:02.574748 21073 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:02.583369 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:02.585425 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:02.592355 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.543020 21069 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:02.610620 20851 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:00:02.635514 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.608476 21116 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.608866 21157 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:02.636987 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.614112 21073 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:02.644639 20085 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:02.647847 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:02.649533 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 109006us
I20260709 13:00:02.652904 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:02.653586 20085 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.636559 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.657640 21155 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:02.657725 20087 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:02.665133 20743 leader_election.cc:336] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.629s (SENT)
I20260709 13:00:02.669108 20743 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:02.673501 20087 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:02.681526 20090 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:02.616708 21064 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:02.656884 21157 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:02.686617 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:02.648512 21073 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:02.665537 21155 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:02.691512 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:02.688598 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:02.694468 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:02.716421 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.689390 20852 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:02.730901 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 65801.3us
W20260709 13:00:02.748705 20743 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.820s (SENT)
I20260709 13:00:02.734555 21182 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.699328 21155 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.741571 21157 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:02.744009 21179 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:00:02.744386 21164 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.776383 21164 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:02.332044 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:02.781956 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:02.745420 21185 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.784866 21185 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:02.788599 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 106904us
I20260709 13:00:02.795198 21142 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:02.800215 21124 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.801088 21124 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:02.749114 20743 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:02.757892 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:02.758558 20088 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:02.762146 21073 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:02.764688 20009 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.769137 21098 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:02.770022 21084 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.771450 20017 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:02.821658 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 349987us
I20260709 13:00:02.775066 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:02.827641 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:02.828935 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 80174.5us
W20260709 13:00:02.829813 20743 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.530s (SENT)
I20260709 13:00:02.830771 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:02.831315 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.832847 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.835210 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.836098 20083 raft_consensus.cc:2376] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3. Re-sending same reply.
I20260709 13:00:02.840314 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:02.843081 20129 leader_election.cc:336] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.504s (SENT)
I20260709 13:00:02.843521 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.844098 20008 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:02.823096 20085 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:02.845769 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.847102 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.847958 20085 raft_consensus.cc:2376] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4. Re-sending same reply.
I20260709 13:00:02.848559 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:02.849107 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.850369 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.851152 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.779822 21164 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:02.854490 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:02.859720 20085 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:02.865023 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.866292 20085 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:02.863832 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.843266 21087 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.867579 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.847826 21098 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:02.868306 20087 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:02.868409 21098 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:02.870545 21098 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:02.871308 21185 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:02.847579 21073 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.872069 21069 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:02.868005 21087 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:02.874938 21087 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:02.875342 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:02.876237 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 45533us
I20260709 13:00:02.876597 20082 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:02.878023 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:02.878744 20857 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:02.877358 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:02.881914 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.895695 21087 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.895877 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:02.896607 20164 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.896899 20091 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:02.897935 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:02.898288 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.900012 20854 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:00:02.902057 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 59010.5us
I20260709 13:00:02.902073 20085 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:02.903760 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:02.909978 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:02.882578 20055 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:02.919330 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:02.880270 21072 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.911258 20013 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:02.923573 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:02.923177 21142 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:02.932595 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:02.881228 20084 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:02.913046 21087 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:02.960214 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:02.985612 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:02.997817 21181 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:02.933105 21142 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:02.916687 20156 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.923777 21124 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:02.928987 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:03.016111 20164 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:02.932332 20157 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:02.936981 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:02.998976 21118 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.039191 21118 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.039759 21118 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.041301 21064 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:02.936723 21159 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:03.028986 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 125335us
I20260709 13:00:03.011147 21142 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.012042 21091 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.015326 21177 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:03.005120 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:03.036615 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 117394us
I20260709 13:00:03.059890 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.030798 21183 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.048411 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:03.030802 21166 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.063032 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.054925 21084 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.054155 21127 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.057447 21159 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:03.065541 21137 consensus_meta.cc:287] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.008s
I20260709 13:00:03.051170 20083 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:03.078310 21159 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:03.076242 21183 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.070366 21080 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.090472 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:03.103791 20129 leader_election.cc:336] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.795s (SENT)
I20260709 13:00:03.104526 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:03.106014 21163 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.061902 21155 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:03.124595 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64786.8us
I20260709 13:00:03.129254 20056 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.131017 21080 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:03.131716 21080 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:03.132668 21080 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.133639 20743 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.136076 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.064752 21177 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.136307 21183 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:03.155836 20852 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:03.137961 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.155879 21183 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:03.156164 21080 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.157166 21183 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.161056 21080 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.135556 20743 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:03.137620 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 77864.3us
W20260709 13:00:03.139452 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.073761 21084 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.067852 21166 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.169219 21166 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:03.169834 21166 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.074005 21127 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.172348 21127 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.174360 21084 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.175228 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.125680 21181 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:03.176111 21181 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:03.176534 21181 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.150619 21118 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:03.151528 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.152832 21155 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:03.153381 20012 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:03.155342 21177 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:03.166159 20129 leader_election.cc:336] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.861s (SENT)
I20260709 13:00:03.181676 20158 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:03.182075 21118 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:03.184165 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46343.5us
W20260709 13:00:03.184679 20129 leader_election.cc:336] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.865s (SENT)
I20260709 13:00:03.185050 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.186043 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.186353 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.186851 20164 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:03.188467 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.189296 20158 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:03.190508 21185 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.190997 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.191365 21185 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:03.191520 21101 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:03.190536 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.192142 21185 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.194072 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.195868 21091 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.195956 21087 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:03.196099 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 186656us
I20260709 13:00:03.204669 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.187325 21080 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.205515 20164 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:03.196879 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.208086 20015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:03.208477 21080 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.209041 20015 raft_consensus.cc:606] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.211140 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.214879 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.215716 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:03.215842 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:03.216892 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.217306 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.215608 20160 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:03.187963 21145 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:03.217649 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.218628 20164 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:03.218518 21145 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.220635 21156 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.221127 21203 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.221191 21156 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:03.222329 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86857.3us
I20260709 13:00:03.224225 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:03.225013 21177 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.226293 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:03.227588 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:03.220902 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:03.238754 20090 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:03.208148 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.239069 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.217918 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:03.243155 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:03.223667 21156 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:03.180580 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:03.251627 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.242704 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:03.254688 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.254935 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.245332 20853 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:03.258402 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:03.261328 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:03.262028 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:03.262769 20015 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:03.233654 21177 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.269125 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:03.283107 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.293435 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:03.293944 21185 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:03.294866 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 87053.3us
I20260709 13:00:03.319901 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.295768 21202 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.298043 21182 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.298864 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:03.275135 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:03.331243 20089 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.220965 21145 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.310667 21069 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.265404 21071 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:03.266193 20163 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.257489 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:03.323719 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.324555 19839 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 4, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:03.321776 21101 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:03.339079 20160 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.328938 20158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:03.263859 21064 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:03.274621 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:03.332281 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:03.330708 20156 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:03.359090 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:03.356161 20158 raft_consensus.cc:606] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.328954 21157 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:03.364111 20015 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:03.364529 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.290800 21181 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.341027 21071 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:03.369364 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49590.4us
I20260709 13:00:03.339406 21166 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.356921 21185 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.358176 21064 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.372504 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:03.375623 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 76837.6us
I20260709 13:00:03.376784 20089 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:03.364009 21202 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:03.380422 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.388427 20856 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:03.373872 21071 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:03.393965 20011 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:03.370422 21182 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.394858 21168 consensus_queue.cc:1059] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:00:03.407846 21141 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:03.403884 21127 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:03.380390 21064 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.407874 21137 raft_consensus.cc:3055] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.411973 21137 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:03.405236 21116 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:00:03.413048 21116 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:03.412912 21137 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.413442 21116 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:03.414147 21137 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.414382 21116 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:03.419127 20129 leader_election.cc:336] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.890s (SENT)
I20260709 13:00:03.421082 21181 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:03.386811 21202 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.421693 21202 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:03.381953 21118 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:03.422844 21118 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.368394 21157 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.366055 21101 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:03.425487 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45148.1us
I20260709 13:00:03.412245 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.430619 20160 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:03.447340 20056 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.415776 21116 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.455577 21205 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.458817 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:03.410250 21080 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:03.463155 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 67716.2us
I20260709 13:00:03.385135 21069 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:00:03.461946 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:03.446727 21162 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.477939 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:03.479074 21181 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.481637 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.483242 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.483721 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.474059 21166 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.492036 21080 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.499161 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.500114 21087 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:03.500494 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.501238 21091 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.504050 21156 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.486490 21157 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.480664 21168 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.504645 21156 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.505134 21168 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:03.505712 21168 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.506768 21156 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:03.518483 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.519698 20160 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:03.491456 21182 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:00:03.484375 21162 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:03.543666 20129 leader_election.cc:336] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.012s (SENT)
I20260709 13:00:03.550019 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.512930 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.554044 20160 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
W20260709 13:00:03.559491 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.539855 21064 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.564040 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:03.568869 21169 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.569272 21169 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.569481 21064 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.569933 21169 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.527235 21164 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.532389 21166 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.491197 21167 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:00:03.547356 21162 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:03.578764 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78354.3us
I20260709 13:00:03.579967 21202 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.544776 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.511759 21203 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:00:03.576766 20852 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:03.544941 21142 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.601305 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:03.602486 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.599507 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.600312 21203 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:00:03.604177 21087 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:00:03.577546 21164 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:03.612691 21101 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:03.623510 21203 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.624123 21203 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:03.625056 21203 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.626252 20850 raft_consensus.cc:1240] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:03.626909 21203 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.548972 21183 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.627879 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.630291 20158 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
W20260709 13:00:03.643606 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44244.5us
I20260709 13:00:03.612546 21164 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:03.644958 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:03.617367 21127 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:03.618388 21145 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.647687 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.647047 19982 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:03.652308 21101 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.653144 21217 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.655026 21142 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:03.656020 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.656972 20158 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
W20260709 13:00:03.657788 21207 log.cc:927] Time spent T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.255s	user 0.004s	sys 0.000s
W20260709 13:00:03.671175 21209 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:00:03.679013 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.685824 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:03.648279 21080 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:03.647758 21127 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.690490 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:03.689898 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.684506 21164 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.692191 21080 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:03.693547 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:03.696143 21213 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.696448 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:03.696219 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:03.714169 20086 raft_consensus.cc:2376] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2. Re-sending same reply.
I20260709 13:00:03.651218 21182 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:00:03.692003 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:03.707687 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.720366 21080 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:03.712615 21216 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.724184 21216 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:03.725139 21216 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.727121 21216 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.729657 20091 raft_consensus.cc:2376] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2. Re-sending same reply.
I20260709 13:00:03.735857 21183 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:03.712975 21087 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:00:03.743429 21087 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.743839 21087 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:03.744818 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.744946 21087 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.745734 21091 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:03.746186 21091 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:03.747081 21071 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.749032 20008 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:03.701632 21127 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:03.751667 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:03.761253 20161 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:03.702898 21164 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.704156 21217 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.688419 21116 consensus_queue.cc:1059] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:00:03.760496 21117 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.789009 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:03.719177 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:03.712384 21213 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.760327 21142 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.784221 21164 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:03.808820 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 167310us
I20260709 13:00:03.813745 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:03.803171 21064 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.814642 21223 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:03.823552 21223 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.824673 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:03.825168 21072 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.825553 21072 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:03.826534 21223 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.826493 21072 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:03.838089 21213 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:03.840142 21206 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.465s	user 0.002s	sys 0.000s
I20260709 13:00:03.840950 21118 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:03.841980 21071 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:03.812625 21217 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:03.747136 21091 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.807179 21117 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.845062 21091 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.846632 21117 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:03.849922 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.852494 21064 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.853184 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.854187 21202 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:03.854959 21064 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:03.856251 21210 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:03.843860 20857 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:03.856556 21163 raft_consensus.cc:3055] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.810750 20852 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.862666 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:03.855667 21064 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:03.885784 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 154626us
I20260709 13:00:03.885321 20161 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:03.805480 21168 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.888469 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.892220 20853 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 13:00:03.827776 21072 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.889863 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:03.858940 21217 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:03.824697 21166 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:03.896793 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:03.901134 20856 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:03.886555 20163 consensus_meta.cc:287] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.004s
I20260709 13:00:03.910981 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:03.911193 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:03.864046 21163 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:03.879545 21106 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:03.879539 21145 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:03.920576 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 106906us
I20260709 13:00:03.886646 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:03.912969 20163 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:03.922503 21069 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:03.931087 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:03.910318 21217 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:03.941498 21118 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:03.944985 21163 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.948094 21145 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:03.909963 21166 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:03.950349 21166 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.913110 21077 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:03.953089 21163 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.883769 21169 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:03.955823 21169 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:03.925341 21106 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:03.931113 21069 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:03.961800 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:03.961585 21069 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:03.964742 21166 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:03.978127 21168 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:03.992779 20017 raft_consensus.cc:606] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:03.995146 20163 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:03.959703 21106 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:04.000113 21073 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:04.059765 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:04.068168 19835 catalog_manager.cc:5697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 3, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:04.095587 21212 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.096501 21212 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:04.097039 21145 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.097678 21127 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:04.099315 21212 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:04.100154 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:04.124763 20158 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:04.126502 21127 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:04.105059 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
W20260709 13:00:04.106019 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 361269us
W20260709 13:00:04.110555 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:04.132553 20856 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:04.135769 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.136173 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:04.136929 20161 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.139556 20859 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:04.140878 20090 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:04.128512 20017 raft_consensus.cc:606] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.143890 21177 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.139463 20008 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:04.118050 21145 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.122229 21137 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:04.123420 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 212537us
I20260709 13:00:04.150735 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.104316 21087 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.141618 20086 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:04.110890 20852 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:04.145323 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:04.111640 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 188919us
I20260709 13:00:04.141512 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:04.154743 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.158247 20056 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.172281 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:04.172917 20017 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.173784 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.175886 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.180573 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:04.182927 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.171993 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.185196 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.193323 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.198114 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.199369 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.200073 20086 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:04.200418 20055 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:04.206804 20161 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:04.153306 20157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:04.169879 21203 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.180857 20163 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.223726 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:04.210664 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:04.165473 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:04.185407 21098 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:04.183868 21212 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.185074 20016 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:04.223618 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.219023 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:04.190357 21091 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:04.229091 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.191044 21124 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.186223 21231 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:04.153604 21145 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.208667 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:04.220178 21069 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:04.218315 21216 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.248018 21216 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:04.248767 21064 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:04.248998 21216 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.251201 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.251477 21124 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:04.225327 21203 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.253446 21167 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.253774 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:04.252548 21124 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.199472 21181 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.260553 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.261261 20089 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:04.261530 20163 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:04.261597 21159 raft_consensus.cc:993] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:04.262444 21232 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.262660 21091 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:04.263000 21169 raft_consensus.cc:1081] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:04.263348 20084 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:00:04.264288 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:04.263751 21091 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.266305 21183 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:04.266736 21183 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:04.267838 20853 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.224967 20157 raft_consensus.cc:606] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.269227 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.221701 21210 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:04.271189 20086 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:04.273315 20743 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.278241 20017 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:04.238176 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:04.279961 20008 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:04.270304 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:04.235464 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.267597 21183 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:04.289592 21145 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.280563 21069 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:04.291075 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.292761 21072 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.292109 21069 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.293691 20852 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:04.283598 21167 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.296864 21177 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.298254 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.298105 21164 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.300019 20855 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:04.300665 21182 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.300849 21181 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:04.301450 21182 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.301963 21182 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:04.289592 21183 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.300102 21101 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.308287 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.311786 20085 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:04.247401 21162 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:04.245833 21098 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:04.317760 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:04.317848 21162 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.321084 21162 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.322803 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:04.325381 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:04.305449 21106 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:04.319973 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.330468 21240 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:00:04.337671 21232 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:04.342222 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.343031 20090 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:04.343380 20084 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:04.346449 21158 consensus_queue.cc:1059] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:04.306428 21210 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:04.350598 21210 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:04.353494 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.324132 21159 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:00:04.362671 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.363463 20084 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:04.369405 21212 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.369992 21123 raft_consensus.cc:993] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:04.370549 21123 raft_consensus.cc:1081] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:04.370973 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:04.332496 20855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 13:00:04.372182 21232 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:04.372269 20855 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:04.372833 21232 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:04.373039 20855 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.374162 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:04.373906 21232 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.375032 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:04.375489 21232 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:04.375720 20157 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.376459 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:04.376235 21072 consensus_queue.cc:1175] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e5eb27db496b47129df3e689e546d94a" responder_term: 5 status { last_received { term: 3 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:04.378567 21231 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:04.378733 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.372504 21166 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.380280 21166 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.387630 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:04.388338 20090 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:04.388154 21098 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.389696 20091 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:04.392262 20161 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:04.394274 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:04.395145 20091 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.396207 21217 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.329885 21182 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:04.400836 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:04.379230 21231 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.402767 21228 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.401940 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:04.403364 21228 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.404075 21203 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:04.404446 21231 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:04.405644 21228 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.408633 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:04.413767 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.411473 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:04.371670 20852 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 13:00:04.415484 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:04.417331 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.417987 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:04.419972 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:04.421093 20163 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:04.406831 20853 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:04.358253 21164 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:04.355535 21106 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:04.447485 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.450383 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:04.457777 20856 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:04.459033 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.459824 21137 raft_consensus.cc:3055] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:04.460273 21137 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:04.460602 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:04.461203 21137 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.433761 21167 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.462514 21137 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.462386 21167 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.463330 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:04.434243 21181 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.464761 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:04.465229 21167 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.464980 21106 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.466681 21118 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:04.448119 21182 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.468665 21182 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:04.469079 21182 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.470713 20083 raft_consensus.cc:1240] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:04.450217 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.477551 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.479676 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:04.464169 20850 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:04.410665 21231 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.497632 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.402482 21166 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.408849 21203 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.523569 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.407500 21101 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:00:04.491281 21124 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.491459 21091 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:04.489878 20851 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 13:00:04.500001 21210 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.493546 21228 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:04.494415 21142 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.473134 21073 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:00:04.565268 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.578704 21210 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:04.579331 21210 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.539806 21231 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.556102 21240 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:00:04.581067 21142 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.584295 21166 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:04.584870 21248 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:00:04.595561 21116 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:04.618432 21162 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:04.619493 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 71430.5us
I20260709 13:00:04.560779 21071 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:04.621991 21071 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.622403 21071 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.585541 20851 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:04.556525 21124 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.644426 21124 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:04.628150 21225 raft_consensus.cc:993] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:04.647681 21225 raft_consensus.cc:1081] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:04.559387 21203 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.504774 21169 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:00:04.583602 21230 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.551661 21141 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:04.640097 20851 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.703685 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:04.687707 21141 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:04.705766 21091 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.740406 21162 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.749954 21231 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.721163 21217 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.713591 20157 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:04.729887 21141 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:04.758882 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:04.752128 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:04.711112 20855 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:04.703079 21230 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.765605 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:04.751801 21242 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.391s	user 0.006s	sys 0.000s
W20260709 13:00:04.780087 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57675.1us
I20260709 13:00:04.745344 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:04.760196 21203 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:04.797691 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 347538us
I20260709 13:00:04.746161 21091 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.793974 21228 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:04.815673 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:04.816180 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:04.819643 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:04.818372 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:04.808099 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:04.813787 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.837448 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:04.851190 21087 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.693s	user 0.010s	sys 0.007s
I20260709 13:00:04.836082 21182 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:04.866189 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.860175 20858 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:04.858134 20091 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:04.864990 21091 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:04.860134 20852 raft_consensus.cc:2393] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:04.860261 20084 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:04.855872 21166 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:04.891944 21261 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:00:04.858894 21087 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:04.855911 20857 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:04.861863 21182 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:04.822077 21228 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:04.902295 21120 raft_consensus.cc:993] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:04.903157 21120 raft_consensus.cc:1081] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:04.905627 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 521) took 1003 ms. Trace:
I20260709 13:00:04.906252 20853 rpcz_store.cc:276] 0709 13:00:03.902409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.906946 (+  4537us) service_pool.cc:224] Handling call
0709 13:00:04.059557 (+152611us) spinlock_profiling.cc:243] Waited 8.14 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:04.905319 (+845762us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000037d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:04.905492 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492280576}
W20260709 13:00:04.929596 21232 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.003s
I20260709 13:00:04.932418 21080 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:04.933349 21080 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.934089 21210 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:04.940368 21228 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.947510 21124 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:04.951099 20156 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:00:04.951998 20156 pending_rounds.cc:85] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 3
I20260709 13:00:04.952596 20156 pending_rounds.cc:107] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:00:04.953845 20156 raft_consensus.cc:2889] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:04.958303 21124 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:04.958613 21264 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.965196 21264 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.959944 21071 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:04.970219 21071 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:04.962462 21230 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:04.963531 20855 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:04.964126 20082 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:04.981439 21264 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:04.993296 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 175058us
I20260709 13:00:04.981618 20851 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:04.996315 20855 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:04.997331 21232 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:04.997543 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:04.997895 21232 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:04.983441 21210 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:04.988441 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:04.989943 21080 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:05.000845 20008 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:05.001754 21232 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:05.002547 20851 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:05.017210 20010 raft_consensus.cc:2393] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 2: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:05.004020 21163 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:05.005765 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:05.006572 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:05.021379 21232 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.021960 21232 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.008688 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:05.009320 21270 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:05.013068 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:05.037221 21071 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.019407 21141 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.019912 21163 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.038511 21163 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.029831 21212 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.032313 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:05.041952 21262 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.041749 21212 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.032874 21258 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:00:05.042733 21262 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.037608 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:05.022378 21232 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.045064 21262 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.045608 21212 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:05.046191 21212 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:05.046666 21212 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.006608 21142 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:05.042174 20082 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:05.053946 21101 consensus_queue.cc:1059] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:05.055634 21213 raft_consensus.cc:3055] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:05.056100 21213 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:05.056679 21120 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:05.057394 21071 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:05.057116 21213 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.058310 21164 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:05.058688 21164 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:05.058852 21141 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:05.059508 21164 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.059513 19838 catalog_manager.cc:5697] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:05.061045 21120 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:05.061928 21120 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.061707 19838 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:05.064738 19838 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:05.054091 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:05.066550 19835 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32772 (request call id 148) took 1615 ms. Trace:
I20260709 13:00:05.077018 21212 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:05.079085 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.081658 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.082507 20089 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.082738 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:05.085503 21262 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:05.086493 21262 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:05.086886 20083 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:05.087126 21262 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:05.085469 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.090950 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:05.099792 20129 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:05.099803 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:05.090314 20008 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:05.076030 19835 rpcz_store.cc:276] 0709 13:00:03.450868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.451041 (+   173us) service_pool.cc:224] Handling call
0709 13:00:05.066469 (+1615428us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:04.138168 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:04.138751 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:04.138797 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:00:04.138818 (+    21us) tablet.cc:662] Decoding operations
0709 13:00:04.157116 (+ 18298us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:04.180339 (+ 23223us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:04.190557 (+ 10218us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:04.220914 (+ 30357us) tablet.cc:701] Row locks acquired
0709 13:00:04.223156 (+  2242us) write_op.cc:260] PREPARE: finished
0709 13:00:04.230155 (+  6999us) write_op.cc:270] Start()
0709 13:00:04.245461 (+ 15306us) write_op.cc:276] Timestamp: P: 1783602004227869 usec, L: 0
0709 13:00:04.269922 (+ 24461us) op_driver.cc:348] REPLICATION: starting
0709 13:00:04.356037 (+ 86115us) log.cc:844] Serialized 537 byte log entry
0709 13:00:04.865307 (+509270us) op_driver.cc:464] REPLICATION: finished
0709 13:00:04.987409 (+122102us) write_op.cc:301] APPLY: starting
0709 13:00:04.987510 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:04.987606 (+    96us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:04.987722 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:04.987909 (+   187us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:04.988095 (+   186us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:04.988136 (+    41us) write_op.cc:312] APPLY: finished
0709 13:00:04.988490 (+   354us) log.cc:844] Serialized 22 byte log entry
0709 13:00:04.988674 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:04.988755 (+    81us) write_op.cc:454] Released schema lock
0709 13:00:04.988791 (+    36us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":29709,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":121202,"apply.run_cpu_time_us":2242,"apply.run_wall_time_us":2381,"num_ops":1,"prepare.queue_time_us":8790,"prepare.run_cpu_time_us":3969,"prepare.run_wall_time_us":268036,"replication_time_us":595173,"thread_start_us":156083,"threads_started":2}]]}
I20260709 13:00:05.105430 20852 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 13:00:05.106070 20852 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:05.106261 20088 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:00:05.106544 20852 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.107362 20088 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:05.108065 21064 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:05.108206 21270 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.109160 21064 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.094478 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:05.112725 20858 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.107857 20088 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.126595 20853 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:05.091041 21163 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.129635 21163 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.108100 21167 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.108090 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.134266 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:05.135026 20091 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.139125 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:05.141297 21216 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.142067 21216 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:05.145310 20088 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:05.146507 19840 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32774 (request call id 150) took 1493 ms. Trace:
I20260709 13:00:05.146912 19840 rpcz_store.cc:276] 0709 13:00:03.652722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.655863 (+  3141us) service_pool.cc:224] Handling call
0709 13:00:05.146343 (+1490480us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":395037,"reported_tablets":3}
I20260709 13:00:05.147465 20088 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:05.148408 20855 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.150135 20857 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:05.152438 21142 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:05.152716 21064 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:05.160034 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:05.167927 20158 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.175097 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.183815 20089 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
W20260709 13:00:05.186021 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:05.187897 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:05.186103 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.188977 20088 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.166884 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:05.179953 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:05.193965 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.195734 20160 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:05.197238 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.197885 20089 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.160853 21167 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:05.214721 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 523) took 1021 ms. Trace:
I20260709 13:00:05.214484 21167 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.215185 20859 rpcz_store.cc:276] 0709 13:00:04.193632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:04.196713 (+  3081us) service_pool.cc:224] Handling call
0709 13:00:04.388534 (+191821us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:04.456630 (+ 68096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:04.503022 (+ 46392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:04.505596 (+  2574us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:04.520469 (+ 14873us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:04.537489 (+ 17020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:05.213672 (+676183us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b54000046d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:05.213726 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:05.214502 (+   776us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54000046d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:05.214660 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178061312}
I20260709 13:00:05.218170 21137 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.219096 21137 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.219504 21137 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.199040 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:05.221101 20857 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.165750 21124 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:05.222226 21124 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.224113 20008 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:05.206645 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.225694 20083 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:05.253046 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86250.5us
I20260709 13:00:05.257184 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:05.230914 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:05.266395 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.275560 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:05.281924 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.170076 21212 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:05.286087 20858 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
W20260709 13:00:05.295446 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64835.4us
I20260709 13:00:05.281936 20158 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:05.305037 20089 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:05.261175 21216 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:05.232637 21142 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.315172 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.316291 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:05.318853 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:05.319855 20164 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:05.320428 20164 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:05.321266 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:05.317654 20089 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.322280 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.322400 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:05.247072 21098 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:05.318116 21142 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:05.324304 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 66704.7us
I20260709 13:00:05.326083 21142 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:05.324267 21098 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:05.328516 20087 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:05.328858 21098 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.330087 21098 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.244145 21124 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.332175 21216 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.332082 21183 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:05.333639 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.336750 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:05.338233 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:05.233632 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:05.227594 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:05.240593 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:05.251574 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:05.335891 20158 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:05.287528 21118 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.348594 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:05.380280 20161 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:05.341480 20088 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
W20260709 13:00:05.348824 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51349.4us
I20260709 13:00:05.346087 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:05.283387 21101 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.385166 21101 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:05.385218 21232 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.385921 20091 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:05.385844 21101 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.387445 20091 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:05.314169 21123 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.388585 21118 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:05.388756 20857 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:05.388760 21101 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:05.389832 21142 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:05.389684 21118 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:05.366124 21183 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.391429 21183 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:05.391633 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45681.6us
I20260709 13:00:05.392573 21183 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.394333 21064 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:05.340121 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:05.284016 21182 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:05.395892 20160 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.395998 21182 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.398571 21182 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:05.398706 21270 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.401386 20854 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:05.402819 20086 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:05.402570 20083 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:05.324697 21259 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.334509 20090 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.321585 20164 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.411767 20164 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:05.334260 20162 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:05.383608 20850 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:05.371744 21216 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:05.381722 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.419999 20158 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:05.406509 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:05.420936 20091 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:05.421149 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:05.313949 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:05.428208 20859 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.319732 21212 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.430496 21212 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:05.383721 20016 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:05.431339 20016 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:05.423741 21203 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:05.432046 20163 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.433187 21232 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.422570 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:05.418700 21137 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.434309 20852 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.435847 21137 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.432729 20016 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:05.436731 20016 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.413432 21069 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:00:05.442883 20159 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:05.444161 21257 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:05.436131 20855 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:05.451141 20852 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:05.417160 21259 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:05.459103 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:05.477090 20164 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
W20260709 13:00:05.483599 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 61062.5us
W20260709 13:00:05.487169 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 63611.7us
I20260709 13:00:05.490447 20160 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:05.490731 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:05.493834 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 89463us
I20260709 13:00:05.499616 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:05.491567 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:05.559257 20090 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.455783 21203 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:05.413796 21225 consensus_queue.cc:1175] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "48da76e971b84219a6ba80d1ba83b7eb" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:00:05.579730 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:05.395150 21064 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.587110 20164 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:00:05.588582 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91618.2us
W20260709 13:00:05.591763 21262 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.010s
I20260709 13:00:05.592252 21262 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.594099 21167 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.594581 21262 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:05.594686 21167 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.602002 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:05.603317 21064 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:05.482888 21270 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.125s
I20260709 13:00:05.576398 21257 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:05.576967 21212 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:05.586834 21203 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:05.583599 21080 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.492105 21069 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:05.610945 20165 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:05.588750 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:05.626686 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:05.627730 20091 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
W20260709 13:00:05.628492 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 302141us
I20260709 13:00:05.597925 21259 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.633193 20859 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:05.645391 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:05.662294 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:05.653838 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:05.664757 21265 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:05.634171 21084 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:05.624109 21069 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:05.681468 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:05.633855 21137 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:05.631500 21080 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:05.659458 20090 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:05.620834 21212 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.768711 21120 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.699827 21075 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:05.724229 21064 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:05.673604 20016 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:05.725907 21084 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.748287 21069 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:05.832909 21264 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:05.833437 21120 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:05.839170 21084 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:05.834196 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:05.768649 21137 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:05.773152 21265 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:05.812924 21073 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:05.833309 21080 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:05.815197 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:05.834682 21156 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:05.861848 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:05.839217 21064 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:05.841029 21075 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:05.835608 21124 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:00:05.842653 21264 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:05.834965 21257 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:05.905825 21230 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:05.860262 21073 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.900676 21270 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:05.902972 21142 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:05.909116 21071 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:05.952481 21075 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.868330 21156 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:05.916594 21282 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 13:00:05.978195 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:05.979205 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 267086us
I20260709 13:00:05.979442 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:05.982151 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:05.982718 20013 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:05.983073 20013 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:05.964404 21279 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:00:05.985466 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:05.986066 21264 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.986130 21262 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:05.986804 21262 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:05.987373 21262 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.988314 21156 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.917181 21230 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.961278 21120 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.994275 21230 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:05.966699 21265 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:05.996608 21286 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:00:05.984800 20013 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.997530 20013 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:05.998399 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:05.967954 21071 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.000875 21137 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:06.001175 21071 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:05.961789 21270 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:05.965269 21167 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.015673 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.016966 20853 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:06.018778 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:06.019429 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:06.019590 20859 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.026108 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.027087 20858 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:06.034768 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.035612 20089 raft_consensus.cc:2368] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
W20260709 13:00:06.051281 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:06.061564 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:06.004137 21084 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.068878 20010 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:06.069922 21084 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:05.964954 21142 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.072513 21142 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.073374 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.073549 21069 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.078481 20013 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:06.078830 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.079566 20087 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 1.
I20260709 13:00:06.082636 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:06.083473 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.084029 20013 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:06.084491 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.085160 20009 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:06.098475 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:06.005357 21259 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:06.003890 21270 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.058986 21230 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:06.115476 20859 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:06.126443 20087 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:06.132440 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.099637 21289 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.133601 20013 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:06.108197 21137 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:06.111714 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:05.982295 21064 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:06.139844 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:06.128325 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.125838 21230 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.128978 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.127506 21164 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:06.130597 21270 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.143460 21127 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:06.130576 21259 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:06.135979 21284 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:06.164762 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 971508us
I20260709 13:00:06.149386 21230 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:06.185395 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.154250 21212 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.161616 21270 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.149268 20009 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:06.199512 21270 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.161684 21127 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.195096 21156 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:06.199770 21203 consensus_meta.cc:287] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.005s
I20260709 13:00:06.196465 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:06.230928 20015 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:06.201434 21287 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
I20260709 13:00:06.201642 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.203352 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.205314 20010 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:06.205957 21259 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.140435 21289 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.198323 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:06.156576 20857 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:06.226631 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.236295 20089 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:06.233963 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.239253 20858 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:06.193310 21120 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.246031 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:06.254768 20013 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:06.261843 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.262472 20854 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:06.263513 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.233777 21156 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.281539 21127 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.288240 20011 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.161768 21284 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.252941 21071 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:06.291209 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:06.268107 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.279676 21156 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.240979 21203 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.296901 20013 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
W20260709 13:00:06.317816 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 48269.7us
I20260709 13:00:06.270329 21120 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:06.319622 20056 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.68:46223 timed out after 1.537s (SENT) [suppressed 37 similar messages]
I20260709 13:00:06.298969 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:06.314742 21295 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.295846 21071 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.325907 20852 raft_consensus.cc:2393] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:06.342844 20014 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:06.327855 20157 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:06.328935 21298 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:00:06.329242 21203 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:06.330324 21289 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.330235 21264 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:06.312312 21156 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:00:06.332522 21213 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.335599 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 518) took 1211 ms. Trace:
I20260709 13:00:06.388559 20856 rpcz_store.cc:276] 0709 13:00:05.120535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.120708 (+   173us) service_pool.cc:224] Handling call
0709 13:00:05.322614 (+201906us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:06.331411 (+1008797us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:06.331598 (+   187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:06.331787 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:06.331813 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:06.331980 (+   167us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:06.332017 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.332071 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:06.332178 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41143552}
I20260709 13:00:06.346675 20015 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:06.355595 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 635) took 1187 ms. Trace:
W20260709 13:00:06.354039 21301 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:00:06.404031 20055 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:06.376937 21120 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:06.394555 20157 rpcz_store.cc:276] 0709 13:00:05.167757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.169865 (+  2108us) service_pool.cc:224] Handling call
0709 13:00:06.322658 (+1152793us) spinlock_profiling.cc:243] Waited 59.4 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:06.327395 (+  4737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:06.351525 (+ 24130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.355503 (+  3978us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136580352}
W20260709 13:00:06.378216 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:06.379482 21071 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:06.380661 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:06.380663 21069 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:06.380384 21296 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.190s	user 0.005s	sys 0.000s
I20260709 13:00:06.347795 21295 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.409185 20011 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:06.412961 21203 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:06.415772 21116 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.416352 21145 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:06.418141 20163 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:06.419816 20159 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:06.420684 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.422335 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.424015 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.423776 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.332455 21262 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.429117 21203 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.436622 20164 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:06.391206 21307 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:00:06.430235 21306 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:06.439688 21203 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:06.455121 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 138324us
I20260709 13:00:06.437940 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.445735 21306 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:06.466331 20157 raft_consensus.cc:2368] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
W20260709 13:00:06.556679 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:06.603377 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:06.435346 21145 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.437098 21116 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:06.600270 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 134548us
I20260709 13:00:06.577402 21270 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.576874 21142 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.602077 21069 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.468037 21164 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.150s
I20260709 13:00:06.603500 21167 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:06.644858 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 565) took 1398 ms. Trace:
I20260709 13:00:06.635494 21098 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:06.631187 20156 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:06.649529 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:06.650591 20156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 563) took 1404 ms (client timeout 1628 ms). Trace:
I20260709 13:00:06.650908 20857 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:06.652245 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:06.651175 20156 rpcz_store.cc:269] 0709 13:00:05.245884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.313682 (+ 67798us) service_pool.cc:224] Handling call
0709 13:00:06.430255 (+1116573us) spinlock_profiling.cc:243] Waited 9.88 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:06.650449 (+220194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22728064}
I20260709 13:00:06.630535 21311 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:06.653474 21311 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.649470 21098 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.655534 21145 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.625687 21289 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:06.657090 21098 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:06.657773 21262 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.656653 21289 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.636041 21142 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.650643 21116 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.663254 21142 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.665287 21295 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.625514 21284 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:06.647336 21069 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.673132 20854 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:06.620270 21182 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.647984 20161 rpcz_store.cc:276] 0709 13:00:05.246523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.335506 (+ 88983us) service_pool.cc:224] Handling call
0709 13:00:05.372145 (+ 36639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:05.380941 (+  8796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.635135 (+1254194us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b540001a9d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:06.644708 (+  9573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83029248}
W20260709 13:00:06.684275 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 364638us
I20260709 13:00:06.626075 21303 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.692731 21216 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.693508 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.701470 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.706975 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.709316 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.709940 21145 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:06.710824 21257 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.714169 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:06.723229 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.716220 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:06.711644 21072 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.726393 20161 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:06.637137 21270 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.676966 21167 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.724092 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.703948 21311 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.755340 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:06.750185 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 428519us
I20260709 13:00:06.718035 21225 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:06.689869 21182 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.798897 21142 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:06.799537 21142 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.801446 21182 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:06.730182 21145 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.719907 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.803558 20857 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:06.804740 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.805369 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 131) took 1426 ms. Trace:
I20260709 13:00:06.805615 20850 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:06.701990 21303 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.808061 21142 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:06.809530 21167 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.810248 21303 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.704376 21216 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.812866 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.813418 21164 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:06.813907 21164 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.814234 21164 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.814613 21216 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.812738 21145 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.817222 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.818054 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:06.818969 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.826834 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:06.782680 20161 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:06.783769 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.784742 20856 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:06.785445 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.785957 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:06.831658 20013 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:06.786338 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:06.778683 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:06.788933 20014 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:06.832300 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:06.833150 20853 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:06.827787 20156 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:06.718075 21316 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:00:06.749641 21072 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.739207 21257 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.780522 21064 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:06.775728 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:06.819195 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.820266 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:06.821382 20016 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:06.809495 21142 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:06.856848 21142 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.792692 21270 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.827772 20164 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:06.860327 20852 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.806137 20162 rpcz_store.cc:276] 0709 13:00:05.378899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.449077 (+ 70178us) service_pool.cc:224] Handling call
0709 13:00:06.769646 (+1320569us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:06.804636 (+ 34990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:06.804853 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:06.804886 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:06.804932 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:06.804972 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.805034 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:06.805167 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65830784}
I20260709 13:00:06.852890 21072 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:06.855090 21257 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:06.855824 21064 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.862996 20853 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:06.863276 21064 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:06.860006 21142 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.860651 20164 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:06.865077 20164 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.865995 20164 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:06.875185 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:06.876617 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:06.906034 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 88104.2us
I20260709 13:00:06.910980 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.911860 20853 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:06.833565 21289 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:06.914225 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 135611us
W20260709 13:00:06.914919 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 94760us
I20260709 13:00:06.920331 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.921092 20163 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:06.921676 20164 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
W20260709 13:00:06.923237 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 148020us
I20260709 13:00:06.927688 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:06.910279 21257 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:06.926052 21213 consensus_meta.cc:287] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.005s
I20260709 13:00:06.934916 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.948231 20088 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:06.835064 21225 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:06.937094 19982 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:06.929047 21325 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:06.962533 21230 consensus_meta.cc:287] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.754s	user 0.008s	sys 0.006s
I20260709 13:00:06.962692 21325 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:06.963152 21325 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:06.962977 21230 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.963392 21073 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:06.964165 20087 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:06.965302 21230 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:06.947291 21257 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.935762 21264 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:06.972344 21264 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.975229 21264 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:06.939217 21310 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:06.926755 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:06.977213 20850 raft_consensus.cc:2410] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:06.977264 21283 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:06.940066 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.930233 21303 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:06.941908 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:06.978302 20090 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 13:00:06.978771 21230 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:06.979092 21292 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.866s	user 0.011s	sys 0.000s
I20260709 13:00:06.979328 21230 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:06.940573 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.943008 21295 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:06.980248 21230 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:06.981173 21073 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:06.981571 21295 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:06.982051 21295 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:06.982897 21295 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:06.983104 21073 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:06.944428 21297 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.747s	user 0.003s	sys 0.011s
I20260709 13:00:06.984602 20091 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:06.942593 21270 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:06.924897 21311 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:06.925163 21075 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:06.986143 21311 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:06.986222 21075 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:06.986639 21311 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.987964 21306 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.930399 21106 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:00:06.988695 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:06.989411 21306 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:06.991072 21295 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:06.992247 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 567) took 1745 ms. Trace:
I20260709 13:00:06.992336 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.992740 20158 rpcz_store.cc:276] 0709 13:00:05.247504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.445225 (+197721us) service_pool.cc:224] Handling call
0709 13:00:05.709920 (+264695us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:05.710136 (+   216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:05.979379 (+269243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:05.979410 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:05.979840 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:00:06.144042 (+164202us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:06.144066 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.144137 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:06.848627 (+704490us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:06.992006 (+143379us) raft_consensus.cc:1685] finished
0709 13:00:06.992035 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:06.992184 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64985,"raft.queue_time_us":103576,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":11662,"spinlock_wait_cycles":118473216,"thread_start_us":367649,"threads_started":2,"wal-append.queue_time_us":188178}
I20260709 13:00:06.994648 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.994764 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:06.995360 20088 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:06.998096 20743 leader_election.cc:400] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:06.997951 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.998458 20743 leader_election.cc:403] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:06.998857 20084 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:06.940912 21326 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:06.985596 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:06.999647 21326 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.000965 21213 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:07.001384 21213 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:07.001937 21326 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:07.002318 21213 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.002767 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:07.003129 20088 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:07.003499 21213 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:07.003619 21117 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:00:07.005052 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:06.934362 21142 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:07.005687 20087 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:07.006064 21142 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.006753 21142 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:07.007754 21257 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:06.955789 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:06.961993 20852 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:07.008123 21118 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:07.008778 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 53107.3us
I20260709 13:00:07.008615 21118 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:07.009492 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:06.985535 21225 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.986701 21075 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:07.011439 21118 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:06.947000 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 521) took 1725 ms. Trace:
I20260709 13:00:07.012382 20855 rpcz_store.cc:276] 0709 13:00:05.220529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.263749 (+ 43220us) service_pool.cc:224] Handling call
0709 13:00:05.332120 (+ 68371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:05.450882 (+118762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:06.927149 (+1476267us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54000078d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:06.945099 (+ 17950us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238718336}
I20260709 13:00:07.013545 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:06.938366 20086 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:06.961306 21203 consensus_meta.cc:287] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.002s
I20260709 13:00:07.004475 21303 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.015380 21303 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:07.015437 21203 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.016948 20854 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:07.018281 21203 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:07.020659 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.021790 20854 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:07.022243 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 117) took 1041 ms. Trace:
I20260709 13:00:07.004215 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.022642 20016 rpcz_store.cc:276] 0709 13:00:05.980303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.981406 (+  1103us) service_pool.cc:224] Handling call
0709 13:00:06.295738 (+314332us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5800867998. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:07.022015 (+726277us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b54000faad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:07.022136 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142040704}
I20260709 13:00:07.023571 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:07.025602 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:07.026293 20851 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:07.009622 21326 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.027753 21326 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:06.980712 21182 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.030233 21182 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.030968 21326 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:07.031548 21326 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:07.032174 21071 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:07.033212 21071 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:07.034034 21071 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:07.034689 21071 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.035575 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:07.035482 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.032298 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.036717 21283 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:07.037405 20857 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:07.038353 21177 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:07.038838 21177 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:07.039161 19982 leader_election.cc:400] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:07.040158 19982 leader_election.cc:403] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:07.040989 21177 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:07.041603 21106 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:07.038856 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.041653 21177 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:07.006779 21069 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:07.046140 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 46793us
I20260709 13:00:07.049803 21073 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:07.055949 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:07.056675 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 71152.3us
I20260709 13:00:07.057417 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:07.057343 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.061440 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.064633 20160 raft_consensus.cc:2368] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:07.064254 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.067361 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.068029 20086 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:07.070133 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.072225 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.073742 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:07.061152 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:07.074990 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.075774 20856 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.077764 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.079510 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.081296 20165 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:07.081827 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:07.084743 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.085886 20858 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:07.091856 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.092145 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.092562 20859 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:07.092667 20164 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:07.108431 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.081498 21069 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:07.082151 21177 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:07.087725 21164 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.076084 21168 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:07.091289 21283 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 13:00:07.143662 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67885.5us
I20260709 13:00:07.145802 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:07.150974 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:07.151666 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 95846.5us
I20260709 13:00:07.070133 21326 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.153990 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:07.155282 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.091391 21120 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 13:00:07.091588 21156 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:07.192020 21262 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.207882 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.199544 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:07.211663 20164 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:07.091701 21259 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:07.179595 20858 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:07.150070 21072 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:07.156970 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63194.7us
I20260709 13:00:07.187656 20016 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:07.186673 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.214849 20164 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:07.216017 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:07.191738 21145 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:07.200820 21156 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.145072 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.149747 21069 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.169255 20130 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:07.172297 21064 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:07.225893 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:07.172047 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.232470 20160 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:07.232672 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.169207 20017 raft_consensus.cc:2368] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:07.189988 21182 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:07.262902 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.237744 20853 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:07.276631 20013 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:07.185931 21075 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:07.166527 21327 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:07.143704 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.199069 21326 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.241119 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.279443 21216 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.240507 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:07.222802 21116 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:00:07.302300 21156 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.242146 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.284410 19838 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35376 (request call id 95) took 3940 ms. Trace:
W20260709 13:00:07.325789 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 156642us
I20260709 13:00:07.285784 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.218323 20856 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:07.154289 21098 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:00:07.295200 20086 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:00:07.296044 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 80143.9us
I20260709 13:00:07.306102 21163 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:07.333709 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:07.180763 21303 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.220358 21167 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:07.313210 21225 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:07.342388 20015 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:07.299525 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 147874us
I20260709 13:00:07.167476 21164 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:07.307327 20163 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:07.175112 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.224464 21072 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.168552 21168 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.173014 21311 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.365146 21337 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.366042 21337 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.367324 21311 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:07.275190 21270 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:07.242894 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.368199 21270 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.370523 21270 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:07.317044 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91246.7us
I20260709 13:00:07.372117 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:07.373307 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:07.373440 21168 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:07.348443 21163 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:07.374962 21284 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:07.223672 21141 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 13:00:07.210460 21262 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.375720 21284 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:07.376187 21284 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.222846 20851 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:07.376578 21284 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.376873 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 535) took 1277 ms. Trace:
I20260709 13:00:07.362615 21072 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.377297 20851 rpcz_store.cc:276] 0709 13:00:06.099022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:06.103743 (+  4721us) service_pool.cc:224] Handling call
0709 13:00:07.025418 (+921675us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:07.376783 (+351365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18505088}
I20260709 13:00:07.241654 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.305410 21069 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.327040 20164 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:07.304703 21260 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:07.311455 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.308961 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.385298 21284 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:07.387686 20157 raft_consensus.cc:2410] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:07.376365 21163 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:07.390311 21262 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:07.339757 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.306924 21177 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:07.311254 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.315407 21327 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:07.392016 21327 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:07.392484 21177 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:07.324064 21203 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.325682 19838 rpcz_store.cc:276] 0709 13:00:03.310889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:03.328066 (+ 17177us) service_pool.cc:224] Handling call
0709 13:00:03.480622 (+152556us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b54001cb010. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68e0b2 00007fa034607a89 00007fa03459fda6 00007fa034571088 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc
0709 13:00:04.023586 (+542964us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b54008b1fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 13:00:07.250844 (+3227258us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:05.598805 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:05.599413 (+   608us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:05.599465 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:00:05.599487 (+    22us) tablet.cc:662] Decoding operations
0709 13:00:05.672853 (+ 73366us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:05.675355 (+  2502us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:05.705841 (+ 30486us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:00:05.839618 (+133777us) tablet.cc:701] Row locks acquired
0709 13:00:05.839651 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:00:05.853000 (+ 13349us) write_op.cc:270] Start()
0709 13:00:05.860371 (+  7371us) write_op.cc:276] Timestamp: P: 1783602005849637 usec, L: 0
0709 13:00:05.860418 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:00:06.005629 (+145211us) log.cc:844] Serialized 1307 byte log entry
0709 13:00:06.791033 (+785404us) op_driver.cc:464] REPLICATION: finished
0709 13:00:07.174375 (+383342us) write_op.cc:301] APPLY: starting
0709 13:00:07.174470 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:07.174608 (+   138us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:07.174732 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:07.175278 (+   546us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:07.175452 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:00:07.175495 (+    43us) write_op.cc:312] APPLY: finished
0709 13:00:07.175716 (+   221us) log.cc:844] Serialized 34 byte log entry
0709 13:00:07.175891 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:07.176006 (+   115us) write_op.cc:454] Released schema lock
0709 13:00:07.176043 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:07.176412 (+   369us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:07.176438 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":59318,"reported_tablets":6,"spinlock_wait_cycles":247628928,"child_traces":[["op",{"apply.queue_time_us":383028,"apply.run_cpu_time_us":2025,"apply.run_wall_time_us":2026,"num_ops":3,"prepare.queue_time_us":524237,"prepare.run_cpu_time_us":49885,"prepare.run_wall_time_us":588536,"raft.queue_time_us":298399,"raft.run_cpu_time_us":84593,"raft.run_wall_time_us":308628,"replication_time_us":916744,"thread_start_us":811733,"threads_started":4,"wal-append.queue_time_us":227735}]]}
I20260709 13:00:07.323465 21334 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:07.304594 21145 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.368635 20091 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:07.300318 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.399289 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:07.410444 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 110220us
I20260709 13:00:07.411902 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.411902 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:07.421092 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:07.403003 20085 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:07.405982 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.407222 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.387678 21284 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:07.426592 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 64698.5us
I20260709 13:00:07.340961 20016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:07.424398 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.424361 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:07.326958 21216 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:07.463328 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 106001us
I20260709 13:00:07.416553 21259 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:07.464943 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:07.409109 21270 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.416922 21182 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:07.408843 21334 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.429082 21289 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:07.425372 21328 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.477000 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.416052 21080 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:07.396627 21142 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.440387 21284 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:07.486996 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:07.519686 20085 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.396777 21203 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.531662 21307 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:07.532162 21307 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:07.533331 21216 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:07.513515 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.536232 21123 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:07.536902 21337 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:07.497632 21073 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.537966 21073 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.538463 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:07.538910 21307 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.540242 21307 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:07.498174 21116 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:00:07.545607 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.546586 20082 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:07.547906 21284 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.548523 21284 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:07.549129 21145 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:07.551095 21071 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.001s
I20260709 13:00:07.508080 21306 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:00:07.438002 21325 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.517297 21311 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:00:07.554574 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:07.533939 21203 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:07.556162 21325 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:07.499752 21321 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.557449 21075 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:07.499114 21334 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.559074 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:07.559809 20855 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.499812 21257 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:07.464071 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.527945 21080 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.499421 21270 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.564465 21080 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.566991 21270 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:07.567065 21080 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:07.567632 21257 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:07.568800 21257 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.570103 21164 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:07.456107 21260 raft_consensus.cc:697] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:07.571208 21164 consensus_queue.cc:1243] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:07.463474 20016 raft_consensus.cc:606] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.571978 21118 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:07.572378 21118 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:07.572271 21260 consensus_queue.cc:237] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:07.528800 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51875.7us
I20260709 13:00:07.573299 21069 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.573666 21118 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.574733 20854 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:07.575253 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 61885.2us
I20260709 13:00:07.528602 20084 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:07.529397 20083 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:07.547258 20851 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:07.563350 21072 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.528961 21142 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:07.575845 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.472584 21230 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:07.583896 20008 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:07.562598 19839 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 156) took 3334 ms. Trace:
W20260709 13:00:07.596232 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 175190us
I20260709 13:00:07.599534 20085 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:07.605168 19982 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:07.606974 19839 rpcz_store.cc:276] 0709 13:00:04.228056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:04.228233 (+   177us) service_pool.cc:224] Handling call
0709 13:00:07.562529 (+3334296us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":8}
I20260709 13:00:07.618286 20744 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:07.584945 21213 consensus_meta.cc:287] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.000s
I20260709 13:00:07.581667 21325 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:07.631621 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54329.5us
I20260709 13:00:07.585852 21141 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:07.642841 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:07.643199 20085 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:07.551626 21071 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:07.650408 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45349.3us
I20260709 13:00:07.606668 21225 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:07.650806 20744 leader_election.cc:400] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:07.651786 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.657727 20744 leader_election.cc:403] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:07.652674 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:07.621083 21118 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:07.624482 21123 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:07.664888 20855 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
W20260709 13:00:07.669485 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 245162us
I20260709 13:00:07.673072 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.621673 21142 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:07.657840 21213 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:07.709270 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:07.702185 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 375394us
W20260709 13:00:07.706466 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60301.7us
W20260709 13:00:07.700474 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 196418us
I20260709 13:00:07.677162 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.631083 21182 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:00:07.628336 21311 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:00:07.557502 21321 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.656860 21124 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:07.624327 21127 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 13:00:07.721763 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 69151.8us
I20260709 13:00:07.637010 21232 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:00:07.735008 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.738245 20164 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:07.769483 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:07.740368 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:07.682688 21339 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.080s	user 0.000s	sys 0.002s
W20260709 13:00:07.756524 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 79484.9us
W20260709 13:00:07.761160 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 52012.5us
I20260709 13:00:07.650750 21325 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.708012 21080 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:07.707710 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:07.703754 21167 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:07.777534 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.721735 21337 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:07.687880 21230 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:07.729995 21073 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:07.720352 20157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:07.788909 21346 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:07.790421 21340 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:00:07.757535 21141 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.804746 21203 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:07.797248 21259 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.797832 21321 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:07.798673 21345 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:07.761015 21322 log.cc:927] Time spent T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:00:07.729310 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.800168 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.801702 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.801887 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:07.795571 21225 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:07.911371 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 125482us
W20260709 13:00:07.810508 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 46089.4us
I20260709 13:00:07.708462 21142 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.760838 21265 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.781997 21106 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:07.871104 20008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 120) took 1840 ms (client timeout 1677 ms). Trace:
I20260709 13:00:07.745997 21075 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:07.805757 21064 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.806262 21080 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:07.891963 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 130315us
I20260709 13:00:07.827350 21337 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.862967 21117 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:00:07.908221 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.979072 20855 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:07.921458 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:07.926239 20008 rpcz_store.cc:269] 0709 13:00:06.004700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:06.023989 (+ 19289us) service_pool.cc:224] Handling call
0709 13:00:07.002609 (+978620us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5800867998. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:07.826055 (+823446us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54000fa5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:07.826164 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038591104}
I20260709 13:00:07.920109 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:07.980558 20859 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:07.980679 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:07.958853 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 158761us
I20260709 13:00:07.879021 20157 raft_consensus.cc:606] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:07.881791 21213 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:07.899505 21259 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.831283 21327 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:07.777614 21326 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.972252 21337 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:07.968482 21265 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:07.973759 21351 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:07.974229 21365 consensus_queue.cc:1059] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:00:07.976011 21142 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:07.976740 21075 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:07.878623 21230 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:07.901896 21203 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:07.963363 21072 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:07.965570 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:07.966019 21345 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:08.082022 21345 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:07.978444 21331 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:00:08.082480 21345 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:08.041893 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 216251us
I20260709 13:00:08.083592 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:08.084784 21355 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:00:08.085367 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:08.055960 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.056959 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:08.089805 21075 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:08.093518 21326 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.094223 21230 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.096107 21325 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:08.098349 21156 consensus_meta.cc:287] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.760s	user 0.009s	sys 0.000s
I20260709 13:00:08.098853 21156 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:08.100121 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.986552 21073 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:00:08.101851 21203 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:08.072801 21351 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:08.060493 21225 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.106170 21351 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:08.106752 21351 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:07.969518 21311 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:00:08.109054 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.038826 21167 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:08.110935 21363 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.314s	user 0.006s	sys 0.000s
I20260709 13:00:08.111433 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.977830 21080 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:08.113009 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:08.113816 20850 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:08.114090 21080 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.072337 21071 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.115849 20164 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:08.116171 21145 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:08.118615 21213 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:08.038295 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:07.971115 21064 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.080590 20091 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:08.081030 20859 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:08.082139 21318 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:07.894685 21141 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.138541 21159 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.138965 21159 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:08.140125 21141 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.094517 20087 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:00:08.142576 21284 consensus_meta.cc:287] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.594s	user 0.007s	sys 0.003s
I20260709 13:00:08.143023 21284 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.143772 21064 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:08.145424 21259 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:08.066306 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 153341us
I20260709 13:00:08.110986 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.094195 21326 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:08.106192 21354 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:00:08.135830 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:08.163476 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:08.164723 20017 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:08.132685 21334 consensus_meta.cc:287] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.006s
I20260709 13:00:08.135110 21118 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:08.139741 21159 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:08.146344 21316 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:07.976070 21349 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.165597 21326 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:07.999650 21321 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:00:08.152680 21203 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.155860 20858 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:08.157723 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.158457 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.223703 20015 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:08.225391 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:08.226264 20015 raft_consensus.cc:2376] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4. Re-sending same reply.
I20260709 13:00:08.159317 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.158442 20056 leader_election.cc:400] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:08.128925 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:08.230624 20056 leader_election.cc:403] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:08.166673 21141 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.197885 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:08.231765 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 82146.2us
I20260709 13:00:08.231781 21141 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.198421 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.233459 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:08.198782 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 63071.1us
I20260709 13:00:08.106458 21225 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:08.192368 21328 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.662s	user 0.005s	sys 0.007s
I20260709 13:00:08.213438 20017 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:08.215595 20009 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:08.292634 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 54463us
I20260709 13:00:08.220111 21349 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:08.200843 21118 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:08.221427 21284 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:08.221684 21265 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.250341 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.304322 20015 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:08.194613 21303 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.314674 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.191210 21334 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.198998 21376 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.288604 21141 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:08.288769 21380 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:00:08.223418 21326 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:08.233492 21321 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:08.304876 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.305343 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:08.362835 20858 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 13:00:08.362074 21376 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:08.363168 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57952.4us
I20260709 13:00:08.364034 21373 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:08.364826 21378 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:08.365396 21378 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:08.317044 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:08.366441 21378 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:08.368526 21072 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:08.369155 21072 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:08.370131 21072 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.372920 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.310382 21156 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.308275 21377 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:00:08.286404 21270 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:00:08.350229 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:08.324399 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:08.301088 21159 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.306831 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.234844 21203 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:08.223167 21316 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.365331 20744 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:08.297331 21328 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:08.285372 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:08.383019 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:08.424063 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58782.4us
W20260709 13:00:08.355965 21257 consensus_meta.cc:287] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.787s	user 0.013s	sys 0.001s
I20260709 13:00:08.390678 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:08.365574 21376 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:08.444621 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:08.444708 20015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:08.445672 20015 raft_consensus.cc:606] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:08.426512 21284 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:08.381455 21346 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:08.426335 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.474800 20853 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
W20260709 13:00:08.428242 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 142959us
I20260709 13:00:08.428570 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:08.425954 21072 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:08.483610 21334 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:08.378459 21340 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:08.434916 21257 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:08.462919 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 654) took 1371 ms. Trace:
I20260709 13:00:08.428840 21120 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:08.465842 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:08.428354 21327 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:08.467641 21349 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.469074 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:08.529937 20085 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:08.471071 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.402819 20851 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:08.485731 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:08.485913 21346 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.465361 21316 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:08.495114 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66668.7us
I20260709 13:00:08.496836 20086 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:08.466428 20008 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:08.444615 21270 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:08.493180 21164 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:00:08.504905 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.494032 20165 rpcz_store.cc:276] 0709 13:00:07.094476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.153801 (+ 59325us) service_pool.cc:224] Handling call
0709 13:00:07.154082 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:07.154487 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:07.154515 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:07.154969 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 13:00:07.321708 (+166739us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:07.321749 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:07.321814 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:08.461422 (+1139608us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b24005ad148. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b703fb0 00007fa02e37a488 00007fa02e37abe8 00007fa02e352398 00007fa02e34d76a 00007fa02e34cbb6
0709 13:00:08.462629 (+  1207us) raft_consensus.cc:1685] finished
0709 13:00:08.462650 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:08.462858 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3168,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":67541376,"thread_start_us":46466,"threads_started":1,"wal-append.queue_time_us":272721}
I20260709 13:00:08.505512 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:08.505764 21340 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:08.557940 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.559406 21340 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.506690 21328 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.507422 21373 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:08.507834 21257 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.571983 20084 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:08.575271 21203 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.575947 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:08.509013 21382 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.119s	user 0.002s	sys 0.000s
I20260709 13:00:08.577891 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:08.579103 21164 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:08.579746 21164 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:08.559636 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.581874 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:08.581143 21164 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.512544 21385 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:08.513092 21370 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:00:08.582729 20157 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:08.582785 21385 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:08.585356 21385 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.586373 20160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:08.586977 20160 raft_consensus.cc:606] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.587373 20160 raft_consensus.cc:614] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:08.572585 21071 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:08.588361 21071 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:08.594650 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:08.595381 20015 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:08.597687 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:08.514206 21383 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:00:08.516191 21334 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:08.519749 21381 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:00:08.522303 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.524561 21118 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:08.526379 20082 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:08.527907 20158 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:08.541689 21270 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:08.489252 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:08.537720 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52054.2us
I20260709 13:00:08.494748 21284 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:08.636351 21284 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:08.496222 21120 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.640441 21120 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.513080 21225 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.540079 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:08.550315 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.555289 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.645169 21182 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:08.560235 21328 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:08.645624 21182 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.646257 21182 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.563570 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.473021 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 767) took 1492 ms. Trace:
I20260709 13:00:08.648336 20083 rpcz_store.cc:276] 0709 13:00:06.980675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:06.983115 (+  2440us) service_pool.cc:224] Handling call
0709 13:00:06.983472 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:07.585997 (+602525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.472754 (+886757us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400781c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:08.472934 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254615936}
W20260709 13:00:08.649091 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 85615.1us
I20260709 13:00:08.649952 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:08.602500 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:08.539288 21346 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:08.611394 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.608594 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:08.594295 20086 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:08.572374 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.609086 21064 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:08.608940 21177 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:08.615733 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.620323 21334 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:08.620939 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:08.542737 21377 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:08.657455 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.652443 21230 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:08.644695 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104713us
I20260709 13:00:08.651880 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:08.653258 20014 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:08.616289 21164 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.682374 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:08.683115 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.683468 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.684960 20017 raft_consensus.cc:606] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.512717 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.624529 21071 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:08.689092 20015 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
W20260709 13:00:08.680516 21345 consensus_meta.cc:287] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.598s	user 0.013s	sys 0.001s
I20260709 13:00:08.647605 21328 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.681425 21311 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.698336 21334 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.692730 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.702041 20157 raft_consensus.cc:2368] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:08.702337 21177 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.704880 21366 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:08.655213 21118 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.739140 21118 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.741918 21118 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.695323 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 555) took 1060 ms. Trace:
I20260709 13:00:08.744204 20854 rpcz_store.cc:276] 0709 13:00:07.635286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.637614 (+  2328us) service_pool.cc:224] Handling call
0709 13:00:08.122572 (+484958us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:08.122767 (+   195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:08.123203 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.123246 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:08.128273 (+  5027us) log.cc:844] Serialized 26 byte log entry
0709 13:00:08.225180 (+ 96907us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:08.225261 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.225396 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:08.624750 (+399354us) raft_consensus.cc:1685] finished
0709 13:00:08.624772 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:08.693287 (+ 68515us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":203,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":3434752,"thread_start_us":92566,"threads_started":1,"wal-append.queue_time_us":172847}
I20260709 13:00:08.745997 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.705716 20083 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:08.725662 21311 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:08.749922 21311 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.710034 20017 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:08.750527 20017 pending_rounds.cc:85] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 1
I20260709 13:00:08.750767 20017 pending_rounds.cc:107] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:08.751019 20017 raft_consensus.cc:2889] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:08.631035 21270 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:08.705886 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 137) took 1468 ms. Trace:
I20260709 13:00:08.756979 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.716634 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:08.759045 20015 raft_consensus.cc:2368] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:08.717573 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:08.689077 21377 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:08.688911 21385 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.690305 21069 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:08.689489 21064 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:08.732281 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:08.734993 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:08.725401 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:08.599350 21373 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:08.736996 21177 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:08.738536 21124 raft_consensus.cc:993] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:08.743278 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:08.707654 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.751149 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.790007 20855 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:08.761842 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:08.686479 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.712589 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:08.724614 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:08.777042 20015 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:08.755224 20087 rpcz_store.cc:276] 0709 13:00:07.237851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.240788 (+  2937us) service_pool.cc:224] Handling call
0709 13:00:08.705547 (+1464759us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b540020e4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:08.705797 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790332288}
I20260709 13:00:08.715579 21345 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:08.792600 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106170us
I20260709 13:00:08.754915 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:08.764317 21127 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:08.813917 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 89412.3us
I20260709 13:00:08.816236 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.837018 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:08.848044 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.848634 21072 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:08.849205 21072 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:08.851114 21327 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.851593 21072 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:08.851890 21327 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:08.850404 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:08.855731 21230 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.810966 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:08.859877 20056 leader_election.cc:400] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:08.910442 20056 leader_election.cc:403] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:08.781134 21377 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:08.784797 21069 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:08.912107 21069 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:08.913215 21069 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:08.809897 21328 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.864158 20084 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:00:08.966449 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106748us
I20260709 13:00:08.833721 21310 raft_consensus.cc:1081] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:08.963356 20016 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:08.809652 21177 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:09.012215 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 333235us
I20260709 13:00:09.013608 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:08.965018 20009 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:09.025115 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:09.025516 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 423476us
W20260709 13:00:09.023353 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 345082us
I20260709 13:00:08.997434 21203 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:08.997769 21378 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:08.861076 21326 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:09.026415 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:09.091856 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 65524.7us
W20260709 13:00:08.967386 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 278824us
I20260709 13:00:08.976061 21349 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:08.872100 21127 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.031095 20743 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.030155 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:09.005622 21346 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:09.123153 21393 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:00:09.124080 21389 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:00:09.188812 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 157800us
I20260709 13:00:09.124342 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:09.189476 21159 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.189648 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:09.189839 21346 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:09.125062 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:09.191027 21346 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.191330 20156 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:09.188441 21399 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:09.125461 21064 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:09.193917 21127 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:09.124351 21265 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:09.194631 21399 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:09.195462 21265 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:09.196239 21265 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:09.197374 21373 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.126477 21378 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.198138 21399 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:09.200031 21378 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.127377 20087 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:09.129349 21177 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.082129 21071 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:00:09.134776 21213 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.200917 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:09.084903 21310 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:09.201612 20016 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:09.201499 21213 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:09.202168 21310 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:09.203135 21310 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.136186 20084 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:09.023772 21338 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:09.180622 21337 consensus_meta.cc:287] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.104s	user 0.012s	sys 0.003s
W20260709 13:00:09.188396 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 158321us
I20260709 13:00:09.212965 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:09.191136 21397 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.177379 21124 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:09.174893 21327 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:09.184505 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:09.210108 21338 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:09.235016 21391 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:00:09.259958 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:09.270268 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 57391.3us
I20260709 13:00:09.271320 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:09.272233 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44798.7us
I20260709 13:00:09.283833 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.286024 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.247080 21213 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.301334 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.301438 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.228902 21337 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.301602 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.263859 21338 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:09.260944 21334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.021s	sys 0.052s
W20260709 13:00:09.324012 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:09.250633 21327 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:09.325584 21334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.021s	sys 0.052s
I20260709 13:00:09.300987 21385 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:09.330528 20857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
W20260709 13:00:09.330716 21120 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.006s
I20260709 13:00:09.246752 21124 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.332311 21397 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.296932 21156 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.343441 20011 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:09.323652 21159 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.297724 21398 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:09.298874 21377 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:09.357599 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 168027us
I20260709 13:00:09.349030 21345 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:09.364920 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 254225us
I20260709 13:00:09.360872 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.352401 20857 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:09.364025 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:09.386835 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:09.388410 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.356217 21156 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:09.398082 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.400771 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.405998 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:09.406996 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.404228 21345 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:09.415669 21164 consensus_meta.cc:287] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.003s
I20260709 13:00:09.352646 21120 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.418838 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:09.384660 21398 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.387521 21349 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.392472 20857 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.423301 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:09.424140 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.403853 21177 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.363808 21159 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:09.438360 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.438964 21159 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.404757 21156 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:09.440136 21407 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:09.441977 21407 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:09.439644 21373 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.432524 20856 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:09.406337 21399 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:09.434183 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 106101us
W20260709 13:00:09.421427 21182 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.775s	user 0.016s	sys 0.000s
I20260709 13:00:09.425885 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.439548 20056 leader_election.cc:400] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3.
I20260709 13:00:09.451471 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.455001 20014 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:09.456007 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.422586 21064 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.358276 21397 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:09.422771 21257 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:09.460991 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55831.5us
I20260709 13:00:09.465548 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:09.468134 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.424198 21349 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:09.471354 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:09.479346 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:09.480118 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:09.477836 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.482286 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.475294 20158 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:09.489928 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.494495 20014 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:09.460812 21064 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:09.461233 20016 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:09.519202 20013 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:09.475566 21349 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:09.475986 21334 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:09.476142 21230 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.526221 20743 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.531551 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.465512 21378 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:00:09.460878 21182 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.424887 21398 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.505926 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:09.475219 21257 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.536489 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:09.551118 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.551198 20014 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:09.508257 21064 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:09.514756 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:09.533598 21123 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.545104 21357 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.579550 21259 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.544620 21337 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:09.580083 21404 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.164s	user 0.000s	sys 0.002s
I20260709 13:00:09.579963 21177 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.581532 21397 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:09.584162 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 53022.8us
I20260709 13:00:09.584637 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:09.587165 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 61169.7us
I20260709 13:00:09.587656 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 776) took 1072 ms. Trace:
I20260709 13:00:09.607175 20087 rpcz_store.cc:276] 0709 13:00:08.515040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.833896 (+318856us) service_pool.cc:224] Handling call
0709 13:00:08.834274 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.200836 (+366562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.587331 (+386495us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54009803d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:09.587580 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673603840}
I20260709 13:00:09.608697 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:09.567607 21407 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.614033 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:09.577075 21327 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.573261 21164 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:09.601840 21334 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.590633 21124 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.647351 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:09.665362 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59590.6us
I20260709 13:00:09.665871 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:09.667125 21327 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.603848 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.644155 20082 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:09.606037 20852 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:09.644307 21377 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.681166 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:09.683198 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:09.645005 21345 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:09.689975 20158 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:09.685747 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:09.702898 20855 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:09.679263 19982 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.687168 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 779) took 1171 ms. Trace:
I20260709 13:00:09.687987 21424 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:09.688786 21409 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:00:09.700403 21386 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.077s	user 0.006s	sys 0.021s
W20260709 13:00:09.673700 21406 log.cc:927] Time spent T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.269s	user 0.000s	sys 0.007s
I20260709 13:00:09.602526 21357 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.705632 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:09.706630 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:09.719547 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 296288us
W20260709 13:00:09.769503 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 63959.6us
I20260709 13:00:09.670344 21432 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:09.731806 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 124224us
W20260709 13:00:09.702929 21270 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.950s	user 0.016s	sys 0.000s
I20260709 13:00:09.689277 21120 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.704541 20088 rpcz_store.cc:276] 0709 13:00:08.515851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.838561 (+322710us) service_pool.cc:224] Handling call
0709 13:00:08.839001 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:08.848657 (+  9656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.848801 (+   144us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:08.850938 (+  2137us) log.cc:844] Serialized 26 byte log entry
0709 13:00:09.299050 (+448112us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:09.299090 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.299227 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:09.686899 (+387672us) raft_consensus.cc:1685] finished
0709 13:00:09.686922 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.687107 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7011,"raft.queue_time_us":7387,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"thread_start_us":446665,"threads_started":1,"wal-append.queue_time_us":514055}
W20260709 13:00:09.772018 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 92811.8us
I20260709 13:00:09.782369 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:09.783317 20743 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.673230 21164 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.778683 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:09.784763 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:09.782910 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:09.771759 20016 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:09.716566 21424 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:09.790156 21417 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:00:09.790556 21437 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:00:09.792593 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:09.793365 21156 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.794593 21338 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.795174 21123 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.681923 21327 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:09.795955 20015 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:09.786129 21213 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:09.802366 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:09.797966 21177 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.803053 21440 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:09.780584 21075 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:09.805740 21257 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:09.808342 21434 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:09.819121 21434 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:09.820783 21265 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.005s
I20260709 13:00:09.821409 21265 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:09.822559 21373 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:09.823117 21373 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:09.671271 21407 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:09.824177 20008 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:09.808580 21398 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.825897 21349 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.826351 21407 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:09.796520 21424 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.829941 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:09.830876 20854 raft_consensus.cc:2368] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:09.832923 21375 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.833500 21375 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:09.834080 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:09.837682 21349 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.838641 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 142) took 1653 ms. Trace:
I20260709 13:00:09.839035 20011 rpcz_store.cc:276] 0709 13:00:08.185531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.195042 (+  9511us) service_pool.cc:224] Handling call
0709 13:00:08.195361 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.356736 (+1161375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.838409 (+481673us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54005f2154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:09.838582 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2639467392}
I20260709 13:00:09.782739 21270 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.840559 20091 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:09.840600 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:09.841724 20164 raft_consensus.cc:2393] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:09.842058 21270 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.844071 21326 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.781992 21346 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.808827 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 592) took 1296 ms. Trace:
I20260709 13:00:09.817792 20087 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:09.852015 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:09.857986 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44724.8us
I20260709 13:00:09.859321 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.853816 20163 rpcz_store.cc:276] 0709 13:00:08.528534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.622113 (+ 93579us) service_pool.cc:224] Handling call
0709 13:00:08.622507 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:08.622865 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.622897 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:08.626389 (+  3492us) log.cc:844] Serialized 26 byte log entry
0709 13:00:08.862378 (+235989us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b1800030ac0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:00:09.257766 (+395388us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:09.257807 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.257874 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:09.808411 (+550537us) raft_consensus.cc:1685] finished
0709 13:00:09.808441 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.808752 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2785,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":25429,"spinlock_wait_cycles":5498112,"thread_start_us":394415,"threads_started":1,"wal-append.queue_time_us":734299}
I20260709 13:00:09.795907 21156 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.801195 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:09.815188 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.781546 21432 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.805271 21164 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.807088 21357 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:09.809265 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:09.873488 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.929216 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:09.931597 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:09.939236 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65831.4us
I20260709 13:00:09.863215 21346 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:09.872053 21120 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.943877 21346 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:09.944449 20857 consensus_meta.cc:287] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.003s
I20260709 13:00:09.944257 21120 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:09.944880 20857 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:09.946492 21120 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:09.815567 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 144) took 1619 ms. Trace:
I20260709 13:00:09.941757 21270 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:09.947402 21270 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:09.947978 21270 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:09.808074 20010 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:09.956159 20012 rpcz_store.cc:276] 0709 13:00:08.196074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.220081 (+ 24007us) service_pool.cc:224] Handling call
0709 13:00:08.220537 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:08.292459 (+ 71922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.296526 (+  4067us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:08.362061 (+ 65535us) log.cc:844] Serialized 26 byte log entry
0709 13:00:08.504642 (+142581us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:08.504931 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.505044 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:09.699758 (+1194714us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:09.815214 (+115456us) raft_consensus.cc:1685] finished
0709 13:00:09.815243 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.815477 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39139,"raft.queue_time_us":71767,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":53067,"spinlock_wait_cycles":12811136,"thread_start_us":134173,"threads_started":1,"wal-append.queue_time_us":197916}
I20260709 13:00:09.958423 20858 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:09.958907 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:09.959161 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 491) took 1238 ms. Trace:
I20260709 13:00:09.898051 21338 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:09.894645 21327 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:09.960408 20016 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:09.909042 21071 raft_consensus.cc:993] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:09.960875 21385 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:09.961263 21071 raft_consensus.cc:1081] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:09.961540 21385 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:09.962502 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:09.900880 21418 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:09.964253 21432 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.964577 21418 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:09.965792 21418 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.959816 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:09.942145 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:09.816496 21440 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:09.817044 21182 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:09.815160 21306 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:09.943200 21435 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:09.864590 21434 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:09.896601 21427 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:09.970062 20858 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:09.951144 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:09.974516 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 187264us
I20260709 13:00:09.864118 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:09.990623 20855 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:09.872289 21127 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:09.990558 21424 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.992228 21435 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:09.996382 21127 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:09.810075 21213 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:09.947045 20857 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:09.997639 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.896530 21064 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:10.001080 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 141854us
I20260709 13:00:10.000430 20082 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:10.014945 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.001991 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:10.006745 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 781) took 1490 ms. Trace:
I20260709 13:00:09.959550 20010 rpcz_store.cc:276] 0709 13:00:08.721108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.734699 (+ 13591us) service_pool.cc:224] Handling call
0709 13:00:09.958965 (+1224266us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400567f54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:09.959100 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476461440}
I20260709 13:00:09.999153 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:09.866158 21373 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:10.021091 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.026973 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:10.029062 19981 leader_election.cc:400] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:10.029367 19981 leader_election.cc:403] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:10.005196 21257 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:00:10.030221 21435 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:10.030326 21182 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.031190 21440 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:10.005942 21120 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.024345 21327 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.035197 21424 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:09.996229 21118 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:00:10.061273 21327 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.062157 21118 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:10.062582 21118 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:10.062503 21338 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:10.063421 21445 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:10.037964 21064 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:10.038902 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:10.072993 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:10.073608 20082 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.038924 21127 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:10.022744 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:10.031980 21310 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:10.077469 21310 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:10.077991 21310 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:10.016844 21429 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.040395 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.084189 20858 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:10.085608 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.005429 21365 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:00:10.088119 21365 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:10.088603 21365 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:10.041016 20851 raft_consensus.cc:2368] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:10.044051 20163 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:10.092796 20163 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:10.093981 20163 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.095003 20163 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.097177 20858 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:10.045634 21213 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.134768 21385 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:10.138180 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 83000.1us
I20260709 13:00:10.140003 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.053076 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.058033 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.142246 20158 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:10.055992 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:10.039467 21413 log.cc:927] Time spent T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:00:10.059571 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:10.068200 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.068269 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.068804 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.045436 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:10.140223 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.160046 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:10.154047 21264 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:10.159333 20159 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.164558 21270 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:10.168220 21064 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:10.168552 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.168706 21064 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:10.169157 21064 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:10.169399 21328 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.169808 21328 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:10.170179 21328 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.156196 21365 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.174203 21365 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.177168 20014 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:10.023411 20083 rpcz_store.cc:276] 0709 13:00:08.516368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.749161 (+232793us) service_pool.cc:224] Handling call
0709 13:00:09.941961 (+1192800us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:10.003230 (+ 61269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13565312}
I20260709 13:00:10.048331 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.146543 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:10.150710 20089 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 773) took 1637 ms (client timeout 2179 ms). Trace:
I20260709 13:00:10.063881 21118 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:10.180267 21407 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:10.181048 21407 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:10.181041 21118 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.181951 21182 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:10.182092 21407 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.182492 21182 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.183224 21327 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:10.184414 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 151) took 1005 ms. Trace:
I20260709 13:00:10.184669 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.184851 20008 rpcz_store.cc:276] 0709 13:00:09.178639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.178850 (+   211us) service_pool.cc:224] Handling call
0709 13:00:09.179447 (+   597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.824661 (+645214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.184157 (+359496us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54000faad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:10.184347 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482246400}
W20260709 13:00:10.185714 21430 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.497s	user 0.001s	sys 0.000s
I20260709 13:00:10.186503 21414 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.187160 21399 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.187804 21399 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:10.189198 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.189994 21399 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:10.191161 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.192924 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:10.152051 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 96203.1us
I20260709 13:00:10.055078 21432 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:10.151583 20013 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
W20260709 13:00:10.196120 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 147859us
I20260709 13:00:10.200663 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:10.198768 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.205652 20082 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:10.205557 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.206014 21346 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.206858 21069 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.190272 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.209800 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.066208 21120 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:10.179968 20089 rpcz_store.cc:269] 0709 13:00:08.513832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.524071 (+ 10239us) service_pool.cc:224] Handling call
0709 13:00:10.139805 (+1615734us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:10.150648 (+ 10843us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6686976}
I20260709 13:00:10.217273 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:10.217975 20089 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.160491 20852 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:10.214393 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.219360 20854 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:10.222205 21310 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:10.218933 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 43772.3us
I20260709 13:00:10.220419 20858 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.221052 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.228334 20014 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:10.243468 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.190543 21399 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:10.266913 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.268545 20163 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:10.269083 20129 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:10.244910 20010 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 2: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:10.221871 20857 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:10.219655 20082 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:10.282809 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.287252 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.229403 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.278323 20089 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:10.289921 20083 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 1.
I20260709 13:00:10.272104 21257 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:10.230113 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.221390 21328 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.221318 21429 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.275499 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:10.305780 19981 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:10.266896 21432 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.294453 21264 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:10.318015 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:10.187312 21414 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.320835 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:10.326407 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.297688 21270 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:10.335786 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.338035 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:10.282457 21418 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:00:10.310170 21429 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:10.343967 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.344889 20161 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.317600 21338 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.348893 21338 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:10.348961 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.349429 21338 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.349653 20165 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:10.274391 21399 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.316531 21346 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:10.362227 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.274969 21357 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:00:10.305045 21447 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:10.380484 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.382215 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.384886 20858 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:10.399513 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.457221 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.375190 21447 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:10.459293 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.378705 20161 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:10.441066 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.347004 21429 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.360276 21182 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:10.453722 20744 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:10.390306 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:10.425231 20083 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:10.425707 20157 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:10.484987 21120 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:10.487418 20086 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:10.488085 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 775) took 1973 ms. Trace:
I20260709 13:00:10.488394 20086 rpcz_store.cc:276] 0709 13:00:08.514789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.753005 (+238216us) service_pool.cc:224] Handling call
0709 13:00:10.487028 (+1734023us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:10.487250 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.487858 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.488033 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6363008}
I20260709 13:00:10.489794 20086 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:10.490078 21442 log.cc:927] Time spent T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:00:10.490693 21385 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:10.466799 21447 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:10.493459 21265 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:10.497095 21414 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:10.470786 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.472384 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:10.476092 21213 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.477013 21069 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:10.477885 21427 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.477945 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.379379 21346 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:10.374704 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.384111 21064 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.508687 21410 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:10.509130 21213 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:10.509569 21069 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:10.509640 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.510792 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:10.511760 21425 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.841s	user 0.005s	sys 0.007s
I20260709 13:00:10.512287 21455 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.500036 21310 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:10.528256 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.527700 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.528589 21213 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.531117 21410 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:10.526861 21346 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:10.546407 21443 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 13:00:10.549942 21399 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.550575 21410 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.538260 21455 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.555404 21447 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.556668 21455 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.557981 21447 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.594123 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 784) took 1122 ms. Trace:
I20260709 13:00:10.596951 20014 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:10.597805 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.599516 20014 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:10.600801 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.361778 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:10.595228 20017 rpcz_store.cc:276] 0709 13:00:09.471030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.471294 (+   264us) service_pool.cc:224] Handling call
0709 13:00:09.471583 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:09.478923 (+  7340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.478953 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:09.496515 (+ 17562us) log.cc:844] Serialized 26 byte log entry
0709 13:00:09.544100 (+ 47585us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:09.544143 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.544240 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:10.510769 (+966529us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:10.592844 (+ 82075us) raft_consensus.cc:1685] finished
0709 13:00:10.592947 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.593720 (+   773us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48225,"raft.queue_time_us":25939,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":10536,"spinlock_wait_cycles":65404672,"thread_start_us":47205,"threads_started":1,"wal-append.queue_time_us":88748}
I20260709 13:00:10.501859 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:10.409785 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.617166 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:10.625411 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.626387 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:10.638250 21310 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.638921 21397 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.637254 21069 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.642329 21064 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.642879 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 560) took 1212 ms. Trace:
I20260709 13:00:10.627813 20852 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:10.556834 21346 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:10.643863 20859 rpcz_store.cc:276] 0709 13:00:09.430237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.865354 (+435117us) service_pool.cc:224] Handling call
0709 13:00:09.865988 (+   634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:09.866353 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.866387 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:09.872155 (+  5768us) log.cc:844] Serialized 26 byte log entry
0709 13:00:10.015216 (+143061us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:10.015256 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.015323 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:10.642220 (+626897us) raft_consensus.cc:1685] finished
0709 13:00:10.642254 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.642561 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29786,"raft.queue_time_us":6230,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":22077,"thread_start_us":120252,"threads_started":1,"wal-append.queue_time_us":242524}
I20260709 13:00:10.647338 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.627526 20010 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:10.648639 20010 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:10.649766 20089 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:00:10.650635 21447 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:10.648345 21338 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:10.652268 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.652500 21447 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.653393 20857 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:10.654870 21338 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:10.651535 20089 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:10.656103 20089 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.643509 21064 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:00:10.657348 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.657361 20087 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:10.662317 20055 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:10.664219 21378 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.664565 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.651264 20010 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.667709 20010 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.675990 21069 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.677757 21467 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.679092 21455 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:10.679982 21455 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:10.680361 21455 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.682089 20012 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:10.686664 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.655588 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.690279 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.690506 21447 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.693689 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.700063 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.697216 20082 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:10.704588 21307 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.691211 20857 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:10.710816 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:10.716930 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52817.6us
I20260709 13:00:10.722045 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.699101 21467 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:10.721604 21455 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.730192 20164 raft_consensus.cc:2368] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6.
I20260709 13:00:10.732780 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:10.739159 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:10.740885 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.741796 21230 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:10.742391 21334 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:10.742703 21230 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.743717 21334 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.744647 21334 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:10.744752 20163 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:10.745173 21334 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:10.745353 20163 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:10.748169 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.748070 21467 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:10.749488 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.746798 21334 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.737792 21447 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.750519 20156 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:10.751554 21334 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:10.751590 21467 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:10.752591 21230 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.764961 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 569) took 1300 ms. Trace:
I20260709 13:00:10.754050 20744 leader_election.cc:400] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6.
I20260709 13:00:10.746901 20163 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.771600 20744 leader_election.cc:403] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:10.758883 21257 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:10.760447 20082 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:10.775673 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:10.776758 20085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 777) took 2261 ms (client timeout 1724 ms). Trace:
W20260709 13:00:10.777860 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 91547.5us
I20260709 13:00:10.778299 21257 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:10.778857 21257 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:10.778993 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.771431 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.770857 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.752319 21447 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:10.770377 21459 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.771463 20854 rpcz_store.cc:276] 0709 13:00:09.464327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.831597 (+367270us) service_pool.cc:224] Handling call
0709 13:00:10.764495 (+932898us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54000069d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:10.764722 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495250304}
I20260709 13:00:10.774806 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:10.777196 20085 rpcz_store.cc:269] 0709 13:00:08.515352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.532000 (+ 16648us) service_pool.cc:224] Handling call
0709 13:00:09.178444 (+646444us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:10.775430 (+1596986us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68df8c 00007fa02b68def6 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:10.776645 (+  1215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17728000}
W20260709 13:00:10.783898 21284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.992s	user 0.005s	sys 0.027s
I20260709 13:00:10.784430 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:10.785166 20163 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.763114 21378 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.787650 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.796259 21168 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:10.798501 21141 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.797458 20089 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:10.791605 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.807163 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.808867 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 669) took 1525 ms. Trace:
I20260709 13:00:10.809501 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.813014 21459 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.809309 20162 rpcz_store.cc:276] 0709 13:00:09.283144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.284456 (+  1312us) service_pool.cc:224] Handling call
0709 13:00:10.783877 (+1499421us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:10.808604 (+ 24727us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37273984}
I20260709 13:00:10.796782 21168 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:10.812529 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:10.816746 21168 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.818687 21168 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:10.818979 21123 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:10.824061 21230 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:10.825030 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 115942us
I20260709 13:00:10.826290 21375 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:10.826960 21422 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.140s	user 0.003s	sys 0.000s
I20260709 13:00:10.827229 21345 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:10.828709 21177 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:10.819459 21123 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:10.832630 21467 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:10.797293 21284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.011s	user 0.006s	sys 0.027s
I20260709 13:00:10.833164 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 565) took 1397 ms. Trace:
W20260709 13:00:10.836748 20159 consensus_meta.cc:287] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.565s	user 0.018s	sys 0.000s
I20260709 13:00:10.828303 21345 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.837646 20159 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
W20260709 13:00:10.834605 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 97106us
I20260709 13:00:10.836921 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 564) took 1405 ms. Trace:
I20260709 13:00:10.825186 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.830363 21123 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:10.849196 20084 raft_consensus.cc:2410] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:10.786312 21213 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.844135 21230 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:10.860662 20089 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:10.839799 20852 rpcz_store.cc:276] 0709 13:00:09.431486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.220716 (+789230us) service_pool.cc:224] Handling call
0709 13:00:10.220985 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.644868 (+423883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.835723 (+190855us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400006ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:10.836815 (+  1092us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711364992}
I20260709 13:00:10.864444 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:10.865437 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:10.865590 20085 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:10.866384 20852 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:10.866837 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 572) took 1355 ms. Trace:
I20260709 13:00:10.867321 20855 rpcz_store.cc:276] 0709 13:00:09.511697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.777388 (+265691us) service_pool.cc:224] Handling call
0709 13:00:10.866441 (+1089053us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54002409d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:10.866642 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411887872}
I20260709 13:00:10.861783 21230 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:10.869858 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.869768 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 566) took 1434 ms. Trace:
I20260709 13:00:10.871739 21230 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:10.876696 21455 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:10.877507 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 98681.7us
I20260709 13:00:10.878896 21213 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:10.842219 21257 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:10.880754 20858 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:10.870765 20855 rpcz_store.cc:276] 0709 13:00:09.435579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.868239 (+1432660us) service_pool.cc:224] Handling call
0709 13:00:10.868822 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.869159 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.869183 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.869456 (+   273us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:10.869516 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.869574 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.869697 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:10.886968 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.887967 21345 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:10.857163 21378 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:10.835640 20854 rpcz_store.cc:276] 0709 13:00:09.435349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.818424 (+1383075us) service_pool.cc:224] Handling call
0709 13:00:10.818983 (+   559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.819481 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.819517 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.819595 (+    78us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:10.819843 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.832911 (+ 13068us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.833083 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.900542 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:10.900879 21123 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.902036 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:10.902153 21437 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:10.903225 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:10.903954 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 781) took 1449 ms. Trace:
I20260709 13:00:10.907819 20085 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:10.908342 20163 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:10.920712 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:10.920403 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:10.936497 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50821.2us
I20260709 13:00:10.922611 20015 rpcz_store.cc:276] 0709 13:00:09.454447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.455316 (+   869us) service_pool.cc:224] Handling call
0709 13:00:09.455576 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.803039 (+347463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.903370 (+1100331us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54000f9bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:10.903746 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782290688}
I20260709 13:00:10.922254 21437 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:10.938867 21284 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:10.943742 19982 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:10.961781 21429 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:10.963729 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 568) took 1506 ms. Trace:
I20260709 13:00:10.964267 20090 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:10.964241 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 590) took 1465 ms. Trace:
W20260709 13:00:10.969439 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 144438us
I20260709 13:00:10.949651 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:10.971333 20130 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:10.969799 20855 rpcz_store.cc:276] 0709 13:00:09.498691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.954855 (+1456164us) service_pool.cc:224] Handling call
0709 13:00:10.963493 (+  8638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.963827 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.963861 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.963910 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:10.963972 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.964026 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.964150 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.952303 20010 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:10.969218 21257 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.977558 20009 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:10.985745 20015 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:10.977694 21375 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:10.992311 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:10.969188 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:10.997972 21437 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:10.998769 20859 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:10.996877 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:11.001822 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 571) took 1496 ms. Trace:
I20260709 13:00:11.006422 20859 rpcz_store.cc:276] 0709 13:00:09.503810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.996965 (+1493155us) service_pool.cc:224] Handling call
0709 13:00:10.998379 (+  1414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.999793 (+  1414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.000095 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.008128 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.010788 20013 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:10.964336 20851 rpcz_store.cc:276] 0709 13:00:09.457419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.798085 (+340666us) service_pool.cc:224] Handling call
0709 13:00:10.962958 (+1164873us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54000050d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:10.963525 (+   567us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231006080}
I20260709 13:00:10.995550 20012 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 1.
I20260709 13:00:11.013976 20014 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:11.015022 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 151609us
I20260709 13:00:10.999763 21437 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:11.002872 20850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.144s	user 0.015s	sys 0.039s
I20260709 13:00:11.019017 20857 raft_consensus.cc:2410] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:11.008615 21177 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6.
I20260709 13:00:11.018864 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.019193 21467 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:11.022328 20850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.016s	sys 0.040s
I20260709 13:00:11.043255 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:11.044355 20086 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:11.029407 20854 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:11.029595 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 498) took 1159 ms. Trace:
I20260709 13:00:11.059007 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 591) took 1555 ms. Trace:
I20260709 13:00:11.033972 21447 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.045421 21378 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:11.065326 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:11.065421 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 158950us
I20260709 13:00:11.060874 20855 rpcz_store.cc:276] 0709 13:00:09.499095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.985107 (+1486012us) service_pool.cc:224] Handling call
0709 13:00:11.021393 (+ 36286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.032983 (+ 11590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:11.033001 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:11.053755 (+ 20754us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:11.053792 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.054366 (+   574us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.054485 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.073662 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.084518 20858 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:11.089069 20858 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.089517 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 586) took 1760 ms. Trace:
I20260709 13:00:11.090009 20853 rpcz_store.cc:276] 0709 13:00:09.329278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.330452 (+  1174us) service_pool.cc:224] Handling call
0709 13:00:09.928990 (+598538us) spinlock_profiling.cc:243] Waited 9.88 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:11.089064 (+1160074us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400240254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:11.089328 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781882496}
I20260709 13:00:11.090701 21412 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:11.093441 21259 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.095010 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:11.096309 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 155) took 1323 ms. Trace:
I20260709 13:00:11.096985 20088 rpcz_store.cc:276] 0709 13:00:09.773013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.784642 (+ 11629us) service_pool.cc:224] Handling call
0709 13:00:11.094670 (+1310028us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:11.096233 (+  1563us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7067392}
I20260709 13:00:11.064402 21447 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:11.109295 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 120192us
W20260709 13:00:11.110919 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47654.2us
I20260709 13:00:11.061352 20016 rpcz_store.cc:276] 0709 13:00:09.869128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.953084 (+1083956us) service_pool.cc:224] Handling call
0709 13:00:11.028412 (+ 75328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.076663 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 567) took 1636 ms. Trace:
I20260709 13:00:11.121481 21447 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:11.122435 21328 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.114271 21069 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:11.124243 21069 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:11.124644 21069 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:11.126564 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 788) took 1650 ms. Trace:
I20260709 13:00:11.127197 20084 rpcz_store.cc:276] 0709 13:00:09.476185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.483627 (+  7442us) service_pool.cc:224] Handling call
0709 13:00:10.774421 (+1290794us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:11.126268 (+351847us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b54005a1254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:11.126484 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91289728}
I20260709 13:00:11.128752 20855 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.128901 21328 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:11.129647 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 577) took 1115 ms. Trace:
W20260709 13:00:11.131471 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.133294 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:11.134968 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.135531 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:11.135900 21470 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.178s	user 0.000s	sys 0.002s
I20260709 13:00:11.137187 21159 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.137864 20856 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:11.139154 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.139600 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 499) took 1270 ms. Trace:
I20260709 13:00:11.140013 20851 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:11.140635 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 559) took 1711 ms. Trace:
I20260709 13:00:11.141045 20856 rpcz_store.cc:276] 0709 13:00:09.429691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.171576 (+741885us) service_pool.cc:224] Handling call
0709 13:00:11.093377 (+921801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.138378 (+ 45001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.140427 (+  2049us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b5400240250. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:11.140576 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116781824}
I20260709 13:00:11.141484 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.142231 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:11.142418 20851 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:11.144349 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.145846 21412 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:11.146716 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.145846 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.148108 20851 raft_consensus.cc:2410] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:00:11.148525 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 131976us
I20260709 13:00:11.140085 20008 rpcz_store.cc:276] 0709 13:00:09.869377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.185914 (+316537us) service_pool.cc:224] Handling call
0709 13:00:10.767084 (+581170us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b5800867998. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:10.767582 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:10.773244 (+  5662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.773279 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:10.774013 (+   734us) log.cc:844] Serialized 26 byte log entry
0709 13:00:10.838968 (+ 64955us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:10.839007 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.839189 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:11.139090 (+299901us) raft_consensus.cc:1685] finished
0709 13:00:11.139112 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.139391 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3820,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":117,"spinlock_wait_cycles":14256384,"thread_start_us":64525,"threads_started":1,"wal-append.queue_time_us":122563}
I20260709 13:00:11.150497 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.151263 21213 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.151875 21213 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.154422 21213 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:11.156002 21284 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.156643 21284 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:11.157557 21471 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.152s	user 0.000s	sys 0.002s
I20260709 13:00:11.158632 21373 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.159690 21284 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:11.159742 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:11.130059 20855 rpcz_store.cc:276] 0709 13:00:10.014720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.109877 (+1095157us) service_pool.cc:224] Handling call
0709 13:00:11.128461 (+ 18584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.129364 (+   903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.129571 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.163647 20085 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
W20260709 13:00:11.165870 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 215205us
I20260709 13:00:11.173354 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.184947 21408 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.185031 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 557) took 1756 ms. Trace:
I20260709 13:00:11.185719 21408 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.189548 21337 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:11.190030 21337 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:11.190433 21337 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:11.192427 21378 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.192852 21378 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.193199 21378 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.197962 20091 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:11.199357 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 156) took 1426 ms. Trace:
I20260709 13:00:11.199735 20091 rpcz_store.cc:276] 0709 13:00:09.773459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.226391 (+452932us) service_pool.cc:224] Handling call
0709 13:00:10.946937 (+720546us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:11.172309 (+225372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.198506 (+ 26197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.199087 (+   581us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400781250. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:11.199274 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519350400}
I20260709 13:00:11.166772 21177 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:11.200810 21159 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.160164 20854 rpcz_store.cc:276] 0709 13:00:09.435753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.943237 (+1507484us) service_pool.cc:224] Handling call
0709 13:00:10.987338 (+ 44101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.058269 (+ 70931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.071976 (+ 13707us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.180755 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:11.203446 21431 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.223s	user 0.001s	sys 0.005s
I20260709 13:00:11.208010 21072 consensus_queue.cc:1059] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:11.208815 20130 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:11.208228 21424 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.180090 21447 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:11.185598 20850 rpcz_store.cc:276] 0709 13:00:09.428750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.434136 (+  5386us) service_pool.cc:224] Handling call
0709 13:00:09.510209 (+ 76073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:09.565022 (+ 54813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.576520 (+ 11498us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:09.675389 (+ 98869us) log.cc:844] Serialized 26 byte log entry
0709 13:00:10.864763 (+1189374us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02b6a80ac 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b
0709 13:00:11.080450 (+215687us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:11.112651 (+ 32201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.155754 (+ 43103us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:11.168603 (+ 12849us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:11.170483 (+  1880us) raft_consensus.cc:1685] finished
0709 13:00:11.170495 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.184924 (+ 14429us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1625,"raft.queue_time_us":58273,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":28021,"spinlock_wait_cycles":19132928,"thread_start_us":1240770,"threads_started":1,"wal-append.queue_time_us":1254437}
I20260709 13:00:11.216140 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.222239 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:11.221066 20744 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:11.234512 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 64574.1us
I20260709 13:00:11.227411 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.234059 21424 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.221750 21159 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.239828 21295 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:11.250291 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.241909 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:11.214656 21447 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:11.258249 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 79397.5us
I20260709 13:00:11.249322 20856 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:11.260593 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.248775 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.262938 20855 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.262498 21064 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:11.264285 21064 raft_consensus.cc:697] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:11.265479 21159 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.248555 21069 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:11.259625 21123 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.250090 21295 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.268254 21069 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:11.244169 21462 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:11.269186 21462 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.269531 21462 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.241587 21127 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.273257 21259 raft_consensus.cc:3055] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:11.263878 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:11.276264 20856 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.259893 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:11.274518 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52356.2us
I20260709 13:00:11.284147 21284 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:11.284459 21216 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.240998 21424 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:11.248606 21429 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:11.273887 21259 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:11.284195 21326 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:11.274644 21295 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.249004 21408 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:11.291095 21216 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:11.288476 20855 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:11.228228 21177 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:11.303104 21216 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.274128 21447 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:11.301257 21326 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:11.305246 21259 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.306963 21259 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:11.302824 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 75511.7us
I20260709 13:00:11.265810 21064 consensus_queue.cc:237] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:11.310335 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.314977 21327 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.316425 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:11.317696 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:11.319128 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.320179 21336 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:11.286906 21159 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.320811 21159 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.321337 21159 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.323135 21141 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:11.323757 21141 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:11.324119 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:11.324896 21141 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.326256 21072 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.326658 21072 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:11.327005 21072 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.327144 20852 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:11.328816 20859 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:11.320813 21336 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:11.331386 21336 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.332639 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.333488 20161 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:11.333570 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.332818 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:11.334697 20163 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:11.336650 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 94904.7us
I20260709 13:00:11.338426 21284 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.339090 21284 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.341578 21467 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:11.342270 21284 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:11.342752 21123 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:11.343130 21123 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:11.343272 20130 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:11.344278 20163 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:11.344127 21123 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.345610 21123 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.345837 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:11.346849 21117 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:11.351214 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:11.364642 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 45371us
W20260709 13:00:11.397951 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 88226.3us
W20260709 13:00:11.401026 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.410663 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:11.416468 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 65780.7us
I20260709 13:00:11.368191 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 573) took 1847 ms. Trace:
I20260709 13:00:11.422823 20859 rpcz_store.cc:276] 0709 13:00:09.520605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.028499 (+1507894us) service_pool.cc:224] Handling call
0709 13:00:11.328567 (+300068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.329310 (+   743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.367963 (+ 38653us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540000aad0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:11.368107 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609787264}
I20260709 13:00:11.374212 20853 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:11.361933 21378 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:11.434105 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 50136.9us
I20260709 13:00:11.434988 21378 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:11.437525 21375 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:11.437922 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:11.438721 20859 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.440014 21412 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.440553 21412 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.353514 20852 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:11.368089 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:11.447439 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 79458.4us
I20260709 13:00:11.410394 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.374317 21177 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:11.451952 21412 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.418076 21467 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.484133 20850 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:11.484638 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.485491 20087 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:11.456161 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:11.459847 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.488837 21424 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.489225 21424 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.489734 21424 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.492239 21412 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.493402 21412 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.494302 21412 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.497066 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.497530 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.498632 20159 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
W20260709 13:00:11.501580 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 54517.4us
I20260709 13:00:11.366998 21072 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:11.461076 21117 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.461453 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.504252 20854 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:11.504703 20156 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:11.464226 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:11.464089 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.505750 20160 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.466125 20161 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:11.466900 20157 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:11.467335 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 124192us
I20260709 13:00:11.468622 20743 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:11.468201 21338 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:11.472560 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.473034 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:11.474529 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:11.475517 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:11.463263 21337 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.478062 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:11.479009 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:11.479254 20853 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:11.545892 20011 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:11.480126 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:11.505036 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.476398 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:11.524931 20856 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:11.546414 20011 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:11.549746 20744 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.551466 20011 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:11.553038 20011 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:11.556233 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.558478 21373 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:11.559402 21373 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:11.561707 21328 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.562347 21328 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:11.562721 21373 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.564344 21326 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:00:11.566627 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 93677.1us
I20260709 13:00:11.569202 21310 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:11.572211 21311 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:11.573187 21311 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:11.574216 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.575990 21429 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:11.576602 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:11.579887 21311 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.573421 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.580457 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:11.585763 20085 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.586115 21326 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.586723 21326 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.546166 21345 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.589650 21345 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:11.590306 21345 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.590670 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:11.581638 21375 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.571259 21310 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.583004 20055 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.584218 21338 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:11.582123 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:11.593370 21257 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.593468 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:11.504318 21117 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:11.597980 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.567593 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.595893 21408 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:11.605123 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:11.605537 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:11.526592 21072 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.640182 20089 raft_consensus.cc:2368] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
W20260709 13:00:11.621937 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 72284us
W20260709 13:00:11.644150 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 61262.5us
I20260709 13:00:11.576725 21429 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:11.599216 21177 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.602763 21284 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.595599 21326 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:11.653180 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 209180us
I20260709 13:00:11.605666 21328 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:11.671633 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 104117us
I20260709 13:00:11.632617 21373 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:00:11.583937 21424 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.653894 21123 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.686919 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.606428 21462 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.655507 20085 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:11.698041 20016 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:11.689112 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.692976 20162 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:11.721081 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:11.725097 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:11.725780 20015 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:11.673017 21338 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:11.725625 21478 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
I20260709 13:00:11.761245 21069 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:11.763991 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.685277 21177 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:11.768170 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 162699us
I20260709 13:00:11.733023 21328 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.722359 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 601) took 1255 ms. Trace:
I20260709 13:00:11.741500 21072 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:11.719646 21334 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:00:11.788806 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:11.797649 20016 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:11.797672 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.799047 20853 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:11.763417 21467 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:11.792775 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 93696.9us
W20260709 13:00:11.775646 20129 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.65:38607 timed out after 2.158s (SENT) [suppressed 4 similar messages]
I20260709 13:00:11.792436 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.791568 20859 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:11.801734 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:11.773087 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:11.812964 21424 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:11.801517 21064 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:11.813658 20156 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:11.813931 21064 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:11.814429 21310 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.815448 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:11.815126 21064 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.815989 21270 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:11.778522 21123 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.819520 20011 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:11.826591 20015 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:11.825376 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:11.816730 21270 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:11.798053 21257 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.839380 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.836674 21259 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:11.841073 21259 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:11.842618 21216 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.817166 21480 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.844355 21480 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:11.844767 21480 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.845999 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:11.847419 20164 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.848560 21424 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:11.850466 21424 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.851272 21424 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:11.851694 21424 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:11.852916 21424 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.836403 21373 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:00:11.854404 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:11.854699 21373 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:11.855158 21373 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:11.855542 21259 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:11.854466 21424 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.857311 21338 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:11.827296 20160 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
W20260709 13:00:11.835070 21159 consensus_meta.cc:287] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.005s
I20260709 13:00:11.867095 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 604) took 1229 ms. Trace:
I20260709 13:00:11.828912 21072 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:11.835413 21123 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:11.859244 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:11.880124 21270 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.780252 21429 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:11.903183 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 260580us
I20260709 13:00:11.904584 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.867542 20858 rpcz_store.cc:276] 0709 13:00:10.643887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.090136 (+446249us) service_pool.cc:224] Handling call
0709 13:00:11.109156 (+ 19020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:11.146345 (+ 37189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:11.146379 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:11.147886 (+  1507us) log.cc:844] Serialized 26 byte log entry
0709 13:00:11.235788 (+ 87902us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:11.235841 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.235943 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:11.866891 (+630948us) raft_consensus.cc:1685] finished
0709 13:00:11.866912 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.867038 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33742,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":493,"thread_start_us":86712,"threads_started":1,"wal-append.queue_time_us":407988}
I20260709 13:00:11.914772 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.915864 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:11.916934 21408 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:11.929085 21408 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:11.929886 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 69377.3us
W20260709 13:00:11.936954 21481 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.161s	user 0.000s	sys 0.003s
W20260709 13:00:11.938196 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 433767us
I20260709 13:00:11.940511 20162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:11.870563 21159 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.941402 20162 raft_consensus.cc:606] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:11.941779 20162 raft_consensus.cc:614] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:11.785347 20857 rpcz_store.cc:276] 0709 13:00:10.466596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.106935 (+640339us) service_pool.cc:224] Handling call
0709 13:00:11.122487 (+ 15552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.123373 (+   886us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:11.217766 (+ 94393us) log.cc:844] Serialized 14 byte log entry
0709 13:00:11.645928 (+428162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:11.649918 (+  3990us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:11.668735 (+ 18817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.685582 (+ 16847us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.717690 (+ 32108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3478,"prepare.queue_time_us":194932,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":93120,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":15811,"thread_start_us":401511,"threads_started":2,"wal-append.queue_time_us":622911}
I20260709 13:00:11.917840 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:11.865334 21412 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:11.967063 21338 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:11.967550 21462 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:11.968020 21117 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:11.866688 21257 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:11.973981 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.976995 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:11.919739 21216 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.979588 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:12.001560 20161 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:00:11.969655 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51862us
I20260709 13:00:12.004205 20016 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:12.004506 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:12.005983 20082 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.006878 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:12.009567 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:12.009835 21072 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:12.010409 20856 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.011370 21123 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:12.012554 21345 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.013288 21412 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:11.908311 21373 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.016369 21373 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.016798 21177 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:12.021078 21482 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:00:12.021816 21423 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:12.024140 20010 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:12.026723 21484 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.054s	user 0.001s	sys 0.001s
W20260709 13:00:12.040097 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.029007 21069 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:12.042088 21069 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:11.917819 21337 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:11.972680 20009 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:11.923467 20854 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
W20260709 13:00:11.970073 21447 consensus_meta.cc:287] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.666s	user 0.015s	sys 0.000s
I20260709 13:00:12.066967 20856 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:12.007640 20087 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
W20260709 13:00:11.990185 21295 consensus_meta.cc:287] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.675s	user 0.013s	sys 0.002s
I20260709 13:00:12.085592 21295 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:11.979230 21328 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:00:12.075450 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 125999us
W20260709 13:00:12.076074 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.033502 21429 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:12.094990 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 118205us
W20260709 13:00:12.093643 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 119737us
I20260709 13:00:12.066668 21345 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:12.102612 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.104249 21203 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.104893 21203 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.107128 21203 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.114714 21257 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:12.088809 21327 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:12.115679 21327 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.101060 21216 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:12.116776 21216 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.117210 21216 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:12.117920 21327 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.081913 21412 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:12.118747 21412 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:12.119674 21412 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.088626 21447 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:12.125555 21447 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:12.077579 21326 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.127054 21326 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:12.129642 21326 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:12.098294 21423 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.130674 21423 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.132831 21423 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:12.134088 21488 log.cc:927] Time spent T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:00:12.129629 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:12.138164 20082 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.141392 21295 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.102921 21328 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:12.145377 20157 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.146631 21328 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.153090 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:12.153949 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51748.3us
I20260709 13:00:12.159392 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:12.160339 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:12.166230 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 64472us
I20260709 13:00:12.155382 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:12.169487 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.170706 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:12.173738 20158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:12.175069 20158 raft_consensus.cc:606] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:12.066288 21338 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.176975 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:12.178313 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.183897 20016 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 2: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:12.156148 21467 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.201081 21069 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:12.164350 21462 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.159721 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.160851 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:12.209596 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:12.207472 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:12.179905 21378 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:12.213075 20161 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:12.217078 20855 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.218502 20856 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:12.217041 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 441380us
I20260709 13:00:12.189203 21327 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.233855 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:12.235459 20163 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.199388 21345 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.236948 20008 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:12.210997 21203 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.204919 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:12.255419 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:12.263298 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.216724 21159 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:12.269062 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:12.238541 21327 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:12.273507 20089 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:12.288623 20855 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:12.288502 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:12.294727 20856 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:12.214128 21467 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.294729 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:12.297047 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:12.298206 20855 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:12.300307 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.301388 20016 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:12.273327 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.303160 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:12.256734 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:12.315550 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:12.271661 21123 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:12.289637 21327 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.271579 21203 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:12.334403 20859 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:12.238407 21480 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.287591 21284 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:12.339001 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:12.258970 21257 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:00:12.290508 20157 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:12.361545 20163 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
W20260709 13:00:12.366092 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 124612us
I20260709 13:00:12.332357 21123 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.371110 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:12.331060 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:12.392558 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:12.334223 20012 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:12.395524 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:12.314616 21072 consensus_queue.cc:1059] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:00:12.362671 21424 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.359503 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:12.398429 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:12.410311 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.387053 21123 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.414650 21069 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:12.417181 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:12.418128 21495 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.427038 21334 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:12.428599 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:12.432180 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:12.412815 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:12.363163 21284 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.433763 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.462637 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:12.478781 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.479472 20016 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:12.469841 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:12.460134 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 164390us
I20260709 13:00:12.486573 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:12.487406 20086 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.440850 21378 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:12.459864 20857 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:12.499229 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.497906 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:12.500197 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.437506 21424 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:12.495901 20851 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:12.486810 21462 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:12.508453 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.487629 21493 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:12.512166 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 141539us
I20260709 13:00:12.491391 21311 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.447661 21257 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:00:12.509342 20852 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:12.459625 21334 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.499092 21375 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.539958 20859 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:12.495157 20010 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:12.544802 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.552011 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:12.517817 21437 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:12.550465 20017 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:12.532192 21203 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:12.536127 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:12.561982 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:12.562677 20017 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:12.557674 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52132.8us
I20260709 13:00:12.563195 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:12.560034 21338 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.506045 21072 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:00:12.570811 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:12.572860 20743 leader_election.cc:400] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:12.578918 20743 leader_election.cc:403] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:12.580106 20088 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 2: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:12.553004 21375 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.554677 21213 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:00:12.582526 21508 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.582871 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.583125 21508 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:12.584069 21508 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.589306 21462 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:12.614977 20086 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:12.601514 21514 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:00:12.631304 20012 raft_consensus.cc:2368] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:12.544701 21493 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:12.579632 21447 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 13:00:12.618620 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55559.5us
I20260709 13:00:12.599406 21378 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:12.542835 21311 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:12.713008 20014 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:12.714219 21311 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:12.638345 21334 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.582593 21375 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.618697 21509 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.638878 21524 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.715993 21524 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:12.637936 21513 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.674605 20010 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:12.594174 21480 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.641754 21522 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:12.641052 21424 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:12.626748 21499 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:00:12.623585 21213 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
W20260709 13:00:12.626328 21498 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.277s	user 0.000s	sys 0.003s
I20260709 13:00:12.585146 21467 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:12.639492 21496 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
W20260709 13:00:12.677284 21216 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.002s
I20260709 13:00:12.560603 21373 consensus_queue.cc:1059] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
W20260709 13:00:12.694460 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 196657us
W20260709 13:00:12.692808 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 157535us
I20260709 13:00:12.708494 21493 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:12.710357 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:12.761444 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 51173.2us
I20260709 13:00:12.763712 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:12.680998 21429 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.579s	user 0.013s	sys 0.000s
W20260709 13:00:12.725109 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 164440us
W20260709 13:00:12.746289 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 151245us
W20260709 13:00:12.756502 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 263373us
I20260709 13:00:12.764236 21493 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.728811 21522 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:12.748847 21524 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:12.724210 21513 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:12.832662 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 61019.9us
W20260709 13:00:12.846951 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 62558.8us
I20260709 13:00:12.849880 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.777223 21522 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.778680 21429 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.771759 21216 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:12.773353 21467 consensus_queue.cc:1059] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:00:12.781989 21493 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:12.862335 21337 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:12.862811 21123 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.865368 21127 raft_consensus.cc:3055] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.866953 21529 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.860406 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:12.870024 20012 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:12.871774 21408 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.902995 21513 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:12.876133 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:12.877867 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:12.856814 21534 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:12.908530 21534 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:12.909183 21534 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.908576 21424 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.911425 21259 raft_consensus.cc:3055] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:12.891839 21511 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:12.911988 21259 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:12.882011 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:12.861917 21345 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:12.884289 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.857666 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:12.890669 21327 consensus_meta.cc:287] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.002s
I20260709 13:00:12.895644 21127 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:12.896509 20082 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.898597 21493 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:12.892102 21337 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:12.899981 21123 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:12.899787 21529 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:12.901371 21524 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:12.901501 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:12.906731 20008 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.911440 21424 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:12.881685 21375 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:12.914357 21064 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.913699 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:12.921257 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:12.922289 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:12.903244 21408 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.914821 21334 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.972352 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.016440 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.017911 21127 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:12.912406 21511 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.023375 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.138087 20088 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:13.005396 21203 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:12.888535 21429 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.072197 21177 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:12.972754 21327 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:12.971840 21259 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:12.972245 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.104627 20013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:13.167201 20013 raft_consensus.cc:606] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:13.167744 20013 raft_consensus.cc:614] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:00:13.111869 20017 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 13:00:13.006320 21513 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.023114 21492 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.115502 21064 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.103345 21480 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:13.140854 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:13.071987 21467 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.153806 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:13.157431 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 256008us
I20260709 13:00:13.197397 19982 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:12.974500 21345 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.186339 21525 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.202401 20084 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:13.203589 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.203143 21525 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.204504 20084 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:13.204907 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:13.205134 21345 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.117264 21424 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.206373 21525 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.166932 21177 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.207463 21127 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.208691 21327 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.208907 21177 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.210515 21177 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:13.210906 21177 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.211392 21177 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:13.143033 21213 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.079959 21527 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.212177 21213 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.214946 21408 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.215322 21213 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.132735 21123 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.216041 21123 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.137676 21334 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.166183 20085 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:13.220000 21334 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:13.168717 20017 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:13.222100 21375 raft_consensus.cc:2764] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:13.223374 21500 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.224730 21500 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.224997 21467 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.228255 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88eccd78839140bba038b898679a00b0"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:13.229020 21511 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.203652 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.229624 21513 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.232115 21429 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.232443 21513 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.233515 21216 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.167388 21423 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.234491 21423 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.234989 21259 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.236687 21423 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.228271 21500 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:13.237416 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 457183us
I20260709 13:00:13.237964 21123 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.238512 21123 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:13.238654 20129 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:13.239428 21123 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:13.152849 21326 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.240486 21326 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.242976 21326 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.157320 21203 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:13.244849 21541 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:00:13.246928 21528 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.253584 21123 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:13.264048 21123 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.266279 21467 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.239554 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:13.295307 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 55821.7us
I20260709 13:00:13.268354 21480 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.229147 20017 raft_consensus.cc:606] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:13.300598 20017 raft_consensus.cc:614] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:13.304623 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:13.302153 21527 raft_consensus.cc:2764] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:13.310240 20089 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:13.312493 21326 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.286502 20086 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:13.323546 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:13.262238 21424 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.291862 21284 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:13.298084 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.293254 21529 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.326423 20088 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:13.313057 21326 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.333707 20851 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:13.282333 21328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.007s	sys 0.017s
I20260709 13:00:13.280769 21203 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.337738 21064 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:13.341926 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.342185 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.342728 20859 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:13.342563 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.343896 20165 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:13.344022 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.344612 20160 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:13.344369 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.345340 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:13.346258 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 464279us
I20260709 13:00:13.327701 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.325801 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:13.327473 21527 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:13.350126 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 436278us
I20260709 13:00:13.326539 21500 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:13.351881 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:13.264544 21522 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.186220 21337 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.360716 20160 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:13.357210 20744 leader_election.cc:400] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
W20260709 13:00:13.339064 21328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.008s	sys 0.017s
I20260709 13:00:13.361004 20089 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:13.363034 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:13.349205 21508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.006s	sys 0.012s
I20260709 13:00:13.374653 20086 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:13.366426 21534 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.345602 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.373242 20016 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:13.355791 21259 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:13.383312 21069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.008s	sys 0.017s
W20260709 13:00:13.384907 21069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.009s	sys 0.018s
I20260709 13:00:13.369851 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:13.375978 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50289.9us
I20260709 13:00:13.381382 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:13.385815 20011 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:13.380388 21064 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:13.359959 21528 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.358268 21527 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.389530 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.392828 20160 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:00:13.392217 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.388264 21064 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.360167 21203 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.398787 21203 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.393728 20857 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:13.392100 21259 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.385965 20008 raft_consensus.cc:2376] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3. Re-sending same reply.
I20260709 13:00:13.401345 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.401729 21527 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.402627 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.403591 20857 raft_consensus.cc:2410] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:13.404223 20083 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.394951 20160 pending_rounds.cc:85] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 1
I20260709 13:00:13.404852 20160 pending_rounds.cc:107] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:00:13.405158 20160 raft_consensus.cc:2889] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:13.405550 21424 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.406611 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.408061 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:13.408947 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.409159 21064 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:13.407444 21203 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:13.381072 21508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.006s	sys 0.015s
W20260709 13:00:13.397514 21530 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.288s	user 0.000s	sys 0.003s
I20260709 13:00:13.395342 20859 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:13.415477 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:13.416160 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:13.381318 21127 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.375499 21524 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:13.416632 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.372625 21284 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:00:13.422570 20160 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.382905 20157 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:13.426016 20163 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:13.425431 20016 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.422825 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.419483 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:13.425627 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 228314us
I20260709 13:00:13.431519 20016 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:13.422941 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.432572 20130 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:13.427477 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.412225 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.424023 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:13.364697 21500 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.441636 21500 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.444130 20016 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.446118 20016 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.446398 21525 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.446753 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.447321 21525 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.448776 20156 raft_consensus.cc:2410] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:13.450409 21522 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.450847 21525 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.402704 21534 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.452452 21480 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.453778 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.456321 21524 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.457141 21524 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.458153 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 583) took 1138 ms. Trace:
I20260709 13:00:13.458547 20855 rpcz_store.cc:276] 0709 13:00:12.319490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.421557 (+102067us) service_pool.cc:224] Handling call
0709 13:00:12.421994 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.290766 (+868772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:13.290817 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:13.290882 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:13.290929 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.457914 (+166985us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b54001900d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:13.457963 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:13.458099 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1997343488}
I20260709 13:00:13.460146 21524 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.460445 21177 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.461975 20858 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:13.462379 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:13.424393 20165 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.423225 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:13.463760 21527 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.465528 21524 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.438009 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.436158 21069 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:13.465747 21512 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.466356 21527 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.466630 20161 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:13.445137 21259 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:13.468851 20160 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:13.465576 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:13.433507 20086 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:13.473734 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.474699 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:13.477699 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:13.484315 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47420.7us
W20260709 13:00:13.484623 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 60718.9us
I20260709 13:00:13.485878 20743 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:13.475246 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:13.489058 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.489739 20161 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:13.492570 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.493410 20008 raft_consensus.cc:2376] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6. Re-sending same reply.
I20260709 13:00:13.494882 20016 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:13.491510 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:13.474776 21424 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:13.504786 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72317.2us
I20260709 13:00:13.506379 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.562422 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:13.570794 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.422562 21467 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:00:13.467733 21524 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.446409 21326 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.577577 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.496727 21534 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.612349 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:13.646014 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 171356us
I20260709 13:00:13.620813 20008 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:13.648022 20161 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
W20260709 13:00:13.611640 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 97661.3us
I20260709 13:00:13.647853 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:13.649013 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:13.505180 21522 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.618819 20160 raft_consensus.cc:2410] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:13.655634 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:13.571529 21492 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.671219 20162 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:13.672046 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.673041 20160 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:13.673092 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.673952 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:13.674510 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 186666us
I20260709 13:00:13.672612 20162 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:13.676100 20743 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:13.677496 21069 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:13.678056 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.674823 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 888) took 1422 ms. Trace:
I20260709 13:00:13.505614 21513 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:13.507803 21424 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:13.687435 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:13.687840 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.689294 20086 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:13.690619 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:13.692059 20086 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.612361 21527 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.507498 21525 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.698015 20089 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.652714 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.647753 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:13.587054 21429 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.568825 21480 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.681188 20090 rpcz_store.cc:276] 0709 13:00:12.246289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.246395 (+   106us) service_pool.cc:224] Handling call
0709 13:00:12.285738 (+ 39343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:12.388384 (+102646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:12.388464 (+    80us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:12.388524 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:12.388570 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.668469 (+1279899us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54009b8554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:13.668518 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:13.668635 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232709376}
I20260709 13:00:13.703156 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:13.703415 20165 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:13.586421 21529 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:13.612427 21117 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:00:13.665802 21326 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:13.707063 21522 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:00:13.715231 21259 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.717145 21177 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.672580 21492 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:13.718835 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:13.765774 20008 raft_consensus.cc:2376] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6. Re-sending same reply.
I20260709 13:00:13.720263 20086 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
W20260709 13:00:13.720669 21548 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 13:00:13.720830 21424 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.617936 21295 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:00:13.674687 21462 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.728036 21525 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.729518 20162 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.779479 20164 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:13.732471 21524 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:13.734251 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47679.2us
I20260709 13:00:13.735041 21069 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:13.741549 20159 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:13.761031 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:13.768349 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 206046us
I20260709 13:00:13.774003 20160 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:13.701434 21500 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:00:13.774456 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:13.807947 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.808822 20008 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:13.808887 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.809724 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.810523 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.692792 21513 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:13.761655 21326 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.729952 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:13.813987 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 84155.9us
I20260709 13:00:13.814445 20011 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:13.814941 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:13.781718 21069 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.705492 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.770807 21492 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.771142 21123 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:13.792374 21509 raft_consensus.cc:3055] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.794719 21424 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.769569 21259 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.798177 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:13.799162 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.799556 21447 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:13.741057 21429 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.783385 21529 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:13.804296 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 885) took 1600 ms. Trace:
I20260709 13:00:13.862478 20082 rpcz_store.cc:276] 0709 13:00:12.203808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.206143 (+  2335us) service_pool.cc:224] Handling call
0709 13:00:12.273203 (+ 67060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.075136 (+801933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.803988 (+728852us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b5400a5e954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:13.804225 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412677376}
W20260709 13:00:13.805066 21540 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.700s	user 0.006s	sys 0.008s
I20260709 13:00:13.806293 21525 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.814034 20162 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:13.865583 21327 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.866386 20854 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.866447 21327 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.758968 20857 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:13.867837 20854 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:13.816061 21069 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:13.818914 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:13.782976 21203 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.819525 21537 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:13.787087 21216 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:13.883556 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.884377 20089 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:13.884531 21537 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:13.827786 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.816087 21326 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.820087 21524 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:13.851251 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:13.815125 21513 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.852612 21259 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.887264 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.860047 21480 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:13.869725 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50876.1us
I20260709 13:00:13.861351 21429 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.869181 21508 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:13.872587 21338 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:00:13.875824 21177 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:13.860812 21447 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.876230 21525 consensus_queue.cc:1059] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:13.880566 21069 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.820262 21509 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:13.819875 21123 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:13.877558 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:13.824920 21500 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:13.883397 21216 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:13.848722 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 200948us
W20260709 13:00:13.925292 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47827.2us
I20260709 13:00:13.926537 20744 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:13.885001 21537 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:13.826819 20013 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:13.811417 21462 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:13.888365 20089 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:13.881222 20161 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:13.891031 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.847018 21117 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:13.849808 21549 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:13.861634 21529 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.941429 21529 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:13.891080 21327 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:13.901381 21508 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.945860 21508 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:13.899775 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73780.7us
I20260709 13:00:13.899745 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:13.888690 21524 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.947017 21524 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:13.951004 21447 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:13.952179 21462 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.953487 21549 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.938037 21500 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:13.927237 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:13.921036 21328 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.957542 21328 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.920496 21177 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:13.959830 21328 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.960847 20015 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.962584 21177 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.906539 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:13.933168 20162 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:13.963666 20162 pending_rounds.cc:85] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 3
I20260709 13:00:13.963959 20162 pending_rounds.cc:107] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:13.964207 20162 raft_consensus.cc:2889] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:13.965224 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.965515 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.966325 20162 raft_consensus.cc:2368] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:13.967180 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.936249 21069 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:13.970474 20744 leader_election.cc:400] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:13.971108 20744 leader_election.cc:403] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:13.970889 21524 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.959594 21500 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.972148 20008 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:13.939906 21555 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:00:13.934299 20129 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:13.973762 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:13.974676 21447 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:13.975000 21259 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:13.975435 21259 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.939640 21562 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:13.940568 21213 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.972075 21069 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.971654 21177 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:13.980204 20851 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.980096 21177 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:13.987057 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:13.987772 20009 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 1.
I20260709 13:00:13.989079 20009 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:13.987789 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.933772 21545 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:13.990615 21545 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.990970 21545 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.975845 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:13.994472 21127 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.994541 21159 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:13.996121 21159 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:13.990876 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.999790 20165 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:14.001538 21561 raft_consensus.cc:3055] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:14.002027 21561 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:14.002154 20162 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:14.002965 21561 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.003485 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.004434 20162 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:14.004971 20013 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:14.005594 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.006356 20162 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:14.006546 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:14.007117 21512 consensus_meta.cc:287] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.000s
I20260709 13:00:14.001657 20744 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:14.010820 21177 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.011102 21072 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:14.011497 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:14.011395 21177 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.011960 21528 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:14.013207 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.013371 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:13.940172 21216 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:13.961669 21326 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:13.993935 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:14.014151 21509 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.014914 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.015419 21509 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.886178 21203 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.016013 21326 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:13.921221 21123 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.921376 21480 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.016917 21493 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.013466 21500 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.017649 21493 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.017284 21480 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.018043 21525 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.018370 21493 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.011615 21072 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:14.021343 20159 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:14.020661 21072 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:14.001849 21257 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:00:13.975757 21259 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.940517 21117 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.018304 21500 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:14.011713 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:14.025110 19982 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:14.026057 21338 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.026563 21338 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:14.027796 21544 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:14.027518 21338 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.028600 21544 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.031765 20163 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 7 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:14.065583 21069 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3.
I20260709 13:00:14.066082 20015 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
W20260709 13:00:14.048671 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54837.2us
I20260709 13:00:13.982867 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.083539 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.079522 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 806) took 1983 ms. Trace:
I20260709 13:00:14.086635 21326 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:14.087313 21326 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.087707 21326 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:14.089143 21511 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:14.089722 21511 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.095669 19982 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:14.117264 20162 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:14.084630 21529 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:14.078734 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 172259us
W20260709 13:00:14.080200 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 133514us
I20260709 13:00:14.084991 20157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:13.900805 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:14.072674 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:14.138440 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 65874.4us
I20260709 13:00:14.007753 21512 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.119007 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:14.143604 21069 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:14.143299 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.154198 21069 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:13.967231 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.064437 21072 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.162604 20160 raft_consensus.cc:2368] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6.
I20260709 13:00:14.173038 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:14.173754 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:14.173972 20085 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:14.178886 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.144522 21528 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:14.152858 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.158653 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.151995 21561 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.975041 21447 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.158803 20157 raft_consensus.cc:606] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.183090 20091 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:14.148600 21257 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.144035 21550 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:13.984524 21213 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.133446 21480 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:14.198879 21550 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.190141 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.199638 21550 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.154739 21127 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.177862 21069 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:14.156225 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.199811 20158 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.204741 21447 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.162691 21560 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:14.233819 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.233937 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.244071 20017 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:14.187323 21528 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:14.261926 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.192353 21549 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.229019 21508 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:14.247681 21467 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.015s	sys 0.015s
I20260709 13:00:14.269271 20017 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:14.273900 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 228029us
I20260709 13:00:14.235333 21529 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.252628 20850 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.291991 21544 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:14.256542 21560 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.293010 21500 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:14.295589 21500 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:14.297173 21069 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:14.297561 21069 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:14.298357 21069 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.298799 21513 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.264631 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.300649 20084 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:14.301558 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.301930 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:14.302773 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:14.305500 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:14.306176 21203 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.257682 21525 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:14.307060 21525 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:14.265120 21528 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:14.307672 21525 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.281396 21574 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:14.282335 21577 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.284090 21177 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.284562 20859 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:00:14.284775 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.284911 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.275638 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.285295 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:14.286094 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.326459 20015 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:14.287221 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.323889 21127 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.307868 21123 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.333804 21326 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.329556 21574 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.334614 21524 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:14.336524 20084 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:14.324061 21577 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.337749 21123 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:14.338809 21577 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.340782 20156 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:14.288626 21117 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:14.290148 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:14.271467 21467 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.016s	sys 0.016s
I20260709 13:00:14.321599 20085 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:14.319391 20158 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:14.261890 21257 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:14.348155 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.348307 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.269023 21423 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:14.286880 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.328814 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.340996 21512 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.295652 21560 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:14.331843 21177 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.313753 21159 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.309898 21525 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:14.344123 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.303607 21429 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.349912 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.404918 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:14.405156 20158 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:14.272957 21508 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.355217 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:14.407258 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.360061 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:14.408352 20859 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:14.331874 21580 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:14.362375 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.368530 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:14.343451 20082 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:14.343287 20852 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:14.373585 21257 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.374886 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.379909 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 587) took 2056 ms. Trace:
I20260709 13:00:14.370505 21177 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.382576 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.356798 21423 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.295723 21537 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.385208 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.385967 21574 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:14.387163 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.392818 20157 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:00:14.393355 21424 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.002s
I20260709 13:00:14.398823 21545 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.314749 21492 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.411000 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:14.403780 20854 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:14.468477 21177 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:14.413146 21508 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.469271 21528 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.471149 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 623) took 1092 ms. Trace:
I20260709 13:00:14.417579 20013 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:14.421404 20086 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 2: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:14.471556 20851 rpcz_store.cc:276] 0709 13:00:13.378505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.815496 (+436991us) service_pool.cc:224] Handling call
0709 13:00:13.889334 (+ 73838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.986744 (+ 97410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.470918 (+484174us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400380254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:14.471091 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208500608}
I20260709 13:00:14.472854 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.473160 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.474077 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.474466 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.475119 20851 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:14.432068 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.475950 20085 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:14.475965 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.477156 20854 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:14.365615 21512 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.475144 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.445499 21338 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:14.482489 21513 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:14.482342 21338 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.483546 21429 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.483752 21512 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.446270 21375 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:14.477636 21326 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.484512 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 625) took 1100 ms. Trace:
I20260709 13:00:14.451082 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.453727 20162 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:14.444444 21544 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.456929 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:14.458539 21492 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.459033 20016 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:14.459645 21537 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.461562 21423 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.499258 20856 raft_consensus.cc:2368] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:14.467300 21447 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.463724 20852 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:14.085556 20087 rpcz_store.cc:276] 0709 13:00:12.096278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.096421 (+   143us) service_pool.cc:224] Handling call
0709 13:00:12.101565 (+  5144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:12.101847 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:12.361252 (+259405us) log.cc:844] Serialized 14 byte log entry
0709 13:00:14.075485 (+1714233us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b18001229e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a
0709 13:00:14.078933 (+  3448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:14.078993 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:14.079033 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.079108 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:14.079460 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3198,"prepare.queue_time_us":311631,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":3546,"spinlock_wait_cycles":5621632,"thread_start_us":369532,"threads_started":2,"wal-append.queue_time_us":275325,"wal-append.run_cpu_time_us":1345,"wal-append.run_wall_time_us":1010639}
I20260709 13:00:14.506002 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:14.506876 21447 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.476869 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.441682 20089 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:14.422058 21580 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.478320 20086 raft_consensus.cc:2393] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:14.478448 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.481030 20161 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:14.487084 20859 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
W20260709 13:00:14.487104 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 76176.8us
I20260709 13:00:14.484663 21338 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.485323 20858 rpcz_store.cc:276] 0709 13:00:13.384324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.422648 (+ 38324us) service_pool.cc:224] Handling call
0709 13:00:14.484335 (+1061687us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b5400005ad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:14.484448 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66923776}
I20260709 13:00:14.426689 20850 rpcz_store.cc:276] 0709 13:00:12.323391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.449741 (+126350us) service_pool.cc:224] Handling call
0709 13:00:13.963147 (+1513406us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:13.963396 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.292083 (+328687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.379675 (+ 87592us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000055d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:14.379849 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612843136}
I20260709 13:00:14.492147 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:14.484452 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.526414 20086 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:14.462152 21567 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:14.497057 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 154571us
I20260709 13:00:14.529824 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:14.501063 20015 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:14.503293 20744 leader_election.cc:400] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6.
I20260709 13:00:14.531076 20744 leader_election.cc:403] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:14.423420 21493 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.507066 20852 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:14.467211 21527 raft_consensus.cc:3055] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:14.509698 20129 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.509104 20090 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:14.529039 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.536337 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:14.532066 20086 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.492421 21375 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.542474 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.543643 20129 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.545878 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:14.533336 20859 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:14.526201 20851 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:14.552592 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:14.528898 21423 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.527801 21528 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:14.502177 21259 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.499020 21544 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.535897 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.538467 21527 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:14.554319 20089 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:14.556046 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.577692 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:14.579645 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:14.581251 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.533676 21117 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.590780 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:14.541651 21492 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.597923 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.599109 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.602782 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:14.604585 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:14.596290 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.542276 21480 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6.
I20260709 13:00:14.588430 20084 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:14.553520 21579 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.554716 21561 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.610949 20164 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:14.575063 21467 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:14.612977 20129 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:14.589918 21337 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.601446 20855 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:14.574757 21582 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:14.538661 21567 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.588891 21494 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:14.574738 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:14.611446 21569 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:14.620519 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.621142 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 455) took 1178 ms. Trace:
I20260709 13:00:14.622324 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:14.623222 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.632750 20855 rpcz_store.cc:276] 0709 13:00:13.443555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.545542 (+1101987us) service_pool.cc:224] Handling call
0709 13:00:14.621071 (+ 75529us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:14.597651 21577 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:14.553050 21447 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:00:14.615949 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:14.626183 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:14.588649 21524 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:00:14.646307 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:14.655130 20162 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:14.658373 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.658475 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:14.589214 21423 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.625298 21582 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.665031 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:14.624058 21576 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:14.656844 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:14.631520 21492 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.610321 21550 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.620805 21575 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:14.629242 21467 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:14.670858 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.611310 21512 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:00:14.623746 21123 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:00:14.651502 21072 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:14.686980 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.660281 21310 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.664876 21574 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:14.656064 20084 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:14.698519 20017 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:14.656224 21513 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.687814 20163 raft_consensus.cc:2368] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4.
I20260709 13:00:14.660343 21569 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:14.672808 21462 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.671713 21582 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.666109 21494 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:14.661341 21203 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:00:14.714622 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.685997 21311 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.674676 21576 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:14.716382 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 178337us
I20260709 13:00:14.684706 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.696514 21574 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.723083 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:14.724282 20855 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.728312 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:14.729156 20163 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.686852 21560 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:00:14.711213 21462 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.720407 21576 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:14.732594 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67644.9us
I20260709 13:00:14.684322 21565 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 13:00:14.743944 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 67189.3us
I20260709 13:00:14.754455 20163 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:14.736248 21545 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.729036 21574 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:14.728240 21550 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:14.741875 21462 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.754078 21493 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.793306 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.761559 21580 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.798666 21259 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:14.760911 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:14.745448 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:14.751691 21467 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:14.858059 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:14.858752 21509 consensus_meta.cc:287] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.843s	user 0.012s	sys 0.009s
I20260709 13:00:14.894930 20854 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:14.895992 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 150457us
I20260709 13:00:14.897522 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:14.874275 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.687366 21527 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.786402 21494 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.737346 21513 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.777180 21569 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:14.860733 20855 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
W20260709 13:00:14.862335 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.893172 20852 raft_consensus.cc:2410] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:14.911676 20129 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:14.894279 21550 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.894064 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.894634 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:14.932899 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:14.884465 21589 log.cc:927] Time spent T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.155s	user 0.003s	sys 0.002s
W20260709 13:00:14.892086 21592 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.138s	user 0.001s	sys 0.005s
I20260709 13:00:14.882922 21514 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:14.899592 21203 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.901167 21596 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:14.871966 21295 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:00:14.948056 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.953177 20855 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:14.920271 21544 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.922396 21259 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.899940 21509 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:00:14.939218 21338 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:14.930343 21513 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:14.945065 21375 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:14.943701 21514 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:14.981750 21514 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:14.983047 21514 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:14.978266 21338 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:14.984642 21338 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.985925 21423 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:14.923329 21525 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:00:14.971031 20851 raft_consensus.cc:2410] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:14.987411 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:14.988068 20851 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:14.989607 21513 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:14.977727 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:14.994270 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 96827.3us
I20260709 13:00:14.946795 21203 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.950896 21596 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.967182 21310 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:15.001345 20855 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:15.002781 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:15.015116 20859 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:15.015422 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.021435 21423 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:15.021755 20852 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:15.022743 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.027714 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.002177 21203 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.998020 21259 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:14.967098 21528 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:00:14.998337 21567 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:15.001909 21596 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.993808 21544 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 13:00:15.033974 21508 consensus_meta.cc:287] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.565s	user 0.015s	sys 0.003s
I20260709 13:00:15.034453 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.045614 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.104089 20855 raft_consensus.cc:2410] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:00:15.048659 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.108623 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.110140 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.111383 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.054044 21582 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.114944 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:14.989672 21492 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:15.022769 20851 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:15.059126 20854 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:15.117771 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.118499 20854 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:15.118697 21462 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.064941 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.067034 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 813) took 1379 ms. Trace:
I20260709 13:00:15.139168 20091 rpcz_store.cc:276] 0709 13:00:13.687587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.687875 (+   288us) service_pool.cc:224] Handling call
0709 13:00:13.716444 (+ 28569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.229431 (+512987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.066557 (+837126us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54006649d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:15.066939 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072680192}
I20260709 13:00:15.069892 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.140848 21213 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.141412 21213 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.132223 21582 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:15.142829 21574 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:15.072530 20016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 812) took 1602 ms (client timeout 1670 ms). Trace:
W20260709 13:00:15.144083 20016 rpcz_store.cc:269] 0709 13:00:13.470089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.506030 (+ 35941us) service_pool.cc:224] Handling call
0709 13:00:15.072360 (+1566330us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54005cec54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:15.072474 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2110453120}
I20260709 13:00:15.074339 20129 leader_election.cc:400] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:15.144963 21574 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:15.145203 20129 leader_election.cc:403] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:15.145361 21203 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:15.146044 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 72223.2us
I20260709 13:00:15.146564 21576 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.147614 20163 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.071621 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 534) took 1282 ms. Trace:
I20260709 13:00:15.148175 20008 rpcz_store.cc:276] 0709 13:00:13.788696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.890783 (+102087us) service_pool.cc:224] Handling call
0709 13:00:15.071007 (+1180224us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b54002ebb54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:15.071164 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97295104}
I20260709 13:00:15.072885 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.149153 21576 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:15.079450 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 151012us
I20260709 13:00:15.149746 21576 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:15.083415 20744 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.007s
W20260709 13:00:15.150914 20744 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.008s	sys 0.007s
W20260709 13:00:15.151216 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 593884us
I20260709 13:00:15.151571 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 225) took 1467 ms. Trace:
I20260709 13:00:15.152359 20744 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:15.152959 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.153087 21424 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:15.153561 20859 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.119562 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.154934 21575 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:15.155431 21575 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.143728 21213 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:15.156113 20854 raft_consensus.cc:2410] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:15.156864 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:15.157060 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:15.086760 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.162714 21525 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:15.163395 21525 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:15.095022 20011 raft_consensus.cc:2410] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:15.165063 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.166321 20008 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
W20260709 13:00:15.097019 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 131213us
I20260709 13:00:15.087409 21511 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:15.111622 21338 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.168973 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.169821 20012 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:15.169915 21338 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:15.170845 21591 log.cc:927] Time spent T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.444s	user 0.001s	sys 0.001s
I20260709 13:00:15.171118 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.171316 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.091306 21493 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:15.174943 21493 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.167800 21492 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:15.054502 21579 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:15.177964 21492 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.178583 21579 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:15.179097 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.060765 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:15.189890 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 129224us
I20260709 13:00:15.190732 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.119946 20855 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:15.120175 21545 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.093897 21423 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:15.191768 21511 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:15.191337 21338 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:15.191052 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:15.192777 21338 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:15.118793 21588 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.390s	user 0.000s	sys 0.000s
I20260709 13:00:15.197052 21338 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:15.091673 21602 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.213980 20854 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:15.134142 21597 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:15.215191 21584 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.663s	user 0.004s	sys 0.000s
I20260709 13:00:15.215133 21597 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.215778 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.216529 21597 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.217583 21597 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:15.193209 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.218806 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.136289 20852 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:15.151976 20165 rpcz_store.cc:276] 0709 13:00:13.684743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.390603 (+705860us) service_pool.cc:224] Handling call
0709 13:00:14.391133 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.535231 (+144098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.535275 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:14.535322 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:14.535349 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.151354 (+616005us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54006e4e54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:15.151397 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:15.151517 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330839424}
I20260709 13:00:15.147562 21203 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:15.221503 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:15.222316 20165 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:15.166755 21544 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.172290 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.172474 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.172590 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:15.234505 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.235013 20015 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.173264 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.173619 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 234) took 1093 ms. Trace:
I20260709 13:00:15.173928 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.180776 20008 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:15.180686 21524 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:15.183094 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.184055 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.239408 20165 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:15.240559 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.184863 20859 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:15.184957 21579 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.181077 20856 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:15.243880 20858 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:15.242599 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:15.245342 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.246114 20014 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:15.186924 21412 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:15.188330 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.164341 21525 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.192556 21423 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.124763 21259 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.192797 21511 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:15.193476 21545 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.197661 21509 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.198143 21567 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:15.198367 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:15.103835 21508 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.268821 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 638) took 1363 ms. Trace:
I20260709 13:00:15.271041 21508 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:15.269778 20156 rpcz_store.cc:276] 0709 13:00:13.905157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.400869 (+495712us) service_pool.cc:224] Handling call
0709 13:00:14.401270 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.401520 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.401548 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:14.402339 (+   791us) log.cc:844] Serialized 26 byte log entry
0709 13:00:14.442215 (+ 39876us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:14.481553 (+ 39338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.489630 (+  8077us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:15.214424 (+724794us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:15.263913 (+ 49489us) raft_consensus.cc:1685] finished
0709 13:00:15.263979 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:15.268728 (+  4749us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47327616,"thread_start_us":24633,"threads_started":1,"wal-append.queue_time_us":99764}
I20260709 13:00:15.261577 21524 consensus_queue.cc:1243] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:15.250409 21412 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.237133 20157 rpcz_store.cc:276] 0709 13:00:14.080083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.454271 (+374188us) service_pool.cc:224] Handling call
0709 13:00:14.454946 (+   675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:14.455457 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:14.455497 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:14.531960 (+ 76463us) log.cc:844] Serialized 26 byte log entry
0709 13:00:14.671184 (+139224us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:14.675434 (+  4250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.687555 (+ 12121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:15.170061 (+482506us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:15.173261 (+  3200us) raft_consensus.cc:1685] finished
0709 13:00:15.173292 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:15.173550 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":972,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":68,"spinlock_wait_cycles":7373056,"thread_start_us":115661,"threads_started":1,"wal-append.queue_time_us":177308}
I20260709 13:00:15.195497 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.201116 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:15.274791 20160 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:15.275207 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.203187 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.202610 21337 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.276378 20009 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:15.276461 21337 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.205226 21069 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.276996 21567 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:15.277630 21567 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:15.278685 21567 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.189085 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33788 (request call id 226) took 1453 ms. Trace:
I20260709 13:00:15.178061 21493 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:15.266289 21509 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:15.279192 21337 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:15.280085 20164 rpcz_store.cc:276] 0709 13:00:13.735419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.006329 (+270910us) service_pool.cc:224] Handling call
0709 13:00:15.188870 (+1182541us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b5400574254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:15.189029 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380589312}
I20260709 13:00:15.280437 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:15.285645 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 200) took 1112 ms. Trace:
I20260709 13:00:15.285977 20088 rpcz_store.cc:276] 0709 13:00:14.173332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.185579 (+ 12247us) service_pool.cc:224] Handling call
0709 13:00:15.280314 (+1094735us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:15.285581 (+  5267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29444864}
W20260709 13:00:15.207466 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55180.9us
I20260709 13:00:15.287436 21462 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:15.280874 21594 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:00:15.287945 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.290565 21500 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.290675 21064 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:15.165639 19982 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:15.292697 21527 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.294847 21563 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:15.295396 21563 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.246001 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.296277 21563 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.283505 21509 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.185459 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:15.211450 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.247270 20161 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.232525 21602 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:15.249938 20853 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:15.253021 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.254261 20014 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:15.257804 20858 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.171844 20012 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:15.302162 21567 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:15.316571 20090 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:15.260592 21310 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:15.245689 21480 raft_consensus.cc:2764] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260709 13:00:15.272540 21604 log.cc:927] Time spent T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
W20260709 13:00:15.293228 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 127675us
I20260709 13:00:15.246560 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.337574 20016 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:15.337615 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.300347 20015 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
W20260709 13:00:15.088577 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 73367.1us
I20260709 13:00:15.282857 20087 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:15.300595 20162 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:15.301726 20085 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:15.225132 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.256359 21574 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:15.310487 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:15.234953 21311 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:15.237468 21544 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.267753 21492 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:15.281920 21579 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.267400 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.312188 21513 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:00:15.358223 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:15.369138 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50040.5us
W20260709 13:00:15.409333 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 70255.8us
I20260709 13:00:15.411075 21524 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:15.432206 20857 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:15.416438 20850 raft_consensus.cc:2368] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4.
I20260709 13:00:15.421922 21311 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:15.433593 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.433693 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.434602 20853 raft_consensus.cc:2410] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:15.434921 21311 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:15.419309 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:15.441071 21607 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:00:15.441255 20084 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.441946 21500 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:15.447670 20854 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:15.448503 21509 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.449887 21338 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:15.449519 21513 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:15.451259 21513 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.451794 21509 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:15.452615 21513 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.453608 21544 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:15.454985 20086 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.403141 21117 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.411181 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:15.403375 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 65844us
W20260709 13:00:15.413771 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 188726us
I20260709 13:00:15.462424 20055 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.462122 19982 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.263640 21596 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.460661 21337 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:00:15.432607 20091 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:15.464124 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.464641 19982 leader_election.cc:400] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4.
I20260709 13:00:15.426942 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.282114 21605 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.157s
I20260709 13:00:15.410123 21582 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.432542 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:15.476250 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65158.2us
W20260709 13:00:15.476492 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 44037.7us
I20260709 13:00:15.450331 21338 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.421250 21574 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:15.483935 21462 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:15.474294 19982 leader_election.cc:403] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:15.474046 20055 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.420233 21508 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.477715 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.467808 21117 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.478844 20090 raft_consensus.cc:2410] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:15.493970 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.461867 21545 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:15.494582 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.473353 21423 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.495859 20016 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:15.475229 21537 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:15.497449 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:15.526235 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.527420 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:15.502389 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.527812 21310 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:15.528405 21310 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.528775 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.529865 21177 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.530532 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.530719 21213 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.532485 21595 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:15.533205 21595 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.533710 21203 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:15.534356 20055 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:15.535108 21203 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:15.536126 21509 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.536685 21509 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:15.537261 21509 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.540624 21601 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.530560 21177 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.541555 21177 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.505975 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:15.542879 20088 raft_consensus.cc:2368] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3.
I20260709 13:00:15.541420 19839 catalog_manager.cc:5697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.533756 21595 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:15.544247 21528 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:00:15.507483 21462 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:15.544106 19839 catalog_manager.cc:5697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 3 to 6, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 6 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.508512 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.509029 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.545768 21311 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:15.546238 20084 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:15.545719 19839 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 5, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.546243 21311 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.547266 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:15.511164 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:15.512962 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 914) took 1047 ms. Trace:
I20260709 13:00:15.514540 20159 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:15.547952 20087 rpcz_store.cc:276] 0709 13:00:14.465116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.505700 (+ 40584us) service_pool.cc:224] Handling call
0709 13:00:15.512730 (+1007030us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540078f854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:15.512888 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1759774592}
I20260709 13:00:15.482884 21610 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:15.549687 21610 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:15.550117 21610 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:15.549834 21574 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.483429 21582 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.552826 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 725) took 1041 ms. Trace:
I20260709 13:00:15.553169 21596 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:15.500209 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:15.521864 21338 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.579205 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:15.520151 21412 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.523192 21334 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:15.486636 21500 consensus_queue.cc:1059] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:00:15.535611 21203 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.553501 21582 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:15.597393 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 97832.9us
I20260709 13:00:15.486946 21512 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:15.598011 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.475419 21492 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:00:15.553303 20159 rpcz_store.cc:276] 0709 13:00:14.511344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.724953 (+213609us) service_pool.cc:224] Handling call
0709 13:00:15.189024 (+464071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.548128 (+359104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.548721 (+   593us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b5400574254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:15.552617 (+  3896us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b5400574250. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:15.552765 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813868160}
I20260709 13:00:15.554296 21545 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.547619 19839 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:15.555688 21424 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.556360 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.496901 21576 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:15.497314 21508 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.536266 20082 raft_consensus.cc:2368] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:15.578411 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:15.707041 20162 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:15.475108 21565 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:00:15.582772 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:15.583482 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:15.586115 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:15.472465 21602 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:00:15.600253 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.712499 20856 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.573204 21563 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.589069 21525 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.584692 21123 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:15.595253 21072 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.585449 21596 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.578824 21494 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:00:15.622455 21424 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:15.599191 21462 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.595520 21259 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.680850 19839 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:15.587371 21569 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.589084 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:15.690331 20013 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:15.682950 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:00:15.694849 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138555us
W20260709 13:00:15.695101 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 87289us
I20260709 13:00:15.643560 21509 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:15.700821 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.702486 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.703783 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:15.718461 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 135040us
I20260709 13:00:15.680531 21527 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.613381 21605 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:00:15.740473 19839 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 3 to 4, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 4 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:15.693261 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:15.697570 21338 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:15.705710 21508 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.708599 21480 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:15.698673 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 198533us
I20260709 13:00:15.743499 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:15.807088 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.752837 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.755446 21595 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:15.756510 21512 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:15.758150 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.760108 19839 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 3, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:15.769232 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.774893 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:15.830586 20087 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:15.777922 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.780889 20855 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:00:15.832913 20855 pending_rounds.cc:85] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 2
I20260709 13:00:15.833245 20855 pending_rounds.cc:107] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:00:15.833578 20855 raft_consensus.cc:2889] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:15.834584 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:15.733021 21424 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.726598 21127 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.709793 21524 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:15.814162 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 70770.6us
I20260709 13:00:15.842437 20856 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:15.839178 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.842528 19839 catalog_manager.cc:5697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 3, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:15.729009 21123 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.733295 21072 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.858397 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.742267 21596 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.730239 21525 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.854506 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.852880 19839 catalog_manager.cc:5697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
W20260709 13:00:15.850139 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 251236us
I20260709 13:00:15.860555 20087 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:15.776014 21544 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:00:15.738127 21462 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.747588 21569 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.892607 19981 leader_election.cc:400] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3.
I20260709 13:00:15.893735 19981 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:15.828475 21480 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:15.849308 20857 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:15.738036 21259 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:15.890272 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.842722 21590 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.854125 21605 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.877506 19839 catalog_manager.cc:5697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 3, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 3 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:00:15.848789 20083 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.774806 21527 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.897608 19982 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:15.858253 21257 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.841351 20854 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
W20260709 13:00:15.839079 21575 consensus_meta.cc:287] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.683s	user 0.013s	sys 0.000s
I20260709 13:00:15.905190 19981 leader_election.cc:400] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4.
I20260709 13:00:15.874641 21492 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:15.874898 21524 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.912102 19839 catalog_manager.cc:5697] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 4, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 4 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.843590 21512 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:15.902575 21500 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:15.919397 21597 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.701s	user 0.013s	sys 0.000s
I20260709 13:00:15.927112 19839 catalog_manager.cc:5697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:15.841706 21177 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:15.900018 21602 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.900243 21480 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:15.892089 21514 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:00:15.914806 20850 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:15.941902 19839 catalog_manager.cc:5697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:15.893914 21610 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.926035 20852 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:15.939060 21494 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.924042 21492 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:15.919129 21605 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:15.982568 21123 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:15.921411 21590 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:15.988534 21512 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:15.979255 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 62362us
I20260709 13:00:15.946503 21423 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:00:15.994599 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:15.969638 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 216864us
I20260709 13:00:16.000331 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:15.941622 21544 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.968792 19839 catalog_manager.cc:5697] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
W20260709 13:00:16.054061 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 53860.6us
I20260709 13:00:16.044541 19839 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 3, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 3 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.926844 21575 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:15.959308 21203 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:15.964607 21597 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.077224 19839 catalog_manager.cc:5697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 3, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 3 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
W20260709 13:00:16.073988 21117 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.000s
I20260709 13:00:16.054610 21544 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.053699 21123 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:16.100776 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 106260us
I20260709 13:00:16.069594 21576 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:16.108071 20017 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
W20260709 13:00:16.076069 21567 consensus_meta.cc:287] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.000s
I20260709 13:00:16.072310 21338 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.122560 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:16.080753 21597 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:16.137905 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 240350us
W20260709 13:00:16.100873 21601 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 13:00:16.055876 21494 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:16.100502 21629 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:16.124869 20850 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:16.120272 21544 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.112118 21632 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:16.129709 21576 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.125176 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:16.139204 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:15.982568 21492 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:16.101204 21636 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:16.129179 21509 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:16.170142 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:16.402477 21619 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.419423 20743 leader_election.cc:336] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.698s (SENT)
W20260709 13:00:16.593617 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 174243us
I20260709 13:00:16.230611 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:16.222236 20163 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:00:16.218075 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:16.125528 21524 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.225504 21257 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:16.285475 21624 log.cc:927] Time spent T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:00:16.161808 21601 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.219092 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:16.612262 20162 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.425001 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.174559 21576 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.415405 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:16.162955 21629 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:16.482542 20085 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.984599 21590 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:16.643469 21311 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.645301 21311 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.288961 21627 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:16.647512 21627 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:16.648518 21627 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:16.277357 21650 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.643600 21629 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.650379 21529 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:16.650539 21650 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:16.650995 21529 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:16.651427 21629 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.228101 21462 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.419060 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 923) took 1115 ms. Trace:
I20260709 13:00:16.213719 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:16.377332 21467 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:16.151626 21597 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.835083 21595 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.229638 21527 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:16.199069 21514 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.308367 21605 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.205451 21072 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:16.130934 21123 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.192793 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:16.414425 21631 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
W20260709 13:00:16.186312 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 309040us
W20260709 13:00:16.474402 21579 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 1.045s	user 0.012s	sys 0.004s
W20260709 13:00:16.215667 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 356017us
I20260709 13:00:16.497390 21447 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:16.495862 21513 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:16.475564 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:16.215538 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 376453us
W20260709 13:00:16.190052 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 50968.4us
I20260709 13:00:16.288820 21177 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:16.498626 21575 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:16.123636 21117 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.739002 21645 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:16.740036 21117 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:16.242553 21525 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:16.416756 21622 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.304s	user 0.000s	sys 0.002s
W20260709 13:00:16.412999 21616 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
I20260709 13:00:16.278093 21580 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:16.583879 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:16.333245 21620 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.398s	user 0.001s	sys 0.001s
I20260709 13:00:16.498215 21610 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:16.235350 21259 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:16.602370 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:16.567639 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:16.174939 21632 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:16.596534 21480 consensus_meta.cc:287] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.010s
I20260709 13:00:16.657399 20165 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:16.642200 21619 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:16.647598 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 921) took 1344 ms. Trace:
I20260709 13:00:16.592645 21412 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:16.769558 20017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 201) took 1595 ms (client timeout 1908 ms). Trace:
W20260709 13:00:16.770080 20017 rpcz_store.cc:269] 0709 13:00:15.174645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.228886 (+ 54241us) service_pool.cc:224] Handling call
0709 13:00:16.769381 (+1540495us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b54000fb4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:16.769498 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1479083264}
I20260709 13:00:16.606468 21494 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:16.773370 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.650216 21574 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.774217 21574 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.774858 21123 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.683681 20161 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:16.776510 20161 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:16.732520 20016 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.777268 21574 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.732359 20055 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:16.712277 20162 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
W20260709 13:00:16.780349 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 48084.5us
I20260709 13:00:16.736033 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:16.746752 21447 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:16.781455 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45560.1us
I20260709 13:00:16.781790 21569 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.782565 21646 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:16.782730 21447 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.783203 21646 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:16.708276 21467 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.403635 20159 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:00:16.784380 21638 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.194s	user 0.000s	sys 0.003s
I20260709 13:00:16.784171 21646 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:16.785441 21511 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:16.786069 21511 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.786755 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 547) took 1285 ms. Trace:
I20260709 13:00:16.740727 20008 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:16.741655 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:16.787164 20014 rpcz_store.cc:276] 0709 13:00:15.501307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.501456 (+   149us) service_pool.cc:224] Handling call
0709 13:00:15.501836 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.540331 (+ 38495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.540364 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.540903 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:00:15.714208 (+173305us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:15.714254 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.714327 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.786542 (+1072215us) raft_consensus.cc:1685] finished
0709 13:00:16.786563 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.786699 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1274,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":172882,"threads_started":1,"wal-append.queue_time_us":320214}
W20260709 13:00:16.788784 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47252.3us
I20260709 13:00:16.789999 20014 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:16.617210 21257 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:16.749595 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.754403 20163 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.681253 21649 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:16.759358 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:16.657696 20853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.006s	sys 0.103s
I20260709 13:00:16.734705 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.761921 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:16.675743 21492 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.706734 20854 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:16.745882 21653 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:16.785351 21467 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:16.699302 21640 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.476s	user 0.005s	sys 0.000s
I20260709 13:00:16.793900 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 551) took 1105 ms. Trace:
W20260709 13:00:16.795017 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.516s (SENT) [suppressed 11 similar messages]
I20260709 13:00:16.801796 20014 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:16.836513 20015 rpcz_store.cc:276] 0709 13:00:15.688989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.689141 (+   152us) service_pool.cc:224] Handling call
0709 13:00:15.716920 (+ 27779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.913139 (+196219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.913181 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.913759 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.150875 (+237116us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:16.150930 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.151029 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.783748 (+632719us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:16.793646 (+  9898us) raft_consensus.cc:1685] finished
0709 13:00:16.793667 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.793851 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119494,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":10052,"spinlock_wait_cycles":142003968,"thread_start_us":400395,"threads_started":2,"wal-append.queue_time_us":559769}
I20260709 13:00:16.746055 21605 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
W20260709 13:00:16.820273 20853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.006s	sys 0.108s
I20260709 13:00:16.825139 20163 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
W20260709 13:00:16.830722 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 68930.3us
I20260709 13:00:16.777268 21127 raft_consensus.cc:993] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:16.739722 21645 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:16.795090 21117 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:16.809504 20016 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:16.859695 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.769105 20083 rpcz_store.cc:276] 0709 13:00:15.303352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.311040 (+  7688us) service_pool.cc:224] Handling call
0709 13:00:15.542660 (+231620us) spinlock_profiling.cc:243] Waited 8.98 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:15.542831 (+   171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.928987 (+386156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.647346 (+718359us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54009b8554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:16.647537 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642185856}
I20260709 13:00:16.860980 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:16.835790 21653 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.861934 20084 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.835553 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:16.863088 21632 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.792585 20010 raft_consensus.cc:2376] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4. Re-sending same reply.
I20260709 13:00:16.864221 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:16.864814 20089 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.828949 21447 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:16.845512 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.868557 21123 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:16.868364 21447 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.843217 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.785408 21462 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:16.744386 21513 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.766993 21529 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:16.868162 21511 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:16.830551 21423 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:16.760890 21580 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.829583 21636 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:16.868994 20009 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:16.834303 21574 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.836371 21545 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.663117 20082 rpcz_store.cc:276] 0709 13:00:15.308217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.310994 (+  2777us) service_pool.cc:224] Handling call
0709 13:00:16.418642 (+1107648us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b54005a1754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:16.418956 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106671488}
I20260709 13:00:16.908854 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:16.859571 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:16.881958 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 639) took 1695 ms. Trace:
I20260709 13:00:16.900990 20090 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:16.882786 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 214) took 1044 ms. Trace:
I20260709 13:00:16.876900 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 478) took 1305 ms. Trace:
I20260709 13:00:16.833882 21569 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.912719 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:16.909060 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 53116.6us
I20260709 13:00:16.911597 20859 rpcz_store.cc:276] 0709 13:00:15.186797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.242209 (+ 55412us) service_pool.cc:224] Handling call
0709 13:00:15.242495 (+   286us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:15.242861 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.242886 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:15.258359 (+ 15473us) log.cc:844] Serialized 52 byte log entry
0709 13:00:15.488320 (+229961us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:15.509370 (+ 21050us) log.cc:844] Serialized 14 byte log entry
0709 13:00:15.517326 (+  7956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.517419 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.881715 (+1364296us) raft_consensus.cc:1685] finished
0709 13:00:16.881738 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.881893 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7641,"prepare.queue_time_us":22064,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":58,"raft.queue_time_us":8699,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":177,"thread_start_us":165319,"threads_started":1,"wal-append.queue_time_us":445839}
I20260709 13:00:16.876256 21467 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:16.917572 21467 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.900377 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:16.919689 20083 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:16.920743 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:16.920820 21467 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.921285 21513 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.921850 21529 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:00:16.879532 21642 log.cc:927] Time spent T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:00:16.867801 21645 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:16.924163 21127 raft_consensus.cc:1081] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:16.924628 20085 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:16.925490 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:16.926680 21127 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:16.927078 21127 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:16.927207 21508 consensus_meta.cc:287] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 1.118s	user 0.011s	sys 0.000s
I20260709 13:00:16.927466 21127 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.912122 20086 rpcz_store.cc:276] 0709 13:00:15.838012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.838152 (+   140us) service_pool.cc:224] Handling call
0709 13:00:15.838500 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.765696 (+927196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.765742 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.765798 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:16.765841 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.882425 (+116584us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54006658d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:16.882494 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.882709 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2131828608}
I20260709 13:00:16.863644 21653 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:16.889815 21424 consensus_meta.cc:287] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.032s	user 0.013s	sys 0.000s
I20260709 13:00:16.912348 20858 rpcz_store.cc:276] 0709 13:00:15.571539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.571695 (+   156us) service_pool.cc:224] Handling call
0709 13:00:15.572149 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.578206 (+  6057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.578235 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.584902 (+  6667us) log.cc:844] Serialized 26 byte log entry
0709 13:00:15.891153 (+306251us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:15.891207 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.891298 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.876676 (+985378us) raft_consensus.cc:1685] finished
0709 13:00:16.876700 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.876838 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81610,"raft.queue_time_us":2776,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":12029,"thread_start_us":271024,"threads_started":1,"wal-append.queue_time_us":377522}
I20260709 13:00:16.933903 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.913930 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:16.877115 21605 consensus_queue.cc:1243] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:16.792436 21480 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:16.930594 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 71129.9us
I20260709 13:00:16.937585 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.871356 21447 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:16.945549 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:16.939270 20090 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:16.886730 21619 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:16.900571 21595 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:16.869079 21123 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:16.933090 21563 consensus_meta.cc:287] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.206s	user 0.014s	sys 0.001s
I20260709 13:00:16.944274 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:16.898550 21257 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:16.942714 20089 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:16.945616 20015 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:16.940116 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:16.899612 21650 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:16.832800 21649 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.927635 21508 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.936707 21513 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.928387 21423 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:16.963053 20085 raft_consensus.cc:2410] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:16.947921 21641 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:17.052736 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:16.973320 21447 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.951001 21600 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:16.876026 21494 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:00:16.891698 21576 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:16.934396 21569 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:16.929275 21574 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:16.932170 21632 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:00:16.972499 21545 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:00:17.067575 20086 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.943529 21529 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:00:16.942483 21424 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:16.984897 21563 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4.
I20260709 13:00:16.910789 21636 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:17.073727 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:17.062529 21641 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:17.061540 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:17.086869 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 291847us
I20260709 13:00:17.053965 21513 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:17.089879 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.121572 20086 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:17.123801 21574 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:17.124814 21513 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:17.127117 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.127350 20850 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:17.128080 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.129442 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.130117 20082 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:17.061810 21447 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:17.134905 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.136240 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:17.203603 20089 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:00:17.143172 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 149010us
I20260709 13:00:17.166706 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.113929 21643 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.123797 21569 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:17.184067 20859 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:17.160597 21610 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.199391 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:17.199864 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:17.188205 21544 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.994s	user 0.015s	sys 0.000s
I20260709 13:00:17.228431 21544 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:17.231410 21544 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:17.218242 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:17.233085 21569 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:17.235255 21569 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:17.237914 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:17.239833 21569 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:17.219772 21643 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:17.227888 20164 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:17.268852 21660 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:00:17.283753 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.285374 20850 raft_consensus.cc:2410] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:17.286691 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.288784 21423 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:17.292510 21447 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:17.297600 21649 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:17.299777 21636 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:17.301842 21576 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:17.314518 21127 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:17.222967 21641 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:17.325444 21508 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:17.253335 21610 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:17.595062 21610 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:17.331266 21597 consensus_meta.cc:287] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.000s
I20260709 13:00:17.333305 21123 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:17.762291 21597 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:17.765203 21597 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:17.334470 21580 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:17.905798 21596 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:17.556600 21127 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:17.342713 21492 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.003s
I20260709 13:00:17.320904 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.316614 20854 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:17.328727 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.931059 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:18.784219 21659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.023s	sys 0.060s
W20260709 13:00:18.931430 21676 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.161s	user 0.000s	sys 0.002s
I20260709 13:00:20.382907 21715 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:19.710714 21658 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 2.631s	user 0.000s	sys 0.025s
I20260709 13:00:19.749107 21671 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.384935 21671 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.385402 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 483) took 4511 ms. Trace:
I20260709 13:00:20.385805 20855 rpcz_store.cc:276] 0709 13:00:15.874266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.897051 (+ 22785us) service_pool.cc:224] Handling call
0709 13:00:15.916046 (+ 18995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.648451 (+732405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.648483 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.648990 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.829505 (+180515us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:16.829544 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.924894 (+ 95350us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54000087d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:16.924942 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:19.710067 (+2785125us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:20.385188 (+675121us) raft_consensus.cc:1685] finished
0709 13:00:20.385209 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.385346 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9345,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":17073,"spinlock_wait_cycles":1688978688,"thread_start_us":180079,"threads_started":1,"wal-append.queue_time_us":349450}
I20260709 13:00:20.387099 21671 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:19.750650 21682 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:19.825832 20082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 842) took 2870 ms (client timeout 2004 ms). Trace:
I20260709 13:00:19.837836 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 211) took 4029 ms. Trace:
W20260709 13:00:19.848942 20013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 556) took 4145 ms (client timeout 1706 ms). Trace:
I20260709 13:00:20.387909 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:19.871645 21657 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 2.787s	user 0.080s	sys 0.100s
W20260709 13:00:20.388785 20013 rpcz_store.cc:269] 0709 13:00:15.703759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.774421 (+ 70662us) service_pool.cc:224] Handling call
0709 13:00:19.848513 (+4074092us) spinlock_profiling.cc:243] Waited 4.02 s on lock 0x7b54002ebb54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:19.848805 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9236254464}
I20260709 13:00:19.885398 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 655) took 3520 ms. Trace:
W20260709 13:00:19.888250 20086 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 840) took 2950 ms (client timeout 1538 ms). Trace:
I20260709 13:00:19.894094 21683 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.894747 21685 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:20.398092 20013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 847) took 3474 ms (client timeout 2214 ms). Trace:
I20260709 13:00:20.398314 21685 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.399202 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 929) took 3918 ms. Trace:
I20260709 13:00:19.897397 21686 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.916914 21687 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:19.918562 20011 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:19.920297 21684 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:19.920804 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 249) took 3642 ms. Trace:
W20260709 13:00:19.922854 20016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.060s	user 0.048s	sys 0.089s
W20260709 13:00:19.939303 20055 connection.cc:256] RPC call timeout handler was delayed by -2.7496692009999606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
I20260709 13:00:20.399751 20088 rpcz_store.cc:276] 0709 13:00:16.480420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.480563 (+   143us) service_pool.cc:224] Handling call
0709 13:00:16.518656 (+ 38093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.519379 (+   723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.519423 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.568465 (+ 49042us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.827856 (+259391us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:16.827909 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.828063 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:19.870773 (+3042710us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:20.398867 (+528094us) raft_consensus.cc:1685] finished
0709 13:00:20.398892 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.399127 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4912,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":15620,"spinlock_wait_cycles":4546944,"thread_start_us":174414,"threads_started":1,"wal-append.queue_time_us":447412}
I20260709 13:00:19.940466 21688 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:19.943053 20084 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 3.080s	user 0.028s	sys 0.012s
I20260709 13:00:19.945158 21689 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.420787 20016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.558s	user 0.048s	sys 0.089s
W20260709 13:00:19.952845 20743 connection.cc:256] RPC call timeout handler was delayed by -2.5501873709999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2501s for a response
W20260709 13:00:20.421731 20011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 206) took 4252 ms (client timeout 2233 ms). Trace:
W20260709 13:00:19.958597 20857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 633) took 2834 ms (client timeout 2004 ms). Trace:
W20260709 13:00:19.983270 19839 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 3.332s	user 0.013s	sys 0.052s
W20260709 13:00:20.422758 19839 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.771s	user 0.014s	sys 0.053s
W20260709 13:00:19.987833 20160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.761s	user 0.009s	sys 0.022s
W20260709 13:00:19.998198 20859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 645) took 2755 ms (client timeout 2532 ms). Trace:
I20260709 13:00:20.029322 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:17.069777 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:18.727989 21462 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.557s	user 0.072s	sys 0.111s
I20260709 13:00:19.708875 21480 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:17.908460 21494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.023s	sys 0.035s
W20260709 13:00:20.437613 21494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.366s	user 0.026s	sys 0.035s
I20260709 13:00:19.984855 21582 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:19.751497 21310 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:18.897436 21337 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.000s	sys 0.007s
I20260709 13:00:19.897099 21424 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:17.935067 21600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.019s	sys 0.066s
W20260709 13:00:20.465406 21600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.392s	user 0.021s	sys 0.066s
I20260709 13:00:20.465780 21600 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:20.040482 21500 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:18.727658 21644 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.842s	user 0.004s	sys 0.006s
W20260709 13:00:20.468849 21644 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.584s	user 0.005s	sys 0.006s
I20260709 13:00:20.468261 21500 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:19.970798 21565 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.470376 21565 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.470997 21500 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.466799 21600 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.472548 21565 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:17.939751 21645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.032s	sys 0.080s
W20260709 13:00:20.474849 21645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.484s	user 0.032s	sys 0.082s
W20260709 13:00:18.809942 21590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.163s	user 0.030s	sys 0.030s
W20260709 13:00:20.480515 21590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.834s	user 0.031s	sys 0.030s
W20260709 13:00:19.906917 21601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.041s	user 0.020s	sys 0.016s
W20260709 13:00:20.481947 21601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.618s	user 0.021s	sys 0.017s
W20260709 13:00:18.978106 21641 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.408s	user 0.018s	sys 0.034s
W20260709 13:00:20.486521 21641 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.916s	user 0.018s	sys 0.036s
I20260709 13:00:20.188180 21693 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.042128 21623 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.487329 21693 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.488054 21623 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.489853 21693 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:19.967862 21643 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:20.193591 20008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 554) took 4491 ms (client timeout 3214 ms). Trace:
I20260709 13:00:20.196125 21545 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.198475 20085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 830) took 3363 ms (client timeout 2280 ms). Trace:
I20260709 13:00:20.196595 21177 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.224448 21695 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.491814 21545 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:20.491874 20085 rpcz_store.cc:269] 0709 13:00:16.835166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.835305 (+   139us) service_pool.cc:224] Handling call
0709 13:00:20.198162 (+3362857us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b5400664ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:20.198379 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141690624}
I20260709 13:00:20.493935 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:20.494607 20085 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:20.228183 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 248) took 3954 ms. Trace:
I20260709 13:00:20.492183 21177 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.496085 20157 rpcz_store.cc:276] 0709 13:00:16.277864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.285602 (+  7738us) service_pool.cc:224] Handling call
0709 13:00:20.227224 (+3941622us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:20.227495 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:20.227735 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:20.227769 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:20.227823 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:20.227861 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.227961 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.228105 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7846656}
I20260709 13:00:20.497794 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.229818 21696 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.232661 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 214057us
I20260709 13:00:20.243191 21117 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.498503 21696 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.499374 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.255025 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:20.500569 20157 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:20.500710 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:20.501298 20087 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:20.501493 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:20.502156 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:20.502715 20087 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.502771 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.503690 20157 raft_consensus.cc:2410] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:20.255748 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:20.256369 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:20.504205 20090 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:20.504622 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.504953 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:20.262012 20012 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:20.492422 21695 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.269743 21064 raft_consensus.cc:3055] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:00:20.491607 20008 rpcz_store.cc:269] 0709 13:00:15.702159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.702266 (+   107us) service_pool.cc:224] Handling call
0709 13:00:20.193296 (+4491030us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b54009155d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:20.193486 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278029056}
I20260709 13:00:20.508137 21064 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:20.508837 20008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 841) took 3587 ms (client timeout 3029 ms). Trace:
I20260709 13:00:20.509325 21064 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.510735 21064 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.511268 21696 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:20.273795 20744 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.071s	user 0.025s	sys 0.068s
I20260709 13:00:20.511854 21695 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:20.512751 20744 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.310s	user 0.026s	sys 0.068s
W20260709 13:00:20.513200 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 3.31081e+06us
I20260709 13:00:20.281590 21701 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.505098 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:20.516321 20090 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.516239 21701 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.519207 21701 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.286011 21702 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.520288 21702 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.290405 21493 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:20.494828 21310 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.524307 21493 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.527140 21493 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:20.292572 19982 connection.cc:256] RPC call timeout handler was delayed by -1.73340745600035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 13:00:17.287556 20850 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:20.299160 21690 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:20.299463 21703 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:20.299575 21705 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:20.309824 20161 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:20.310491 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.311373 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.313145 20164 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:20.314525 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 244) took 4094 ms. Trace:
I20260709 13:00:20.314306 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.321579 21708 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:20.320816 21707 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:20.331679 21699 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.346993 20851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 638) took 3221 ms (client timeout 2116 ms). Trace:
W20260709 13:00:20.347662 21627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.469s	user 0.012s	sys 0.023s
I20260709 13:00:20.348027 21709 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.367820 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 58733.1us
W20260709 13:00:20.376878 20130 connection.cc:256] RPC call timeout handler was delayed by -2.1909905190004793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:00:20.379278 20056 connection.cc:256] RPC call timeout handler was delayed by -3.3284741500001473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
I20260709 13:00:20.379420 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 627) took 3288 ms. Trace:
W20260709 13:00:20.532016 20056 connection.cc:256] RPC call timeout handler was delayed by -3.2384741500000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 13:00:20.381991 20856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 642) took 3175 ms (client timeout 1555 ms). Trace:
I20260709 13:00:20.387797 21682 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:20.532763 20856 rpcz_store.cc:269] 0709 13:00:17.206769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.260550 (+ 53781us) service_pool.cc:224] Handling call
0709 13:00:20.381892 (+3121342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.532895 21705 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.533779 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.389596 20855 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.534889 21682 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.532167 21708 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.532202 21703 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:20.537752 20056 connection.cc:256] RPC call timeout handler was delayed by -3.2384741500000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 13:00:20.534225 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:20.538237 20056 connection.cc:256] RPC call timeout handler was delayed by -2.9882741499995973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 13:00:20.538578 20056 connection.cc:256] RPC call timeout handler was delayed by -2.8298741499993412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 13:00:20.532617 20130 connection.cc:256] RPC call timeout handler was delayed by -2.0973905190003279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 13:00:20.538841 20056 connection.cc:256] RPC call timeout handler was delayed by -2.6489741499999582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23610000000000003s for a response
W20260709 13:00:20.538980 20130 connection.cc:256] RPC call timeout handler was delayed by -2.0034806670009857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 13:00:20.539321 20056 connection.cc:256] RPC call timeout handler was delayed by -1.8812741499996264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3214s for a response
I20260709 13:00:20.539471 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:20.539714 20056 connection.cc:256] RPC call timeout handler was delayed by -1.8812741499996264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3214s for a response
W20260709 13:00:20.396214 20086 rpcz_store.cc:269] 0709 13:00:16.937651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.939624 (+  1973us) service_pool.cc:224] Handling call
0709 13:00:19.887858 (+2948234us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b54004c6154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:19.888160 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203850112}
W20260709 13:00:20.539564 20130 connection.cc:256] RPC call timeout handler was delayed by -1.9647202480000487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:00:20.383592 21659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.586s	user 0.023s	sys 0.061s
W20260709 13:00:20.388415 20082 rpcz_store.cc:269] 0709 13:00:16.955467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.973049 (+ 17582us) service_pool.cc:224] Handling call
0709 13:00:19.825697 (+2852648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.540923 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:20.541599 20164 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.532507 21690 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:20.398813 20013 rpcz_store.cc:269] 0709 13:00:16.923261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.397791 (+3474530us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:20.397999 (+   208us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:00:19.905253 20009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 843) took 2983 ms (client timeout 1672 ms). Trace:
I20260709 13:00:20.544384 20856 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
W20260709 13:00:20.544514 20009 rpcz_store.cc:269] 0709 13:00:16.922300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.904983 (+2982683us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:19.905156 (+   173us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:20.532346 20854 rpcz_store.cc:276] 0709 13:00:17.090397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.112519 (+ 22122us) service_pool.cc:224] Handling call
0709 13:00:17.213900 (+101381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:20.378834 (+3164934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.379177 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.394971 20165 rpcz_store.cc:276] 0709 13:00:16.365402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.778083 (+412681us) service_pool.cc:224] Handling call
0709 13:00:16.778422 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.792920 (+ 14498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.792949 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.819468 (+ 26519us) log.cc:844] Serialized 26 byte log entry
0709 13:00:17.080828 (+261360us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:17.080873 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.884885 (+2804012us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b54004d6f54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:19.884946 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:19.885008 (+    62us) raft_consensus.cc:1685] finished
0709 13:00:19.885024 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.885296 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32807296,"thread_start_us":184232,"threads_started":1,"wal-append.queue_time_us":348284,"wal-append.run_cpu_time_us":2290,"wal-append.run_wall_time_us":1126665}
I20260709 13:00:20.545847 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.546569 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.547168 20165 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:20.546795 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:19.954005 20017 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 3.181s	user 0.019s	sys 0.033s
W20260709 13:00:20.423552 20859 rpcz_store.cc:269] 0709 13:00:17.242361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.977623 (+2735262us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:19.997626 (+ 20003us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:20.534523 20163 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:20.546903 20856 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:20.548859 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.424898 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 831) took 3603 ms. Trace:
W20260709 13:00:20.425693 20160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.199s	user 0.010s	sys 0.022s
I20260709 13:00:20.550758 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 252) took 3351 ms. Trace:
I20260709 13:00:20.423239 20084 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:20.383533 21715 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:20.554572 19982 connection.cc:256] RPC call timeout handler was delayed by -1.6560074560002249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
I20260709 13:00:20.415776 21683 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.423731 21714 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:20.420364 20156 rpcz_store.cc:276] 0709 13:00:16.278403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.286582 (+  8179us) service_pool.cc:224] Handling call
0709 13:00:16.887484 (+600902us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:16.887805 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.919780 (+ 31975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.919818 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:16.919873 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:16.919908 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.920521 (+3000613us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b54004d6554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:19.920580 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.920739 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79712512}
I20260709 13:00:20.420516 21686 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.558342 21706 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.426009 21713 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:20.427285 21716 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.423255 21688 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.425832 21689 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.436684 21704 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.429546 21685 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.420137 21687 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.422540 21684 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.645152 20164 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
W20260709 13:00:20.439360 21462 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.302s	user 0.076s	sys 0.112s
W20260709 13:00:20.439932 20852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 637) took 3306 ms (client timeout 3334 ms). Trace:
W20260709 13:00:20.424778 20857 rpcz_store.cc:269] 0709 13:00:17.123822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.958318 (+2834496us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:19.958499 (+   181us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:20.647251 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:20.444716 20743 connection.cc:256] RPC call timeout handler was delayed by -1.8914455330004785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2891s for a response
W20260709 13:00:20.648181 20743 connection.cc:256] RPC call timeout handler was delayed by -1.631609190000745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 13:00:20.538841 21703 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:20.445264 21719 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.446964 21721 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.659363 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:20.451884 21337 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.455s	user 0.000s	sys 0.007s
I20260709 13:00:20.575096 21683 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:20.490422 21623 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.494828 21545 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.497932 21177 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.228734 21697 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.498710 21117 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:20.499003 20129 connection.cc:256] RPC call timeout handler was delayed by -2.5340437729992118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
I20260709 13:00:20.253199 20014 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:20.505414 20157 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:20.557176 20084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 829) took 3727 ms (client timeout 1793 ms). Trace:
I20260709 13:00:20.278347 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 480) took 4697 ms. Trace:
W20260709 13:00:20.381994 20858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 647) took 3064 ms (client timeout 2117 ms). Trace:
I20260709 13:00:20.388531 20091 rpcz_store.cc:276] 0709 13:00:15.808445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.808625 (+   180us) service_pool.cc:224] Handling call
0709 13:00:15.843265 (+ 34640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.951545 (+108280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.951589 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:15.951645 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:15.951683 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:19.837363 (+3885680us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400551c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:19.837437 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:19.837621 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248495488}
W20260709 13:00:20.540491 20130 connection.cc:256] RPC call timeout handler was delayed by -1.9455072260007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
I20260709 13:00:20.540802 21710 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.535364 21707 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:20.422395 20011 rpcz_store.cc:269] 0709 13:00:16.169717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.169857 (+   140us) service_pool.cc:224] Handling call
0709 13:00:20.421552 (+4251695us) spinlock_profiling.cc:243] Waited 3.32 s on lock 0x7b540064f5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:20.421669 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7646893312}
I20260709 13:00:20.551415 20090 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:20.552004 20162 raft_consensus.cc:2410] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:20.695457 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:20.554268 20087 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
W20260709 13:00:20.556794 20017 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.783s	user 0.020s	sys 0.033s
I20260709 13:00:20.555763 19981 leader_election.cc:400] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
W20260709 13:00:20.558387 19982 connection.cc:256] RPC call timeout handler was delayed by -1.6164074560001609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 13:00:20.697316 19982 connection.cc:256] RPC call timeout handler was delayed by -1.523707456000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
I20260709 13:00:20.696373 20853 rpcz_store.cc:276] 0709 13:00:15.581147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.581264 (+   117us) service_pool.cc:224] Handling call
0709 13:00:15.600145 (+ 18881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.861264 (+261119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.861300 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.891447 (+ 30147us) log.cc:844] Serialized 26 byte log entry
0709 13:00:20.025422 (+4133975us) spinlock_profiling.cc:243] Waited 50.7 ms on lock 0x7b240051a5f8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b703fb0 00007fa02e37a488 00007fa02e37abe8 00007fa02e352398 00007fa02e34d76a 00007fa02e34cbb6
0709 13:00:20.277611 (+252189us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b18003b4f40. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1
0709 13:00:20.277838 (+   227us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:20.277894 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.277995 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:20.278050 (+    55us) raft_consensus.cc:1685] finished
0709 13:00:20.278069 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.278238 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81556,"raft.queue_time_us":82514,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"spinlock_wait_cycles":120715904,"thread_start_us":1053102,"threads_started":2,"wal-append.queue_time_us":844664}
W20260709 13:00:20.697510 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 141940us
W20260709 13:00:20.697695 19982 connection.cc:256] RPC call timeout handler was delayed by -1.5030074559999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 13:00:20.698243 19982 connection.cc:256] RPC call timeout handler was delayed by -1.1547074560003239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23010000000000003s for a response
I20260709 13:00:20.545912 20009 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:20.701294 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 834) took 3881 ms. Trace:
I20260709 13:00:20.701768 20009 rpcz_store.cc:276] 0709 13:00:16.819493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.545320 (+3725827us) service_pool.cc:224] Handling call
0709 13:00:20.545594 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:20.700939 (+155345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.701224 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.703404 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:20.703655 20858 rpcz_store.cc:269] 0709 13:00:17.317889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.930528 (+612639us) service_pool.cc:224] Handling call
0709 13:00:20.381907 (+2451379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.553066 20158 rpcz_store.cc:276] 0709 13:00:16.220030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.220193 (+   163us) service_pool.cc:224] Handling call
0709 13:00:16.220508 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.220755 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:16.676912 (+456157us) log.cc:844] Serialized 14 byte log entry
0709 13:00:20.023734 (+3346822us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b18001eb200. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 13:00:20.230807 (+207073us) log.cc:844] Serialized 14 byte log entry
0709 13:00:20.314144 (+ 83337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:20.314206 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:20.314271 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.314335 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.314456 (+   121us) inbound_call.cc:177] Queueing success response
0709 13:00:20.327480 (+ 13024us) spinlock_profiling.cc:243] Waited 42.2 ms on lock 0x7b18001eb200. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
Metrics: {"prepare.queue_time_us":650743,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":85,"raft.queue_time_us":17669,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":14171,"spinlock_wait_cycles":109504768,"thread_start_us":990275,"threads_started":5,"wal-append.queue_time_us":462478,"wal-append.run_cpu_time_us":2888,"wal-append.run_wall_time_us":1180685}
I20260709 13:00:20.549866 20016 rpcz_store.cc:276] 0709 13:00:16.818754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.861136 (+ 42382us) service_pool.cc:224] Handling call
0709 13:00:16.861512 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.861873 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.861901 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.862379 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:00:20.421254 (+3558875us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:20.421294 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.421398 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:20.421443 (+    45us) raft_consensus.cc:1685] finished
0709 13:00:20.421455 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.421575 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6462,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":3558514,"threads_started":1,"wal-append.queue_time_us":349648,"wal-append.run_cpu_time_us":3522,"wal-append.run_wall_time_us":1101110}
I20260709 13:00:20.563475 20089 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:20.564260 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:20.564356 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:20.509186 20008 rpcz_store.cc:269] 0709 13:00:16.921644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.508648 (+3587004us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:20.508751 (+   103us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:20.706418 20163 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:20.544374 20013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 845) took 3621 ms (client timeout 3334 ms). Trace:
W20260709 13:00:20.707096 20013 rpcz_store.cc:269] 0709 13:00:16.922781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.544201 (+3621420us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:20.544312 (+   111us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:20.551102 20160 rpcz_store.cc:276] 0709 13:00:17.199790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.200004 (+   214us) service_pool.cc:224] Handling call
0709 13:00:17.200370 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.200609 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:17.226610 (+ 26001us) log.cc:844] Serialized 14 byte log entry
0709 13:00:20.550429 (+3323819us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:20.550498 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:20.550539 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.550602 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.550706 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3030235,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":3323389,"threads_started":1,"wal-append.queue_time_us":1417056,"wal-append.run_cpu_time_us":1377,"wal-append.run_wall_time_us":1003528}
I20260709 13:00:20.578147 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:20.532321 21627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.654s	user 0.013s	sys 0.023s
I20260709 13:00:20.540339 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:20.645900 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:20.841524 20856 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.571839 20855 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:20.523209 21702 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.548168 20082 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:20.444061 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:20.444929 21213 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:20.627794 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:20.652590 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 206747us
W20260709 13:00:20.653172 20744 connection.cc:256] RPC call timeout handler was delayed by -3.2978089509997517s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:00:20.507781 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 833) took 3688 ms. Trace:
I20260709 13:00:20.660152 20164 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:20.657022 21493 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.660708 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.574415 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.447032 21582 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.663115 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:20.532871 20850 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 641) took 3326 ms (client timeout 1798 ms). Trace:
I20260709 13:00:20.576040 21686 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.495453 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:20.668952 20129 connection.cc:256] RPC call timeout handler was delayed by -2.3878568640002413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
I20260709 13:00:20.557925 20159 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:20.573284 21715 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:20.557032 21708 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:20.695667 20130 connection.cc:256] RPC call timeout handler was delayed by -1.7695742229998359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 13:00:20.877192 20130 connection.cc:256] RPC call timeout handler was delayed by -1.5508742230003918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 13:00:20.877604 20130 connection.cc:256] RPC call timeout handler was delayed by -1.409250649000569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
I20260709 13:00:20.704978 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:20.713295 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.708343 20161 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:20.662408 21684 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:20.745610 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.714406 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.745376 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.581373 21701 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
W20260709 13:00:20.577014 20851 rpcz_store.cc:269] 0709 13:00:17.125483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.320431 (+194948us) service_pool.cc:224] Handling call
0709 13:00:20.346839 (+3026408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.752645 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:20.860366 21213 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:20.754810 20090 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:20.716560 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56824.7us
I20260709 13:00:20.668434 21643 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:20.662559 21687 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.670424 21693 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:20.664777 21695 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:20.766085 20091 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:20.670301 21683 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:20.894251 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.894475 21683 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.895033 20851 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:20.557312 21699 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.895872 21600 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:20.870082 21493 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.896749 21683 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.898000 21699 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.576480 21713 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.898983 21493 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:20.647354 20852 rpcz_store.cc:269] 0709 13:00:17.125154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.028967 (+2903813us) service_pool.cc:224] Handling call
0709 13:00:20.431526 (+402559us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.899935 21707 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:20.645090 21704 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.900491 21713 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:20.662902 21697 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.902042 21704 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.575574 21706 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:20.902487 21697 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:20.769160 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.904311 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.904482 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.905164 20851 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:20.905473 20852 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:20.769940 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.906935 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.907601 20165 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:20.706704 21645 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:20.908444 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.772030 21705 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:20.696076 20084 rpcz_store.cc:269] 0709 13:00:16.829294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.833559 (+  4265us) service_pool.cc:224] Handling call
0709 13:00:20.557091 (+3723532us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.988564 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:20.746392 19981 connection.cc:256] RPC call timeout handler was delayed by -2.6607550990001982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
W20260709 13:00:20.996734 19981 connection.cc:256] RPC call timeout handler was delayed by -2.6607550990001982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
I20260709 13:00:20.706606 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:20.999387 20083 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:20.745187 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:20.764153 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:20.713800 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:21.002135 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:20.761765 20014 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 836) took 3919 ms (client timeout 1868 ms). Trace:
I20260709 13:00:20.751000 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 211) took 4579 ms. Trace:
W20260709 13:00:21.004364 20014 rpcz_store.cc:269] 0709 13:00:16.842057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.842951 (+   894us) service_pool.cc:224] Handling call
0709 13:00:20.761459 (+3918508us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b540064e6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:20.761669 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7679935744}
I20260709 13:00:20.982429 20012 rpcz_store.cc:276] 0709 13:00:16.819270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.878908 (+ 59638us) service_pool.cc:224] Handling call
0709 13:00:20.261781 (+3382873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:20.506041 (+244260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.507564 (+  1523us) spinlock_profiling.cc:243] Waited 3.38 s on lock 0x7b54009155d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:20.507709 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7778950016}
I20260709 13:00:21.004549 20017 rpcz_store.cc:276] 0709 13:00:16.171036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.771192 (+600156us) service_pool.cc:224] Handling call
0709 13:00:16.771583 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.773002 (+  1419us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:20.697289 (+3924287us) log.cc:844] Serialized 14 byte log entry
0709 13:00:20.750110 (+ 52821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:20.750188 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:20.750256 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.750352 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:20.750846 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2019,"prepare.queue_time_us":471971,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":17983,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":17405,"thread_start_us":3973800,"threads_started":2,"wal-append.queue_time_us":89965}
I20260709 13:00:20.873106 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.002247 20016 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:20.870725 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.010725 20085 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:20.887024 20087 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:20.781697 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:20.872430 20129 connection.cc:256] RPC call timeout handler was delayed by -1.8535501499991369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 13:00:21.017644 20129 connection.cc:256] RPC call timeout handler was delayed by -1.8535501499991369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 13:00:21.018178 20129 connection.cc:256] RPC call timeout handler was delayed by -1.8202501499990831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 13:00:21.018944 20129 connection.cc:256] RPC call timeout handler was delayed by -1.6996501499997976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:00:21.019394 20129 connection.cc:256] RPC call timeout handler was delayed by -1.6556064100004733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 13:00:20.872025 20156 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:20.770577 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:21.025393 20157 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:20.881348 20160 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:21.031148 20008 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:20.883016 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.880227 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:20.889199 20162 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:21.047403 20163 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:20.866611 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:21.051348 21686 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:20.870088 20850 rpcz_store.cc:269] 0709 13:00:17.206373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.286291 (+ 79918us) service_pool.cc:224] Handling call
0709 13:00:20.532785 (+3246494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.906911 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:21.063344 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.066349 20851 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:20.754033 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.765797 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:21.068068 20164 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:21.069257 20854 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.888093 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.870723 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.075649 20859 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:21.077558 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.078538 20859 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:20.917311 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:20.844300 20855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 648) took 3506 ms (client timeout 3373 ms). Trace:
W20260709 13:00:21.089679 20855 rpcz_store.cc:269] 0709 13:00:17.337911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.387620 (+3049709us) service_pool.cc:224] Handling call
0709 13:00:20.844219 (+456599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:20.857764 20856 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:20.922308 21064 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.092232 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.093043 21064 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.093529 20856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 13:00:21.095041 20856 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:20.804324 21117 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.926136 21213 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.087574 21644 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.775775 21338 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.844292 21480 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.925978 21582 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:21.102766 21064 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:20.892177 21643 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.908926 21645 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.121613 21685 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.097386 21706 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.122663 21685 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.777931 21689 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.858970 21690 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:20.912824 21693 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.131774 21685 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.132437 21693 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:20.897527 21695 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.803464 21688 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:20.532240 21709 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.096549 21713 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:21.134563 21713 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.135020 21713 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:20.713837 21710 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:20.753017 21712 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.138530 21712 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:20.573849 21714 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.576743 21716 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.142951 21645 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.143352 20011 raft_consensus.cc:2410] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:21.144568 21716 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.669157 21719 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:20.860778 21723 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.668766 21721 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:20.898150 21729 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:00:20.984308 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.148430 21695 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:20.984898 20015 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:21.149019 20082 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:21.151736 21721 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:21.149017 20009 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:20.990559 20084 raft_consensus.cc:2410] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:00:20.997720 19981 connection.cc:256] RPC call timeout handler was delayed by -2.5365550989999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:00:20.846613 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 306321us
I20260709 13:00:21.162689 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:21.008888 20012 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:21.019804 20129 connection.cc:256] RPC call timeout handler was delayed by -1.6520064100004674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260709 13:00:21.165526 20129 connection.cc:256] RPC call timeout handler was delayed by -1.6520064100004674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260709 13:00:21.166862 20129 connection.cc:256] RPC call timeout handler was delayed by -1.5926064100003714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 13:00:21.167387 20129 connection.cc:256] RPC call timeout handler was delayed by -1.5232501499995124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:00:21.167856 20129 connection.cc:256] RPC call timeout handler was delayed by -1.449450149999393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:00:21.168530 20129 connection.cc:256] RPC call timeout handler was delayed by -1.449450149999393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:00:21.168912 20129 connection.cc:256] RPC call timeout handler was delayed by -1.429650149999361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
I20260709 13:00:21.031682 20008 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:21.007687 20014 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:21.127586 21643 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:21.051584 20884 debug-util.cc:398] Leaking SignalData structure 0x7b08000bc700 after lost signal to thread 12412
W20260709 13:00:20.863190 20744 connection.cc:256] RPC call timeout handler was delayed by -3.1619089510004414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
I20260709 13:00:21.040781 20156 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:21.069169 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:20.888360 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.174552 21645 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:21.185282 20858 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:21.185336 21645 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.052502 21686 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:21.191246 20884 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5a20 after lost signal to thread 19874
I20260709 13:00:21.099745 20857 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:21.098321 21213 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:21.216542 20015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 844) took 4293 ms (client timeout 1651 ms). Trace:
W20260709 13:00:21.217128 20015 rpcz_store.cc:269] 0709 13:00:16.922544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.058663 (+136119us) service_pool.cc:224] Handling call
0709 13:00:21.216079 (+4157416us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540097a254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:21.216394 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284917120}
I20260709 13:00:21.100773 21480 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.095855 20856 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:21.104091 20013 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:21.129472 20161 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:21.099052 21338 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:21.160542 19981 connection.cc:256] RPC call timeout handler was delayed by -2.3655550989997209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
I20260709 13:00:21.153765 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.037492 21705 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:21.171120 20129 connection.cc:256] RPC call timeout handler was delayed by -1.4129565779994664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 13:00:21.096618 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.170060 21719 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.167613 21712 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:21.181653 21690 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.182004 20850 raft_consensus.cc:2410] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:00:21.193225 20744 connection.cc:256] RPC call timeout handler was delayed by -3.0872089510003207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 13:00:21.193100 21686 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.196564 20008 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.086205 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.201942 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.202515 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 839) took 4273 ms. Trace:
W20260709 13:00:21.208005 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 376034us
I20260709 13:00:21.242280 21714 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.221599 21645 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.232216 21480 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:21.235800 21709 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.234721 21710 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:21.098235 21644 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.237195 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 652) took 1034 ms. Trace:
W20260709 13:00:21.219885 20884 debug-util.cc:398] Leaking SignalData structure 0x7b0800221aa0 after lost signal to thread 20041
W20260709 13:00:21.265666 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:21.314852 20884 debug-util.cc:398] Leaking SignalData structure 0x7b08001905a0 after lost signal to thread 20042
I20260709 13:00:21.315402 20010 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:21.314649 20857 rpcz_store.cc:276] 0709 13:00:20.202475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.876903 (+674428us) service_pool.cc:224] Handling call
0709 13:00:21.236909 (+360006us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b54000096d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:21.237128 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55747968}
I20260709 13:00:21.292980 21690 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:21.316412 21690 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:21.317160 20884 debug-util.cc:398] Leaking SignalData structure 0x7b08000f4f40 after lost signal to thread 20115
W20260709 13:00:21.317543 20884 debug-util.cc:398] Leaking SignalData structure 0x7b08002a4b80 after lost signal to thread 20116
I20260709 13:00:21.318481 21644 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:21.313107 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 159612us
I20260709 13:00:21.317216 20857 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:21.241961 20854 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:21.280236 20013 pending_rounds.cc:85] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 1
I20260709 13:00:21.321841 20013 pending_rounds.cc:107] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 3.2
W20260709 13:00:21.281849 19981 connection.cc:256] RPC call timeout handler was delayed by -2.1099550990002172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 13:00:21.294909 20129 connection.cc:256] RPC call timeout handler was delayed by -1.4008501499993145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:00:21.285215 19840 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 4, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:21.296075 20083 rpcz_store.cc:276] 0709 13:00:16.929381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.930668 (+  1287us) service_pool.cc:224] Handling call
0709 13:00:20.573359 (+3642691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.000261 (+426902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.170966 (+170705us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54009803d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:21.202188 (+ 31222us) spinlock_profiling.cc:243] Waited 3.62 s on lock 0x7b54009803d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:21.202426 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9288295936}
W20260709 13:00:21.304551 20744 connection.cc:256] RPC call timeout handler was delayed by -2.7326089509997473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
I20260709 13:00:21.305058 21493 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:21.304978 21705 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.305665 20008 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.306481 20859 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:21.307583 21686 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:21.309890 21684 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:21.310062 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 108466us
I20260709 13:00:21.310632 21338 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:21.311134 21601 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:21.296727 20856 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:21.322631 19981 connection.cc:256] RPC call timeout handler was delayed by -1.86425509899982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24910000000000002s for a response
I20260709 13:00:21.322222 20013 raft_consensus.cc:2889] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:21.319482 20884 debug-util.cc:398] Leaking SignalData structure 0x7b08003ca380 after lost signal to thread 20189
I20260709 13:00:21.324412 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 653) took 1119 ms. Trace:
W20260709 13:00:21.325366 20744 connection.cc:256] RPC call timeout handler was delayed by -2.6525089510005273s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22330000000000003s for a response
W20260709 13:00:21.326373 20129 connection.cc:256] RPC call timeout handler was delayed by -1.4008501499993145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:00:21.329089 20857 rpcz_store.cc:276] 0709 13:00:20.203502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.316298 (+1112796us) service_pool.cc:224] Handling call
0709 13:00:21.316883 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.321392 (+  4509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.322365 (+   973us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:21.335340 20129 connection.cc:256] RPC call timeout handler was delayed by -1.1942565780000223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260709 13:00:21.342116 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:21.342644 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.343380 20014 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:21.343474 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:21.344167 20009 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:21.345642 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 643) took 4125 ms. Trace:
I20260709 13:00:21.346845 19840 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 4, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 4 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:21.348376 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:21.346303 20852 rpcz_store.cc:276] 0709 13:00:17.220054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.097231 (+3877177us) service_pool.cc:224] Handling call
0709 13:00:21.097915 (+   684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:21.098682 (+   767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:21.098726 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:21.099474 (+   748us) log.cc:844] Serialized 26 byte log entry
0709 13:00:21.313816 (+214342us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:21.314289 (+   473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.322985 (+  8696us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.344964 (+ 21979us) raft_consensus.cc:1685] finished
0709 13:00:21.345007 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.345332 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2570,"raft.run_cpu_time_us":439,"raft.run_wall_time_us":19707,"thread_start_us":192322,"threads_started":1,"wal-append.queue_time_us":240343}
W20260709 13:00:21.349562 20129 connection.cc:256] RPC call timeout handler was delayed by -1.1741064099996947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
I20260709 13:00:21.313374 21713 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:21.351902 20129 connection.cc:256] RPC call timeout handler was delayed by -1.1741064099996947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
W20260709 13:00:21.352339 20129 connection.cc:256] RPC call timeout handler was delayed by -1.113971902999765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 13:00:21.352677 20129 connection.cc:256] RPC call timeout handler was delayed by -1.0859064100004616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
W20260709 13:00:21.352957 20129 connection.cc:256] RPC call timeout handler was delayed by -1.0397468370001661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
I20260709 13:00:21.356423 20856 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.349566 19840 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 6, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.358739 21719 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:21.359562 21719 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.360531 21705 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.326707 21493 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.361521 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 644) took 4134 ms. Trace:
I20260709 13:00:21.355754 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 747) took 1006 ms. Trace:
I20260709 13:00:21.361936 20858 rpcz_store.cc:276] 0709 13:00:17.226950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.186525 (+3959575us) service_pool.cc:224] Handling call
0709 13:00:21.187172 (+   647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:21.202159 (+ 14987us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:21.272313 (+ 70154us) log.cc:844] Serialized 14 byte log entry
0709 13:00:21.273932 (+  1619us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:21.275142 (+  1210us) log.cc:844] Serialized 26 byte log entry
0709 13:00:21.275595 (+   453us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:21.275642 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.275742 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.357497 (+ 81755us) raft_consensus.cc:1685] finished
0709 13:00:21.357514 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.361452 (+  3938us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14214,"prepare.queue_time_us":149107,"raft.queue_time_us":15303,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":232,"thread_start_us":70598,"threads_started":2,"wal-append.queue_time_us":1077}
I20260709 13:00:21.363005 20158 rpcz_store.cc:276] 0709 13:00:20.314921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.884154 (+569233us) service_pool.cc:224] Handling call
0709 13:00:20.884529 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.126369 (+241840us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:21.126927 (+   558us) log.cc:844] Serialized 14 byte log entry
0709 13:00:21.266214 (+139287us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:21.266272 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:21.266312 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.294232 (+ 27920us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54001be4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:21.294267 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.321162 (+ 26895us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":971,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":160,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":153,"spinlock_wait_cycles":551424256,"thread_start_us":138148,"threads_started":1,"wal-append.queue_time_us":138721}
I20260709 13:00:21.364763 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 835) took 4537 ms. Trace:
I20260709 13:00:21.365244 20013 rpcz_store.cc:276] 0709 13:00:16.827524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.003762 (+4176238us) service_pool.cc:224] Handling call
0709 13:00:21.013289 (+  9527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.364442 (+351153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.364678 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:21.366096 20744 connection.cc:256] RPC call timeout handler was delayed by -2.4113089510001373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2501s for a response
I20260709 13:00:21.366182 21480 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:21.366786 20744 connection.cc:256] RPC call timeout handler was delayed by -2.0603089510004793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2891s for a response
I20260709 13:00:21.366895 20859 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:21.367053 21480 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:21.367197 20744 connection.cc:256] RPC call timeout handler was delayed by -2.020494817999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
W20260709 13:00:21.368251 20744 connection.cc:256] RPC call timeout handler was delayed by -1.5623948180000298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
W20260709 13:00:21.368577 20744 connection.cc:256] RPC call timeout handler was delayed by -1.5623948180000298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260709 13:00:21.374315 21713 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.377862 21696 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:21.379550 20853 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
W20260709 13:00:21.381513 20884 debug-util.cc:398] Leaking SignalData structure 0x7b0800344b60 after lost signal to thread 20888
I20260709 13:00:21.384713 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 639) took 4258 ms. Trace:
I20260709 13:00:21.385174 20855 rpcz_store.cc:276] 0709 13:00:17.125767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.091348 (+3965581us) service_pool.cc:224] Handling call
0709 13:00:21.125258 (+ 33910us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:21.184008 (+ 58750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:21.184047 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:21.184744 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 13:00:21.273478 (+ 88734us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:21.273538 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.273634 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.384372 (+110738us) raft_consensus.cc:1685] finished
0709 13:00:21.384394 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.384536 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7957,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":189,"thread_start_us":87958,"threads_started":1,"wal-append.queue_time_us":94767}
I20260709 13:00:21.386755 21338 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:21.388247 21712 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:21.389717 21714 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:21.390179 21714 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:21.351086 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.391575 21213 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:21.391575 21714 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:21.392829 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 61791.6us
I20260709 13:00:21.393579 21601 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:21.393818 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.394245 21601 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:21.394690 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:21.394847 21644 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:21.396198 21688 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:21.397009 21644 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:21.397351 21688 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:21.397809 21644 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:21.398147 19982 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:21.398499 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 649) took 1402 ms. Trace:
I20260709 13:00:21.398891 21688 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.395216 21601 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:21.412839 21719 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:21.399199 21644 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:21.392803 21213 consensus_queue.cc:1243] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:21.403761 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:21.399362 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.406513 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:21.402179 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:21.393682 21714 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:21.413389 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.401525 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.398968 20850 rpcz_store.cc:276] 0709 13:00:19.995657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.303507 (+1307850us) service_pool.cc:224] Handling call
0709 13:00:21.321017 (+ 17510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.362157 (+ 41140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:21.362191 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:21.362252 (+    61us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:21.362282 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.398150 (+ 35868us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b54000091d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:21.398215 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.398380 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93086976}
I20260709 13:00:21.409844 21493 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:21.400687 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 832) took 4581 ms. Trace:
I20260709 13:00:21.402395 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:21.416865 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.426507 21480 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:21.444846 21732 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.446026 20014 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:21.458417 20017 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:21.459687 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:44210 (request call id 640) took 4230 ms. Trace:
I20260709 13:00:21.452422 20015 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:21.461208 20009 raft_consensus.cc:2410] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:21.433090 20854 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:21.463975 20856 rpcz_store.cc:276] 0709 13:00:17.126083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.092939 (+3966856us) service_pool.cc:224] Handling call
0709 13:00:21.356834 (+263895us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:21.462905 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 59322.7us
I20260709 13:00:21.470257 20008 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:00:21.471187 20008 pending_rounds.cc:85] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:21.471735 20008 pending_rounds.cc:107] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:00:21.472096 20008 raft_consensus.cc:2889] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:21.461405 20013 raft_consensus.cc:2376] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4. Re-sending same reply.
W20260709 13:00:21.478790 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46488.2us
I20260709 13:00:21.457194 21732 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.493942 21732 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.490497 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.461941 20012 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:21.478776 20852 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.505549 20055 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.518302 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.502592 21713 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:21.529204 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.531010 20009 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:21.511336 21684 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:21.446771 20010 rpcz_store.cc:276] 0709 13:00:16.819011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.864481 (+ 45470us) service_pool.cc:224] Handling call
0709 13:00:16.864866 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:21.315866 (+4451000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.400297 (+ 84431us) spinlock_profiling.cc:243] Waited 4.45 s on lock 0x7b54004a74d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:21.400516 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10230372224}
W20260709 13:00:21.539150 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 47901.6us
I20260709 13:00:21.530681 21713 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:21.543972 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.542421 20015 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
W20260709 13:00:21.542207 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46840us
I20260709 13:00:21.547644 20017 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:00:21.548578 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:21.549311 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.554039 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:21.554814 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 138330us
I20260709 13:00:21.550096 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:21.568111 19840 catalog_manager.cc:5697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 5, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 5 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.546314 21713 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.556704 21565 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:21.570964 19840 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 4, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 4 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.572810 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 489) took 1141 ms. Trace:
I20260709 13:00:21.576612 20016 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:00:21.590067 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 57106.7us
I20260709 13:00:21.581591 20857 rpcz_store.cc:276] 0709 13:00:20.431255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.339873 (+908618us) service_pool.cc:224] Handling call
0709 13:00:21.358952 (+ 19079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:21.359485 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:21.359516 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:21.360146 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 13:00:21.493047 (+132901us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:21.493236 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.493339 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.572561 (+ 79222us) raft_consensus.cc:1685] finished
0709 13:00:21.572581 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.572736 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1370,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":124926,"threads_started":1,"wal-append.queue_time_us":209459}
I20260709 13:00:21.594041 21565 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:21.598471 21565 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:21.601859 21750 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:00:21.603789 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 646) took 4354 ms. Trace:
I20260709 13:00:21.604065 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:21.605067 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 59590.8us
I20260709 13:00:21.605283 21686 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.605808 21686 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.606432 21686 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.604261 20851 rpcz_store.cc:276] 0709 13:00:17.249326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:21.299902 (+4050576us) service_pool.cc:224] Handling call
0709 13:00:21.300552 (+   650us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:21.301377 (+   825us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:00:21.301413 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:21.302142 (+   729us) log.cc:844] Serialized 26 byte log entry
0709 13:00:21.353059 (+ 50917us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:21.355488 (+  2429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:21.357865 (+  2377us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:21.603468 (+245603us) raft_consensus.cc:1685] finished
0709 13:00:21.603494 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:21.603714 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3405,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":168,"thread_start_us":34743,"threads_started":1,"wal-append.queue_time_us":189998}
W20260709 13:00:21.583015 21747 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:00:21.617317 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.618944 21723 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:21.621243 21684 raft_consensus.cc:2764] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:00:21.623876 21723 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:21.624583 21723 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:21.626914 21723 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.606033 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.630906 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.618782 19840 catalog_manager.cc:5697] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:21.611178 21712 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.648926 19840 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:21.650853 19840 catalog_manager.cc:5697] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:21.652038 21716 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:21.653082 21716 raft_consensus.cc:3055] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:21.653446 21716 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:21.654414 21716 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:21.662484 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.645879 21712 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:21.626936 21732 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:21.676870 20857 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:21.648242 21714 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:21.689456 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 83580.4us
I20260709 13:00:21.685189 19840 catalog_manager.cc:5697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 2, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 2 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:21.674746 21712 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.688752 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:21.702378 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.685010 21716 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:21.670466 21690 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:21.707988 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:21.712520 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.713387 21732 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:21.702150 21565 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.716796 21565 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:21.721009 21565 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.721221 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:21.722410 20011 raft_consensus.cc:606] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:21.729885 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 50691.5us
I20260709 13:00:21.731195 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:21.733767 21714 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:21.734268 21714 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.734680 21714 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:21.732766 21732 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:21.736519 21732 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:21.737174 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.736337 19840 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 4, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 4 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:21.737174 21732 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:21.751068 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:21.749562 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 116678us
I20260709 13:00:21.711169 21723 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:21.755487 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44328.4us
I20260709 13:00:21.756408 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:21.758412 20011 raft_consensus.cc:606] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:21.758772 20011 raft_consensus.cc:614] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:21.760131 20014 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:21.764489 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.760674 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.759615 21721 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.749727 21686 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.785609 20857 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:21.791805 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.793809 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.780009 21721 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.784355 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.787822 21338 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:21.808602 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:21.809077 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 178283us
I20260709 13:00:21.816479 20014 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:21.824146 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.824921 20854 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:21.824568 21601 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.826414 21565 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.828372 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.833550 21686 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:21.834213 21713 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.824654 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:21.779587 21643 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:00:21.839291 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.839537 21643 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:21.839993 21644 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.840195 21688 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.840415 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.840642 21644 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.840814 21688 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.841811 20014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:21.842455 20014 raft_consensus.cc:606] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:21.842841 20014 raft_consensus.cc:614] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:21.843091 21644 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.843194 21688 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.828415 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:21.840019 21643 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.844585 20850 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:21.844702 21643 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.844341 21714 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.845968 20012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:21.846598 20012 raft_consensus.cc:606] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:21.846745 21714 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.847031 20012 raft_consensus.cc:614] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:21.847502 21714 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:21.836970 21565 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.822600 20852 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:21.835716 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.848098 21644 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:21.860397 21644 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.860769 21644 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:21.858424 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:21.834718 21686 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.828765 19839 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32772 (request call id 149) took 7974 ms. Trace:
I20260709 13:00:21.857160 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.836261 21761 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:21.836437 21732 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.811658 21338 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.857990 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.804171 21721 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:21.874145 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 164743us
I20260709 13:00:21.877926 20015 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:21.874722 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.850668 21565 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:21.859319 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.881049 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:21.883142 20014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:21.884366 20014 raft_consensus.cc:606] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:21.884843 20014 raft_consensus.cc:614] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:21.887104 21690 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:21.888012 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.874361 21601 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:21.888598 21732 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:21.847893 21714 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:21.890156 20011 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:21.892050 20083 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:21.893038 21713 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:21.894600 20129 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.65:38607 timed out after 1.501s (SENT) [suppressed 27 similar messages]
I20260709 13:00:21.859797 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.875733 21761 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:21.897251 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.898540 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:21.898047 21643 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.880071 21338 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.899782 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:21.901718 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 58976.7us
I20260709 13:00:21.908823 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:21.858654 21688 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:00:21.913597 20853 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.876343 21686 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:21.904115 21714 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:21.894600 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:21.886032 21565 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:21.919265 20858 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:21.924379 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.926545 20158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:21.927248 20158 raft_consensus.cc:606] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:21.927647 20158 raft_consensus.cc:614] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:00:21.928088 20129 leader_election.cc:336] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.793s (SENT)
W20260709 13:00:21.928221 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.933589 20009 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:21.911645 21761 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:21.934463 20055 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:21.867516 19839 rpcz_store.cc:276] 0709 13:00:13.853704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.854002 (+   298us) service_pool.cc:224] Handling call
0709 13:00:21.828378 (+7974376us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:17.125699 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:17.126356 (+   657us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:17.126405 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:00:17.126438 (+    33us) tablet.cc:662] Decoding operations
0709 13:00:17.265079 (+138641us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:17.265238 (+   159us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:17.265284 (+    46us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:00:17.266983 (+  1699us) tablet.cc:701] Row locks acquired
0709 13:00:17.267047 (+    64us) write_op.cc:260] PREPARE: finished
0709 13:00:17.267307 (+   260us) write_op.cc:270] Start()
0709 13:00:17.267495 (+   188us) write_op.cc:276] Timestamp: P: 1783602017267215 usec, L: 0
0709 13:00:17.267539 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:00:17.797425 (+529886us) log.cc:844] Serialized 6309 byte log entry
0709 13:00:20.670519 (+2873094us) op_driver.cc:464] REPLICATION: finished
0709 13:00:20.902264 (+231745us) write_op.cc:301] APPLY: starting
0709 13:00:20.902388 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:20.902840 (+   452us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:20.903039 (+   199us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:20.905566 (+  2527us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:20.906092 (+   526us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:00:20.906365 (+   273us) write_op.cc:312] APPLY: finished
0709 13:00:20.906779 (+   414us) log.cc:844] Serialized 112 byte log entry
0709 13:00:20.913699 (+  6920us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:20.913932 (+   233us) write_op.cc:454] Released schema lock
0709 13:00:20.914032 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:20.914431 (+   399us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:20.914462 (+    31us) write_op.cc:454] Released schema lock
0709 13:00:21.019042 (+104580us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54008b1fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7af32a 00007fa02e734f14 00007fa02e734ec3 00007fa02e734e38 00007fa02e734df3 00007fa02e7340dc 00007fa02e75de16 00007fa02e745cbb 00007fa02e73039d 00007fa02e733d75
Metrics: {"mutex_wait_us":95,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":189313,"apply.run_cpu_time_us":5541,"apply.run_wall_time_us":12150,"num_ops":16,"prepare.queue_time_us":474257,"prepare.run_cpu_time_us":91315,"prepare.run_wall_time_us":3415439,"raft.queue_time_us":55214,"raft.run_cpu_time_us":52873,"raft.run_wall_time_us":580403,"replication_time_us":3377731,"spinlock_wait_cycles":274962176,"thread_start_us":6632729,"threads_started":4,"wal-append.queue_time_us":972299,"wal-append.run_cpu_time_us":40413,"wal-append.run_wall_time_us":3151230}]]}
W20260709 13:00:21.943506 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:21.944312 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:00:21.946993 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:21.947989 21690 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:21.948643 21690 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:21.948976 21690 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.949780 20015 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:21.956073 21712 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:21.956881 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.958616 21712 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:21.959364 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.958660 21690 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:21.962884 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:21.963483 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:21.964167 20858 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:21.965143 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.963704 20089 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:21.922829 21565 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:21.973099 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.973690 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:21.973857 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:21.974077 20083 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:21.974331 20091 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
W20260709 13:00:21.975526 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47491.7us
I20260709 13:00:21.976943 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.977591 20014 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:21.990586 20858 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:21.956130 21643 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:22.000519 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.002976 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.004451 20083 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:22.007645 21690 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:22.009148 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.009778 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.010090 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:22.010109 20157 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:22.010703 20164 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:22.010967 20156 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:22.011344 20156 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:22.012348 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.012929 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.014022 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.014886 20159 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:22.012319 20156 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:22.019742 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:21.991999 21644 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:21.996783 21714 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.024438 20082 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:22.024801 20055 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.025190 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.026294 21713 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:22.024817 21644 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:22.026875 21713 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:22.028204 21714 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:22.030371 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.030776 21713 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:22.034070 21758 log.cc:927] Time spent T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:00:22.035569 20156 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:00:22.030452 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.037222 20856 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.041558 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.042702 20009 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:22.044651 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.045784 20009 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:22.048456 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.050050 20015 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
W20260709 13:00:22.051412 20129 connection.cc:256] RPC call timeout handler was delayed by -1.0393069589999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33730000000000004s for a response
I20260709 13:00:22.069634 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:22.069949 20129 connection.cc:256] RPC call timeout handler was delayed by -1.0386901140000191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33730000000000004s for a response
I20260709 13:00:22.070158 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.080001 20088 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:22.081125 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.053380 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.053556 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.082942 20091 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:22.053982 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.054148 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.054684 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.084025 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.084138 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.084578 20089 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:22.085376 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.085562 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.086054 20088 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:22.086338 20091 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:22.086506 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.087414 20091 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:22.067066 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.084619 20009 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:22.052105 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.070673 20858 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:22.072821 20085 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:22.072996 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.090230 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.090766 20859 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:22.090975 20011 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:22.075814 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.091447 20085 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:22.076643 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.092657 20090 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:22.077919 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.092849 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.094349 20852 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:22.095427 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.077526 21565 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:22.078752 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.079244 21686 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:22.081667 21721 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.082954 20083 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:22.041183 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.075620 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.085500 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.105049 20014 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:22.105360 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.105075 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.106812 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.107487 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.107897 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.108943 20858 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:22.108217 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.109793 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.097962 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.110494 20086 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:22.104355 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.080925 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.113579 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.115944 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.119370 20857 raft_consensus.cc:2368] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:22.123972 20856 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:22.125077 20853 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:22.125479 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.123577 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.133505 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.134194 20089 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:22.135949 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.135994 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.136807 20090 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:22.133473 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52762.9us
I20260709 13:00:22.135949 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.096491 21338 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.140619 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.140671 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.142014 20011 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:22.140365 21690 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:22.147820 20853 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:22.148350 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.149149 20856 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 2: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:22.143133 20010 raft_consensus.cc:2410] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:22.139484 21688 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:22.151634 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.151859 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.152165 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.146726 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.156778 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.145342 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.161568 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.164932 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.145617 20850 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:22.151000 21690 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:22.174283 20853 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:22.164919 21777 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:22.185672 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.164464 21688 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:22.178890 21770 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:00:22.193740 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:22.197908 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 168206us
W20260709 13:00:22.203904 20743 leader_election.cc:336] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.516s (SENT)
I20260709 13:00:22.204739 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:22.205675 21213 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.212560 21688 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:22.187083 20156 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:22.219501 20161 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:22.220979 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.221736 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.222611 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.223658 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:22.224146 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.224834 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.224957 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.225324 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.225697 20159 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:22.172729 20083 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
W20260709 13:00:22.233721 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 74337.5us
I20260709 13:00:22.222610 20164 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:22.222067 20161 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:22.234051 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.235507 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.236409 20014 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:22.237881 20090 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
W20260709 13:00:22.240151 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.226184 20157 raft_consensus.cc:2368] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:22.246254 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.262710 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:22.265097 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 117486us
I20260709 13:00:22.242031 21723 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:22.276117 20084 raft_consensus.cc:2410] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:22.277276 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.279409 21338 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:22.280534 21565 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:22.268512 21684 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:22.296267 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:22.301559 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 120939us
W20260709 13:00:22.309665 21772 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:00:22.310961 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.291040 21721 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:22.309713 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.318849 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.325846 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:22.326637 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:22.304919 21684 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:22.357522 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:22.358951 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:22.360911 21338 raft_consensus.cc:993] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:22.314183 21777 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:22.363538 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63077.6us
I20260709 13:00:22.302054 21716 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:00:22.365298 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.289633 21690 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.366593 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:22.368801 20158 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.372921 20162 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:22.377282 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59051.3us
I20260709 13:00:22.377450 21777 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:22.378244 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:22.379031 21601 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.366071 21719 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:22.380235 21719 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:22.369294 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:22.381461 21780 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:00:22.396391 21791 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:22.328784 21761 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:22.397446 21790 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:22.392222 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:22.382484 21686 raft_consensus.cc:1081] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:22.387844 21644 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:22.409612 20089 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:22.381248 21719 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.408141 21213 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:22.385386 21783 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:00:22.416692 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:22.418810 20857 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:22.432410 20162 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:22.410913 20157 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:22.442179 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:22.442979 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80179.1us
W20260709 13:00:22.445407 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58065.8us
I20260709 13:00:22.446075 21761 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:22.447304 20744 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.447988 21644 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:22.447992 21765 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:22.452389 21785 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:00:22.415194 21213 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:22.455220 20158 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:22.457357 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.456333 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.448045 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:22.457962 21696 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.482451 21696 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:22.482957 21696 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.487080 21213 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.497229 20164 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
W20260709 13:00:22.501874 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 58302.2us
I20260709 13:00:22.503016 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:22.411149 21684 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.496165 20744 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.478866 21565 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:22.513207 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76423.3us
I20260709 13:00:22.545456 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:22.524034 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:22.536936 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.537298 21696 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.541661 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:22.544085 21783 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:00:22.549391 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 132484us
W20260709 13:00:22.552181 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 49275.9us
I20260709 13:00:22.554370 21794 raft_consensus.cc:2764] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:22.559605 21686 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.560050 21686 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.560426 21686 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.508874 21765 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.557308 19981 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:22.557514 21338 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:00:22.588394 21721 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:22.588905 21721 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:22.589970 21696 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:22.591044 21696 raft_consensus.cc:2764] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:22.590261 21721 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:22.592967 21721 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.577730 21791 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:22.599539 21338 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.600086 21338 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.600499 21338 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:22.600914 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.568506 21716 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.597376 20130 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:22.602123 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:22.602337 21716 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.614792 21338 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:22.619609 21338 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:22.621697 20850 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:22.626161 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:22.624408 21716 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:22.625428 21765 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:22.627517 21760 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:22.628433 21765 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:22.653311 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69002.4us
W20260709 13:00:22.663885 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 264830us
W20260709 13:00:22.663748 20743 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.713s (SENT)
I20260709 13:00:22.628459 21338 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.653128 21791 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:22.675172 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:22.697324 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 238005us
I20260709 13:00:22.678184 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:22.678437 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52357.1us
I20260709 13:00:22.666604 21338 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.664680 21787 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:22.700773 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 137928us
I20260709 13:00:22.702836 19982 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:22.706725 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.708802 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.708062 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:22.712689 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 165623us
I20260709 13:00:22.705184 21338 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.730476 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.736545 20858 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:22.731709 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.737795 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.741161 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.704679 21787 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:22.743664 20089 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:00:22.724087 21716 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:22.758091 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.759101 20089 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:22.760481 20089 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:22.752867 21810 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:00:22.762334 21688 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:22.783844 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:22.752341 21787 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:22.781366 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.786229 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:22.788100 20088 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.778868 21721 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:22.759898 21716 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:22.790027 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52341us
I20260709 13:00:22.794368 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.791049 20130 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:22.795543 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 117442us
I20260709 13:00:22.765601 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.766341 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:22.802608 21808 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:00:22.779604 21811 raft_consensus.cc:2764] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:22.796043 21716 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:22.800243 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:22.825429 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.821121 21802 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:22.835665 21802 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:22.835845 20854 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:22.836302 21802 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.836585 21565 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:22.837399 20083 raft_consensus.cc:2376] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3. Re-sending same reply.
W20260709 13:00:22.838584 21804 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:00:22.839413 21760 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:00:22.821221 21812 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:22.840883 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.832075 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.847461 20089 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.850384 20088 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:22.817037 21788 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:22.810411 20082 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.858011 20086 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 5. Ops: []
I20260709 13:00:22.861049 20089 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:22.822244 21803 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:00:22.866647 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.819222 21721 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:22.870729 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 79732.7us
I20260709 13:00:22.822618 21809 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:22.858371 21684 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:00:22.882561 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.873204 21783 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:22.829416 21686 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:22.862448 21788 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:22.891191 21819 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.894624 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.840379 21213 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:00:22.877405 21812 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:22.876427 21809 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.882992 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:22.839891 21801 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 13:00:22.798619 21765 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:22.905470 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:22.895576 21808 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:22.903191 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:22.905265 21809 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.913165 21719 raft_consensus.cc:2764] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:00:22.919225 20851 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:22.912981 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:22.906641 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 143317us
I20260709 13:00:22.901974 21788 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.922653 21716 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:22.920632 21338 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:22.910070 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:22.820415 21791 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:22.896847 21783 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:22.912621 21819 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:22.932195 20010 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:22.932500 21820 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:22.952065 21820 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.952605 21820 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:22.954025 21686 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:22.956928 21802 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:22.960108 21812 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:22.933857 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.961782 20014 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:22.907812 21721 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:22.924358 21808 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:22.941982 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:22.944783 21783 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:22.944783 21796 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:22.949662 21688 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:22.969027 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.971351 19982 leader_election.cc:400] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:22.978427 20857 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:00:22.982314 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.974939 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.983973 21816 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:22.984808 21796 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:22.988114 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 46208.6us
I20260709 13:00:23.019779 20010 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:22.990391 21719 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:23.021919 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:22.991017 21787 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:22.992218 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.992834 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.997812 21716 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:23.017584 19982 leader_election.cc:403] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:22.940596 20854 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.026140 19981 leader_election.cc:400] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:23.026916 20011 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:23.029958 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:22.963582 21801 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:23.038408 21338 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:23.040175 21723 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:23.040805 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:23.026752 21719 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:23.043103 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 115214us
I20260709 13:00:23.058175 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.043570 21796 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:22.969457 21809 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:23.032073 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.030992 20014 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
W20260709 13:00:23.052484 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.974840 21802 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:23.060070 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:23.064368 21686 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.027817 20082 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:23.074085 21808 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:23.076912 21787 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:23.078217 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.081012 20857 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:23.044982 21801 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.056849 21716 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:23.075671 21808 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.060854 21719 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.097683 19982 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.069506 21790 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:23.123898 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.127413 21765 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:23.078754 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:23.112224 21801 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:23.112402 21819 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:23.143690 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:23.141872 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 119847us
I20260709 13:00:23.082072 21820 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:23.118445 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:23.146276 20744 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:23.147722 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:23.126515 21790 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:23.113967 21565 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 13:00:23.157102 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 134710us
W20260709 13:00:23.168265 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 44473.1us
I20260709 13:00:23.171844 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.174116 20086 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:23.175047 21820 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:23.176132 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:23.176607 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:23.177141 20088 raft_consensus.cc:2410] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:23.177508 20089 raft_consensus.cc:2368] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:23.178330 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.161993 21790 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.178860 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.180546 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.176182 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.149500 21819 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:23.183275 20160 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:23.150517 21765 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.176180 21760 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.184491 21760 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:23.147856 21813 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:00:23.177793 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.186477 20156 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:23.186975 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.187433 21760 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:23.178934 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.188103 21820 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:23.203462 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:23.189803 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.188623 21796 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.190753 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.207325 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:23.208629 20089 raft_consensus.cc:2368] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5.
I20260709 13:00:23.170351 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.202574 21828 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.207562 20086 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:23.218205 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.219529 20084 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:23.211001 20085 raft_consensus.cc:2376] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4. Re-sending same reply.
I20260709 13:00:23.223331 21833 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:23.224229 21833 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:23.224552 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.225806 20091 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:23.226843 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.205943 21820 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:23.229043 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 58740.2us
I20260709 13:00:23.229336 21820 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.230486 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:23.165318 20157 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:23.212386 21819 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:23.202594 21783 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:23.242795 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:23.238896 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 92765.3us
I20260709 13:00:23.243961 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.186523 21813 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:23.217356 21784 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.243103 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.237754 21809 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:23.250037 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.247746 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:23.251816 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.250234 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:23.249053 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 103224us
I20260709 13:00:23.250720 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.252707 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:23.253410 20091 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
W20260709 13:00:23.245002 21776 log.cc:927] Time spent T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:00:23.224596 21833 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:23.247001 21338 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.269944 21822 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.272105 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:23.273031 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:23.277213 21817 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.357s	user 0.002s	sys 0.004s
I20260709 13:00:23.278311 21783 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:23.282248 21801 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:23.265401 21787 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:23.249006 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.246873 21813 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:23.265131 21812 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:23.283805 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.286499 20858 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:23.287303 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.288574 20086 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:23.289974 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:23.296339 21824 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:00:23.297091 21788 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:23.298233 21721 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:23.299521 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 351952us
I20260709 13:00:23.301594 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.290078 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:23.300872 21716 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:23.308663 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:23.305740 21820 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:23.282358 20854 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
W20260709 13:00:23.317183 21821 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.405s	user 0.003s	sys 0.000s
I20260709 13:00:23.317970 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.320122 21833 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:23.320544 21213 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:23.321996 21760 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.324054 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.359719 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.338835 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.324970 21723 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:23.340011 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 68007.3us
I20260709 13:00:23.360404 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.360744 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:23.360877 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.361449 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.361811 20014 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:23.364001 20017 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:23.407999 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46641.4us
I20260709 13:00:23.366886 20856 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 5. Ops: []
W20260709 13:00:23.366017 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106179us
I20260709 13:00:23.372797 20162 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.409308 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.410003 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.410899 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.241372 21828 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:23.400174 20161 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:23.401432 20010 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:23.363320 20011 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:23.337592 21787 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.948850 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:23.351214 21813 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.419770 21338 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:23.421243 21338 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:23.421806 20012 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:23.422271 21338 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:23.424758 20162 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:23.404773 21760 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:23.431025 21787 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:23.432704 21790 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:23.364866 20156 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:23.364905 21820 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:23.431002 21721 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.441349 21788 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:23.363358 21801 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:23.444607 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.445158 20017 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:00:23.445397 20009 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:23.445636 20013 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:23.446533 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 180683us
I20260709 13:00:23.447570 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.447959 21688 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.448460 21565 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:23.449147 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.450518 21808 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:23.450695 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.359103 21812 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:23.452705 21812 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:23.361745 21719 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:23.457283 21719 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:23.355767 21834 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:23.461339 21833 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:23.462728 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:23.462790 21833 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:23.451682 21644 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:23.469063 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:23.470923 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:23.485327 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 232713us
W20260709 13:00:23.486910 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 76077.4us
W20260709 13:00:23.405781 21776 log.cc:927] Time spent T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:00:23.463143 20013 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:23.501523 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.501816 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.502758 20013 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.504349 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.419742 21716 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:23.401557 20159 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:23.512413 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.522233 20017 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:23.523468 20011 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:23.464040 21723 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:23.516021 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:23.517176 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 48203.1us
I20260709 13:00:23.524991 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.526557 20010 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:23.489447 21686 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.431002 21760 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:23.533210 20008 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:23.535564 20008 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:23.536826 20008 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:23.537815 20008 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.537670 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:23.509042 21801 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:23.511294 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.544173 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:23.556387 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 85538.2us
I20260709 13:00:23.565282 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.567073 20856 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:23.508055 21213 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:23.568538 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57331.6us
I20260709 13:00:23.576969 20163 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:23.520596 21565 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:23.580565 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.488559 21688 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:00:23.579818 20010 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:23.585564 20161 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:23.587069 20013 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:23.587190 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.597514 20017 raft_consensus.cc:2368] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:23.585598 20015 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
W20260709 13:00:23.582465 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 178844us
I20260709 13:00:23.527422 21832 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:23.603494 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:23.616955 20009 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:23.599651 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:23.542204 21822 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.576440 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:23.578570 21760 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:23.586382 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:23.534327 21719 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:00:23.634536 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:23.635288 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.593327 21686 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:23.555841 21684 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:00:23.637215 19840 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 157) took 3314 ms. Trace:
I20260709 13:00:23.650203 20015 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
W20260709 13:00:23.642719 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 63352.6us
I20260709 13:00:23.652516 20008 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:23.618899 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.665463 20743 leader_election.cc:400] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5.
I20260709 13:00:23.625097 21716 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.664919 20854 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:23.663314 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:23.664960 20011 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:00:23.725855 20014 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:23.645488 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:23.717128 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 145662us
W20260709 13:00:23.708801 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 166524us
I20260709 13:00:23.647485 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:23.665761 20156 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:23.663337 20008 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:23.614578 21796 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 13:00:23.688285 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 128185us
W20260709 13:00:23.721216 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47263.4us
W20260709 13:00:23.702441 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.653363 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:23.734009 20010 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:23.723126 20013 pending_rounds.cc:85] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:23.735072 20013 pending_rounds.cc:107] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:23.615181 21832 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:23.735458 20013 raft_consensus.cc:2889] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:23.727393 20014 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:23.721727 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.732506 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.735399 19982 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:23.640741 19840 rpcz_store.cc:276] 0709 13:00:20.304676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.353629 (+ 48953us) service_pool.cc:224] Handling call
0709 13:00:20.573665 (+220036us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54008b1fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 13:00:23.619201 (+3045536us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:21.813322 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:21.814200 (+   878us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:21.814286 (+    86us) write_op.cc:435] Acquired schema lock
0709 13:00:21.814325 (+    39us) tablet.cc:662] Decoding operations
0709 13:00:21.815536 (+  1211us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:21.815627 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:21.815671 (+    44us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:00:21.900493 (+ 84822us) tablet.cc:701] Row locks acquired
0709 13:00:21.900550 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:00:21.900723 (+   173us) write_op.cc:270] Start()
0709 13:00:21.900879 (+   156us) write_op.cc:276] Timestamp: P: 1783602021900658 usec, L: 0
0709 13:00:21.908884 (+  8005us) op_driver.cc:348] REPLICATION: starting
0709 13:00:22.028533 (+119649us) log.cc:844] Serialized 4003 byte log entry
0709 13:00:22.530879 (+502346us) op_driver.cc:464] REPLICATION: finished
0709 13:00:22.762951 (+232072us) write_op.cc:301] APPLY: starting
0709 13:00:22.781939 (+ 18988us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:22.823360 (+ 41421us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:22.823569 (+   209us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:22.825569 (+  2000us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:22.826095 (+   526us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:00:22.826288 (+   193us) write_op.cc:312] APPLY: finished
0709 13:00:22.826739 (+   451us) log.cc:844] Serialized 76 byte log entry
0709 13:00:22.826947 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:22.827092 (+   145us) write_op.cc:454] Released schema lock
0709 13:00:22.827168 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:22.827729 (+   561us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:22.832147 (+  4418us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":130671,"reported_tablets":26,"spinlock_wait_cycles":408838016,"child_traces":[["op",{"apply.queue_time_us":191679,"apply.run_cpu_time_us":5169,"apply.run_wall_time_us":87082,"num_ops":10,"prepare.queue_time_us":69044,"prepare.run_cpu_time_us":21585,"prepare.run_wall_time_us":347353,"raft.queue_time_us":154657,"raft.run_cpu_time_us":15524,"raft.run_wall_time_us":170927,"replication_time_us":601922,"thread_start_us":316968,"threads_started":4,"wal-append.queue_time_us":146787}]]}
I20260709 13:00:23.745422 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.746263 20161 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:23.746884 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 88696.2us
I20260709 13:00:23.747628 20743 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:23.742182 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.641785 21822 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:23.763553 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.667158 21565 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:00:23.666659 21831 consensus_queue.cc:1059] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:00:23.728642 21765 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:23.775157 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:23.733760 19838 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 3 to 5, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 5 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:23.785157 20130 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:23.740479 21832 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:23.735502 21688 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:23.759136 21338 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:23.732385 21716 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:23.807891 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.814286 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:23.826177 20008 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
W20260709 13:00:23.821977 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 205067us
I20260709 13:00:23.788975 21760 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:23.833300 20129 leader_election.cc:400] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:23.839588 20129 leader_election.cc:403] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:23.797418 21820 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:23.837139 19981 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:23.804234 21828 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:23.790628 21852 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:23.816241 21797 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:23.817945 21338 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.822849 21684 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:23.732699 21213 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:23.798775 21777 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:00:23.855620 21823 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:23.850569 21812 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.831408 21688 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:23.868929 21719 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:23.877099 21796 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:23.878188 21822 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:23.878072 21837 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:00:23.861893 21338 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:23.875770 21823 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:23.864621 21797 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:23.885689 21842 log.cc:927] Time spent T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:00:23.888245 19838 catalog_manager.cc:5697] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:23.861478 21783 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:00:23.862429 21684 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:23.872929 21565 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:00:23.891188 21760 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:23.893481 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56395.5us
I20260709 13:00:23.955505 19981 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:23.860210 21723 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:23.909458 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 177049us
I20260709 13:00:23.957547 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.872148 21845 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:23.899088 21820 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:23.942977 21834 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:23.899628 21719 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:23.953719 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 125175us
I20260709 13:00:23.987288 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:23.902110 21856 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:23.929802 21823 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:23.983619 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:23.988776 19838 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 3 to 5, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 5 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:23.994551 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 246964us
I20260709 13:00:23.995496 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:23.993037 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:23.999892 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.983323 21851 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:24.003741 20162 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:23.979890 21820 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:24.023811 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.023870 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:23.985272 21834 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:24.030046 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:24.030992 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:24.030820 21834 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:24.031875 21834 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.032594 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:24.033854 20090 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:24.043262 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 239942us
I20260709 13:00:23.997387 21723 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:24.040971 20165 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:24.016515 21777 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:24.046005 20160 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:24.001858 21855 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:24.004267 21565 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:24.049259 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.051198 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:24.068292 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 81098.9us
I20260709 13:00:24.071002 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:24.072904 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.073926 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.021484 21857 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:24.081593 21857 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:24.081903 20859 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:24.082091 21857 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:24.083488 20852 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:24.060730 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:24.055686 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 62697.4us
I20260709 13:00:23.996411 21845 raft_consensus.cc:697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:24.049192 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.092629 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:24.097573 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 92757.9us
I20260709 13:00:24.099025 20853 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:24.100318 20130 leader_election.cc:400] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:24.027431 21851 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:24.041862 21856 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.101405 20130 leader_election.cc:403] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:24.101869 20090 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:24.110607 20158 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:24.108583 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:24.116045 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:24.117909 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:24.119571 21777 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:24.120636 21777 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:24.121178 21777 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:24.122277 21777 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.073904 21684 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.125792 21684 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:24.128623 21856 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:24.129280 21861 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:24.105458 21788 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:24.073305 21790 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:24.078456 21565 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.089089 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:24.088246 21723 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:24.078215 21855 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:24.139524 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:24.144863 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.146339 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:24.153677 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 64325.4us
I20260709 13:00:24.105360 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.110934 21716 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:24.159053 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:24.161019 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:24.161952 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:24.163086 20090 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:24.164181 20853 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:24.142356 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.151016 21565 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.156766 20082 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.078872 21719 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:24.156347 21860 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:24.175606 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:24.177448 21850 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:00:24.186712 21850 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:24.187217 21850 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:24.187832 21867 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:24.190933 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:24.161274 21831 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:24.204771 21719 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:24.205037 21831 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.205574 21831 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:24.044359 21797 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:24.209210 21851 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.184877 21860 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:24.213093 21863 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:00:24.214473 21845 consensus_queue.cc:237] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.219513 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:24.178541 21862 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:24.170895 21834 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:00:24.167694 21790 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.193231 21850 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:24.202749 21839 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.484s	user 0.004s	sys 0.000s
I20260709 13:00:24.168753 21688 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:24.232553 20160 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:00:24.252803 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 134992us
W20260709 13:00:24.253549 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62947.8us
I20260709 13:00:24.254664 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:24.255642 20853 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:24.196182 21856 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:24.252024 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 152072us
I20260709 13:00:24.148227 21820 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:24.270617 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:24.276991 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:24.267706 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:24.252048 21787 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.820s	user 0.014s	sys 0.000s
I20260709 13:00:24.289651 20011 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:24.290545 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.261003 21811 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:24.293893 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:24.297356 20013 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.295032 20015 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:24.279106 21859 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.279409 20157 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:24.281490 21850 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.313620 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:24.301757 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 670) took 1289 ms. Trace:
I20260709 13:00:24.285619 21788 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:24.286357 21696 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:24.294695 21856 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:24.293236 21721 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.301808 21820 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.302528 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:24.316555 20159 rpcz_store.cc:276] 0709 13:00:23.003422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.018977 (+ 15555us) service_pool.cc:224] Handling call
0709 13:00:24.284033 (+1265056us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b540049be54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:24.288363 (+  4330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686328064}
I20260709 13:00:24.332852 21868 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:24.333357 21830 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:24.333454 20129 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:24.371716 20165 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:24.334965 21777 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:24.325941 21856 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:24.339123 21844 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.603s	user 0.000s	sys 0.011s
I20260709 13:00:24.212932 21860 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:24.341426 21788 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:24.345541 21721 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.315073 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:24.356956 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 286072us
I20260709 13:00:24.271062 21723 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:24.289892 21787 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:24.297950 21834 consensus_queue.cc:1059] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:00:24.334450 21880 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.385089 21856 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.380013 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 542) took 1091 ms. Trace:
I20260709 13:00:24.383951 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:24.252341 21855 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:24.392371 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.400781 21338 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.403091 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:24.406056 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.409431 20013 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:24.337596 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.400851 21796 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:24.415524 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 82121.3us
I20260709 13:00:24.430868 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:24.426852 21696 raft_consensus.cc:3055] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:24.432093 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:24.432369 21796 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:24.432889 21808 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:24.433106 21813 consensus_meta.cc:287] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.997s	user 0.016s	sys 0.000s
I20260709 13:00:24.433626 21808 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:24.433388 21796 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.433533 21813 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:24.434067 21808 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:24.434459 21808 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.434485 21796 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.386664 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 543) took 1098 ms. Trace:
I20260709 13:00:24.437335 21858 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:24.435987 20851 rpcz_store.cc:276] 0709 13:00:23.280858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.299973 (+ 19115us) service_pool.cc:224] Handling call
0709 13:00:23.300538 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:23.324452 (+ 23914us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:23.324814 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 13:00:23.365566 (+ 40752us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b1800227c20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:23.725518 (+359952us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:23.819803 (+ 94285us) log.cc:844] Serialized 26 byte log entry
0709 13:00:23.832717 (+ 12914us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:23.842083 (+  9366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.855343 (+ 13260us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:24.334622 (+479279us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b1800227c20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:24.358625 (+ 24003us) raft_consensus.cc:1685] finished
0709 13:00:24.358669 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:24.379359 (+ 20690us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23323,"prepare.queue_time_us":9901,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":85876,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":15395,"spinlock_wait_cycles":14316160,"thread_start_us":345146,"threads_started":2,"wal-append.queue_time_us":394372}
I20260709 13:00:24.438750 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.439369 20851 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:24.435810 21813 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:24.431800 21338 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:24.440999 20161 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:24.441324 20163 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:24.420897 20857 rpcz_store.cc:276] 0709 13:00:23.280535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.284133 (+  3598us) service_pool.cc:224] Handling call
0709 13:00:23.303290 (+ 19157us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:23.321260 (+ 17970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:23.321301 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:23.321904 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 13:00:23.548945 (+227041us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:23.549002 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.549096 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:24.195375 (+646279us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:24.360361 (+164986us) raft_consensus.cc:1685] finished
0709 13:00:24.360383 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:24.372274 (+ 11891us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17403,"raft.queue_time_us":19674,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":8944000,"thread_start_us":226545,"threads_started":1,"wal-append.queue_time_us":322459}
W20260709 13:00:24.441533 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.412245 20850 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:24.449538 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:24.490090 20013 raft_consensus.cc:2410] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:24.489871 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:24.499807 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.500293 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:24.442095 21832 consensus_meta.cc:287] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.004s
I20260709 13:00:24.510671 20082 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:24.512163 20857 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:24.512993 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:24.514091 20008 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:24.517202 20010 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:24.521236 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:24.389323 21857 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:24.524672 20009 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:00:24.525311 20855 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:24.432502 21696 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:24.528280 20017 raft_consensus.cc:2393] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:24.374675 21868 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:24.533100 20011 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:24.534092 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:24.541126 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:24.424034 21880 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:24.625521 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.576709 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:24.534152 21760 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.621228 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:24.640252 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63652.6us
I20260709 13:00:24.602182 21881 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:24.645926 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:24.651530 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:24.652354 20858 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:24.652396 20162 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:24.641026 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 60588.5us
I20260709 13:00:24.534435 21883 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.117s
I20260709 13:00:24.537886 21644 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 13:00:24.667059 21765 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:00:24.669083 20157 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:24.645542 21862 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:24.691267 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:24.523114 21832 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:24.697674 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:24.683962 21886 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:00:24.712775 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:24.671597 20854 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:24.624729 21813 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:00:24.716673 21686 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:24.717381 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:24.713498 20009 raft_consensus.cc:2376] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8. Re-sending same reply.
I20260709 13:00:24.644431 21861 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 13:00:24.763974 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.783958 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86339us
I20260709 13:00:24.669480 21878 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:00:24.785848 20856 raft_consensus.cc:1240] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:24.715684 21822 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:24.711517 21862 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:24.653967 21760 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:24.792601 20162 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:24.718890 21787 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:24.722462 21686 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.664588 21696 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:24.813315 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 285005us
I20260709 13:00:24.796254 21845 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.856808 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:24.858402 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.872743 20129 leader_election.cc:336] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Timed out: RequestConsensusVote RPC to 127.12.30.65:38607 timed out after 1.501s (SENT)
I20260709 13:00:24.873458 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:24.876362 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:24.877363 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.879772 21854 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:24.880016 21889 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:00:24.880260 21854 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:24.880285 21696 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:24.881383 21854 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.856629 21686 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.884264 21862 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:24.888581 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:24.889598 20162 raft_consensus.cc:2410] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:24.893582 21868 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:24.901136 21890 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:00:24.902470 19982 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:24.902704 21898 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:24.903215 21898 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:24.903584 20854 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:24.904117 21898 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:24.904479 21765 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:24.811304 21822 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:24.909305 21760 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:24.911194 21716 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:24.916152 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.923605 20157 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:24.924708 21813 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:24.925508 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 49253.3us
I20260709 13:00:24.935468 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:24.878585 21644 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.930130 21897 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:24.854246 21721 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:24.874428 21777 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:24.923552 21894 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:24.920323 21868 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:24.899570 21686 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:24.921415 21565 consensus_meta.cc:287] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.724s	user 0.007s	sys 0.006s
W20260709 13:00:24.953717 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.962002 20008 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:24.934692 21857 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:24.879045 21845 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:24.985212 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 453830us
I20260709 13:00:24.992509 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:24.964866 21898 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:24.984637 21788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.013s	sys 0.006s
W20260709 13:00:24.997066 21788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.013s	sys 0.006s
W20260709 13:00:24.997875 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.998636 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.975929 21721 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:24.971546 21884 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:25.022714 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 158321us
W20260709 13:00:25.017650 21684 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.891s	user 0.014s	sys 0.003s
W20260709 13:00:25.018735 21831 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.813s	user 0.013s	sys 0.000s
I20260709 13:00:24.974748 21868 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.025368 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:24.970080 21644 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:24.967775 21897 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:24.982437 21565 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:25.026516 21888 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.372s	user 0.004s	sys 0.000s
I20260709 13:00:25.012916 21883 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:25.055478 20743 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.027s (SENT)
I20260709 13:00:25.056217 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.044667 20086 raft_consensus.cc:2410] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:00:25.052930 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 117517us
I20260709 13:00:25.025300 21884 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:25.024339 21721 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:25.069753 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67072us
I20260709 13:00:24.980787 21894 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.037140 21880 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:25.036851 21716 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:25.083967 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:25.093060 21686 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.108583 21899 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.109253 21899 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.109882 21895 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:25.017989 21822 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.110096 21874 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:25.110888 21874 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:25.111480 21899 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.047865 21851 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:00:25.113526 21851 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:25.113890 21851 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:25.114715 21851 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:25.115633 21851 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:25.117091 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:25.117770 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:25.118533 21850 consensus_meta.cc:287] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.794s	user 0.012s	sys 0.000s
W20260709 13:00:25.119048 21892 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:00:25.124333 21855 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.125492 21845 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.125622 21906 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
W20260709 13:00:25.103781 21808 consensus_meta.cc:287] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.669s	user 0.008s	sys 0.005s
I20260709 13:00:25.126816 21808 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:25.127660 21893 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:00:25.106472 21859 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.793s	user 0.014s	sys 0.000s
I20260709 13:00:25.128943 21859 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:25.128924 21884 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.130088 20162 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:25.130630 21808 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:25.132854 21850 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:25.137384 21868 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:25.098659 21856 consensus_meta.cc:287] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.656s	user 0.004s	sys 0.008s
I20260709 13:00:25.139354 21856 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.146062 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:25.088415 20858 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:25.040467 21831 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:25.057446 21790 consensus_meta.cc:287] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.818s	user 0.011s	sys 0.003s
I20260709 13:00:25.036592 21858 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 13:00:25.093060 21820 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.000s
I20260709 13:00:25.094962 21857 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:25.095374 21823 consensus_meta.cc:287] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.078s	user 0.015s	sys 0.000s
I20260709 13:00:25.048050 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.096557 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:25.098644 21796 consensus_meta.cc:287] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.663s	user 0.013s	sys 0.000s
I20260709 13:00:25.041424 21832 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.146432 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 297) took 1041 ms. Trace:
I20260709 13:00:25.147455 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:25.147181 21716 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.141711 21874 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.153208 21856 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:25.154882 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 290) took 1152 ms. Trace:
I20260709 13:00:25.158389 21644 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.158794 21864 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:25.158708 20852 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:25.162580 20013 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.163421 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.235265 20008 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:25.192288 21790 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
W20260709 13:00:25.168622 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:25.172000 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 65618.4us
I20260709 13:00:25.241657 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.241851 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.242642 20091 raft_consensus.cc:2410] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:25.084352 20129 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:25.243773 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 159478us
I20260709 13:00:25.243413 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.244760 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:25.245878 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.246685 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.247663 20090 raft_consensus.cc:2368] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:25.249073 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.249877 20090 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:25.251124 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.077064 21883 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:25.272840 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.273161 21831 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.273680 20091 raft_consensus.cc:2368] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:25.274537 20859 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.275010 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:25.275631 20158 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:25.276885 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.227906 20161 rpcz_store.cc:276] 0709 13:00:24.002797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.002911 (+   114us) service_pool.cc:224] Handling call
0709 13:00:24.003227 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:24.441410 (+438183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.154590 (+713180us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400028854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:25.154790 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006192512}
W20260709 13:00:25.164599 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 109164us
I20260709 13:00:25.219404 20165 rpcz_store.cc:276] 0709 13:00:24.105390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.127998 (+ 22608us) service_pool.cc:224] Handling call
0709 13:00:24.128387 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:24.372264 (+243877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.146192 (+773928us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54000be854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:25.146352 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555939200}
I20260709 13:00:25.216352 20853 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:25.251924 20090 raft_consensus.cc:2376] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5. Re-sending same reply.
I20260709 13:00:25.253953 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.254145 21788 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.254197 21920 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.254024 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.254884 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.253897 21820 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:25.217456 21823 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.256739 21796 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:25.257715 21880 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:00:25.260905 20129 leader_election.cc:336] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.581s (SENT)
I20260709 13:00:25.300362 20129 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.261291 21721 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.261715 21790 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:25.268033 21853 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.332s	user 0.006s	sys 0.000s
I20260709 13:00:25.233168 21864 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:25.269172 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.270042 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.277899 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 671) took 1151 ms. Trace:
I20260709 13:00:25.280314 20012 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:25.281494 21874 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:25.281039 21883 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.226986 20854 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:25.280314 20089 raft_consensus.cc:2410] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:25.329293 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.288813 21897 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:25.294438 20743 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.045s (SENT)
I20260709 13:00:25.246971 20088 raft_consensus.cc:2393] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:25.199862 21857 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.295866 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.296193 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.308339 20084 raft_consensus.cc:2368] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:25.305547 20859 rpcz_store.cc:276] 0709 13:00:24.126010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.126120 (+   110us) service_pool.cc:224] Handling call
0709 13:00:24.344936 (+218816us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:24.345163 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.274986 (+929823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.277573 (+  2587us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54000046d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:25.277818 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152263808}
I20260709 13:00:25.343284 21852 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.344480 21852 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.344707 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:25.345494 20859 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.345603 21857 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:25.334522 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 73641.2us
W20260709 13:00:25.350548 20129 leader_election.cc:336] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.585s (SENT)
I20260709 13:00:25.332818 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.351295 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:25.295852 21849 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.332747 20158 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:25.310611 21790 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.356247 20084 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:25.359809 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 666) took 1301 ms. Trace:
I20260709 13:00:25.381570 20855 rpcz_store.cc:276] 0709 13:00:24.058496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.060291 (+  1795us) service_pool.cc:224] Handling call
0709 13:00:25.359576 (+1299285us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54004204d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:25.359697 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000335232}
W20260709 13:00:25.373389 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.363632 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 977) took 1334 ms. Trace:
I20260709 13:00:25.362677 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 291) took 1359 ms. Trace:
I20260709 13:00:25.367726 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 874) took 1924 ms. Trace:
I20260709 13:00:25.331781 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.386225 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:25.387671 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.388165 20158 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:25.383754 20086 rpcz_store.cc:276] 0709 13:00:24.029467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.029608 (+   141us) service_pool.cc:224] Handling call
0709 13:00:25.363348 (+1333740us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540047d1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:25.363540 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323841920}
I20260709 13:00:25.387228 20858 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:25.384671 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:25.385457 20163 rpcz_store.cc:276] 0709 13:00:24.003120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.003829 (+   709us) service_pool.cc:224] Handling call
0709 13:00:24.004662 (+   833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:24.441697 (+437035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.362351 (+920654us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54002445d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:25.362597 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003752192}
W20260709 13:00:25.391036 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 534283us
I20260709 13:00:25.384783 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:25.392513 20129 leader_election.cc:336] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.651s (SENT)
I20260709 13:00:25.393585 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.397588 20088 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:25.336715 21904 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 13:00:25.400233 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.328706 20156 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.403264 20082 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:25.332456 20083 raft_consensus.cc:2393] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:25.399073 20084 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.337298 20850 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.408779 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.421124 20086 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:25.422448 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.423264 20086 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.387243 20016 rpcz_store.cc:276] 0709 13:00:23.442984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.488170 (+ 45186us) service_pool.cc:224] Handling call
0709 13:00:23.525663 (+ 37493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:23.566914 (+ 41251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:23.566945 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:23.572225 (+  5280us) log.cc:844] Serialized 26 byte log entry
0709 13:00:23.737539 (+165314us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:23.737575 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.737629 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:25.267232 (+1529603us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:25.367398 (+100166us) raft_consensus.cc:1685] finished
0709 13:00:25.367419 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.367645 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15798,"raft.queue_time_us":32659,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12353,"spinlock_wait_cycles":15582464,"thread_start_us":99304,"threads_started":1,"wal-append.queue_time_us":255855}
I20260709 13:00:25.347836 21862 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.369499 21788 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:25.402964 21820 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.369127 21823 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.358314 21894 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.365931 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:25.315013 21719 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:00:25.426623 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:25.461004 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 60864.6us
I20260709 13:00:25.461848 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.462195 20859 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:25.428288 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.405608 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.407049 21833 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:25.303207 21920 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.470057 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.414487 21857 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:00:25.406180 21903 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:25.475116 21924 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:25.475699 21924 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:25.475340 21903 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.476172 21924 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.477891 21903 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:25.406670 21878 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.478752 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.478609 21878 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.479164 20851 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:25.479482 20088 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.400535 21849 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.439985 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.440646 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 557) took 1002 ms. Trace:
I20260709 13:00:25.480767 20857 rpcz_store.cc:276] 0709 13:00:24.438609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.199501 (+760892us) service_pool.cc:224] Handling call
0709 13:00:25.234233 (+ 34732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.234539 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:25.284001 (+ 49462us) log.cc:844] Serialized 14 byte log entry
0709 13:00:25.440267 (+156266us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.440342 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:25.440388 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.440456 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.440584 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9102,"prepare.queue_time_us":1016,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":13336,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":143100,"threads_started":1}
I20260709 13:00:25.481818 21852 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:25.441490 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 147107us
I20260709 13:00:25.464694 21801 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:25.482601 20743 leader_election.cc:336] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.105s (SENT)
I20260709 13:00:25.483008 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.441946 21895 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:25.483397 21565 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.429404 21765 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:25.406446 21906 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:25.486510 21906 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.487872 21857 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:25.488313 21857 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:25.489038 21906 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:25.489193 21857 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:25.429718 21788 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:25.490653 21857 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.491084 21788 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.487893 20086 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
W20260709 13:00:25.494100 20743 leader_election.cc:336] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.189s (SENT)
I20260709 13:00:25.494565 21920 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:25.495078 20743 leader_election.cc:336] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.244s (SENT)
W20260709 13:00:25.448834 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.452819 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.415895 21822 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:25.419828 21845 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:25.436733 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:25.464984 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 72513.7us
W20260709 13:00:25.513245 20129 leader_election.cc:336] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.672s (SENT)
I20260709 13:00:25.513754 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.427930 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:25.473475 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46973.1us
I20260709 13:00:25.473925 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.504115 20088 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:25.469353 21832 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:25.481398 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.445302 21854 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.486423 21765 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.336645 21864 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:25.492116 21894 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:25.456693 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.458464 21790 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:25.496905 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.525883 20857 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.526257 20082 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:25.496030 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.523594 20859 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:25.526937 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.527751 20083 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
W20260709 13:00:25.528291 20129 leader_election.cc:336] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.814s (SENT)
I20260709 13:00:25.528424 21213 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:25.528755 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:25.529383 20083 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:25.531613 21864 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.512430 21906 raft_consensus.cc:2764] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:25.520435 21895 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.532944 21895 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:25.535104 21895 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.525239 21894 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:25.536237 21783 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:25.536657 20129 leader_election.cc:336] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.868s (SENT)
I20260709 13:00:25.537138 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.537940 21783 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:25.538528 20129 leader_election.cc:336] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.873s (SENT)
W20260709 13:00:25.532382 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 75248us
I20260709 13:00:25.492650 21833 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:25.540632 21833 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:25.541924 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.492602 21788 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.544055 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:25.544911 21923 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.545414 21923 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:25.546321 21874 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.546828 21874 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.547439 21923 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:25.548586 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 680) took 1420 ms. Trace:
I20260709 13:00:25.548755 21874 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.548971 20852 rpcz_store.cc:276] 0709 13:00:24.128618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.383564 (+254946us) service_pool.cc:224] Handling call
0709 13:00:25.548420 (+1164856us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54000f4bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:25.548532 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605113216}
I20260709 13:00:25.550356 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 553) took 1267 ms. Trace:
I20260709 13:00:25.550729 20853 rpcz_store.cc:276] 0709 13:00:24.282484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.283536 (+  1052us) service_pool.cc:224] Handling call
0709 13:00:25.550170 (+1266634us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b5400938354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:25.550300 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2125911808}
I20260709 13:00:25.551318 20084 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
W20260709 13:00:25.552837 20129 leader_election.cc:336] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.950s (SENT)
I20260709 13:00:25.524144 20016 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:25.525162 21831 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:25.538450 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.496169 21850 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:25.557639 20082 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:25.552137 21884 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:25.584438 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.523202 21845 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.585600 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.585412 21924 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.586750 21845 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:25.587435 21924 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:25.562588 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.575623 21923 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:25.553316 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:25.536900 21894 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:25.588335 21923 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:25.523193 21822 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.588989 21894 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.563894 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.590531 21923 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:25.590939 21822 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:25.591292 20084 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:25.524513 20088 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:00:25.592010 21783 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.592679 21783 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:25.566397 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:25.575219 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 54070.1us
I20260709 13:00:25.582238 20010 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:25.587687 20082 raft_consensus.cc:1240] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.595408 20082 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:25.596480 20082 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.596920 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.597945 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.598007 20855 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:25.598773 20086 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:25.579588 21832 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:00:25.612066 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.612921 20009 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:25.612857 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.614490 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.614774 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.615883 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:25.616616 20015 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.523270 21719 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:25.630227 20016 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:25.481521 21849 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:25.610221 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47777.7us
I20260709 13:00:25.620324 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.622784 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.646796 20743 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.647648 20853 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:25.536988 21765 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:25.600703 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.625946 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.626899 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.529392 21820 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:25.636650 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.580120 21895 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.656109 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.644130 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.644164 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:25.650362 20010 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:25.659626 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.649698 20858 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:25.658982 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.657869 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:25.672926 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.663282 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.627023 21857 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:25.627427 21903 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.675618 20859 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:25.673981 20858 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:25.679072 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:25.689826 20015 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
W20260709 13:00:25.681658 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 74898.1us
I20260709 13:00:25.684404 20082 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:25.691807 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:25.692785 19981 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.699023 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.700227 20858 raft_consensus.cc:2410] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:25.702715 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.703537 21819 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:25.677620 21823 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:00:25.706913 21716 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:25.707369 21716 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.707777 21716 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:25.707818 21874 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:25.708711 21832 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:25.709244 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:25.709391 21832 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:25.709836 21832 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:25.709933 20017 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:25.710686 21832 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.711246 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:25.702996 21851 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.002s
I20260709 13:00:25.678874 21849 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:25.697288 20159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.019s	sys 0.019s
I20260709 13:00:25.658972 21898 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:00:25.746955 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.696480 21920 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.711856 21832 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.692296 21721 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:25.701560 21845 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:25.755321 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 166191us
I20260709 13:00:25.680328 21861 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
W20260709 13:00:25.758715 20129 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.004s (SENT)
I20260709 13:00:25.760572 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:25.739053 21849 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:25.742688 20159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.022s	sys 0.022s
I20260709 13:00:25.697471 21903 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.774627 20084 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:25.787833 21884 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:25.788628 20129 leader_election.cc:336] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Timed out: RequestConsensusVote RPC to 127.12.30.65:38607 timed out after 2.116s (SENT)
W20260709 13:00:25.789331 20129 leader_election.cc:336] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.116s (SENT)
W20260709 13:00:25.787490 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 189746us
I20260709 13:00:25.789803 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:25.736402 21765 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:00:25.791462 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:25.794690 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.797403 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:25.799162 20164 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:25.799324 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:25.800938 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 154207us
I20260709 13:00:25.774689 21849 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.809154 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.775089 21845 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:25.813110 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:25.791986 21856 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.814967 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.758778 21920 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:25.838696 21856 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.822032 21721 consensus_queue.cc:1059] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:00:25.808557 21943 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:25.712618 21811 consensus_queue.cc:1175] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "48da76e971b84219a6ba80d1ba83b7eb" responder_term: 5 status { last_received { term: 5 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:00:25.848548 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.815176 21894 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:25.814730 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:25.881536 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 66878.1us
I20260709 13:00:25.881924 20008 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:25.882511 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:25.851270 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 61972.2us
W20260709 13:00:25.853582 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:25.883949 20129 leader_election.cc:336] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.117s (SENT)
I20260709 13:00:25.884361 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:25.868552 21935 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.119s	user 0.002s	sys 0.000s
I20260709 13:00:25.835613 21845 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.854664 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:25.874469 21885 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 1.226s	user 0.005s	sys 0.000s
W20260709 13:00:25.858461 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.860754 20013 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:25.862010 20015 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:25.815690 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:25.864353 20858 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:25.872540 20743 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:25.869798 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:25.814455 21862 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:25.860734 21796 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:25.885991 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:25.886580 21696 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:25.887260 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:25.858809 21790 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:00:25.888463 21894 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:25.893654 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:25.873382 21787 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:00:25.857389 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.867369 21820 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.867130 21918 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:25.875061 21943 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:25.905556 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:25.887456 21721 raft_consensus.cc:3055] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:25.936689 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.916198 20013 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:25.905586 21696 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.929870 21918 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:25.928732 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.902385 21861 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:25.937927 21696 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:25.930889 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:25.938089 21918 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:25.931356 21874 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:25.977059 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107340us
I20260709 13:00:25.978271 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:25.936465 21721 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:25.935452 21943 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.929561 21894 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.941211 21944 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:25.908229 21862 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:25.977337 20159 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:25.989388 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 173759us
I20260709 13:00:25.992991 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:25.995453 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:25.996587 20164 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:25.936158 21820 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:25.984711 21796 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:25.985479 21861 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:25.989941 21906 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.009037 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:26.013051 21856 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.004333 21944 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:26.013634 21920 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.014070 21953 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.014456 20085 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:25.991601 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:25.994891 21894 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.020521 20161 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:26.042527 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:26.113090 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 70640.1us
I20260709 13:00:26.122565 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:26.058207 21788 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.003s
I20260709 13:00:26.125258 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:26.128712 20853 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:26.129276 20853 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:26.133834 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:26.135298 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 251374us
I20260709 13:00:26.055701 21765 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.059084 21953 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.130224 20853 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:26.138073 20129 leader_election.cc:336] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.119s (SENT)
I20260709 13:00:26.138728 20853 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:26.128633 21939 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:00:26.109527 21874 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:26.112187 21903 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:26.149379 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.127637 21924 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:26.059303 21716 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.147121 20854 raft_consensus.cc:2410] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:26.122368 21796 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:00:26.138654 21765 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:26.045214 21906 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.138073 21955 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.089848 21721 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.172713 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.168138 20853 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:26.140007 21953 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.151170 20159 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:26.158957 21852 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:26.175370 20009 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:26.153970 21878 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.174311 20853 pending_rounds.cc:85] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 2
W20260709 13:00:26.180401 20129 leader_election.cc:336] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.214s (SENT)
I20260709 13:00:26.180850 20853 pending_rounds.cc:107] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:26.181288 20853 raft_consensus.cc:2889] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:26.148082 21861 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:26.168701 21765 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.188412 20129 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.196266 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:26.197110 20158 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.185359 21944 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.200563 21820 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:26.201413 21862 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.168532 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.173274 21955 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:26.203500 21955 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.204358 20159 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.205384 21716 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.205579 21906 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:26.206804 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:26.188007 21948 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.195s	user 0.000s	sys 0.002s
I20260709 13:00:26.208289 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.209115 20163 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:26.210085 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.211160 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.212352 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.212939 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.212680 21918 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.213796 20160 raft_consensus.cc:2410] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:26.217029 20157 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:26.155710 21874 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.203332 21849 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:26.230650 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.230973 20160 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:26.251391 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.239179 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:26.240978 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.253105 20089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 872) took 1011 ms. Trace:
I20260709 13:00:26.221616 21857 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.254376 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.254297 21857 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.256265 20160 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:26.256529 21857 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.230299 20157 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:26.257349 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:26.257757 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 679) took 2129 ms. Trace:
I20260709 13:00:26.255713 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:26.258193 20856 rpcz_store.cc:276] 0709 13:00:24.128348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.545889 (+417541us) service_pool.cc:224] Handling call
0709 13:00:24.649177 (+103288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:24.798759 (+149582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.257527 (+1458768us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400003cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:26.257695 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279671680}
W20260709 13:00:26.233222 21891 log.cc:927] Time spent T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.361s	user 0.002s	sys 0.002s
I20260709 13:00:26.232200 21696 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.259200 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:26.261425 21696 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.200662 21831 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:00:26.262786 21903 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:26.246742 20158 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:26.263549 21903 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.242875 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:26.264382 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 76089.1us
I20260709 13:00:26.212970 20163 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:26.264669 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:26.253759 20089 rpcz_store.cc:276] 0709 13:00:25.241515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.241643 (+   128us) service_pool.cc:224] Handling call
0709 13:00:26.252818 (+1011175us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b5400781c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:26.252998 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86484224}
I20260709 13:00:26.274724 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:26.275380 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 131226us
I20260709 13:00:26.276161 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:26.288622 20163 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:26.230393 20851 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:26.241461 21849 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:26.304219 20129 leader_election.cc:400] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:26.308729 20163 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:26.272024 21855 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:26.270674 21796 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.311843 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:26.275484 21906 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:26.219817 20159 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:26.317234 20016 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:26.285267 21696 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.299252 21901 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.321409 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:26.318873 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:26.267602 21723 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:26.273780 21856 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:00:26.317153 21855 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:26.341023 21855 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.321038 21906 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.342320 21906 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:26.343987 21874 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:26.332111 21820 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.332841 21765 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:26.333158 21857 consensus_queue.cc:1059] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:26.312702 21862 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:00:26.304534 21854 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:26.337091 21696 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.339172 20163 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.326594 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.284830 21858 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:00:26.349897 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:26.310735 21898 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:00:26.348699 21901 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.351759 21796 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:26.378356 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 505876us
W20260709 13:00:26.377727 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:26.384807 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 241770us
I20260709 13:00:26.300854 21961 consensus_queue.cc:1059] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:00:26.391070 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:26.375310 21823 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:00:26.398429 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.354272 21723 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:00:26.399498 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:26.400282 20082 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
W20260709 13:00:26.403520 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 53716.2us
I20260709 13:00:26.349740 21894 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:26.406184 20743 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:26.408690 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:26.401454 21898 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.389918 21901 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.410260 20088 raft_consensus.cc:3055] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:26.413975 20088 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:26.410627 20082 raft_consensus.cc:2393] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:26.389756 21964 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:26.399020 21823 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:26.419901 21953 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.412917 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 673) took 2285 ms. Trace:
I20260709 13:00:26.420855 21696 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:26.421061 21796 raft_consensus.cc:3055] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:26.411329 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:26.403102 21832 consensus_meta.cc:287] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.002s
W20260709 13:00:26.446296 21952 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.444s	user 0.002s	sys 0.000s
I20260709 13:00:26.425380 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.448380 20858 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:26.360791 21943 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.415217 20088 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.452266 20088 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.431044 21823 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.431844 21788 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:26.433121 21862 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:26.435302 21765 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.430220 21796 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:26.461591 21918 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:26.462626 21796 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.463812 21796 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:26.439205 21213 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:00:26.427908 20850 rpcz_store.cc:276] 0709 13:00:24.126565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.531247 (+404682us) service_pool.cc:224] Handling call
0709 13:00:24.658596 (+127349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.434798 (+776202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.403235 (+968437us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b54001900d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:26.411435 (+  8200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542950144}
I20260709 13:00:26.435564 21783 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:26.448851 20858 pending_rounds.cc:85] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 2
I20260709 13:00:26.474972 20858 pending_rounds.cc:107] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:26.475237 20858 raft_consensus.cc:2889] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:26.404129 21962 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:26.448587 20090 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.407955 21944 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:26.402007 21861 raft_consensus.cc:476] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Not starting pre-election -- already a leader
I20260709 13:00:26.452924 20163 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:26.423032 21898 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.454149 20164 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:26.454530 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.412962 21854 raft_consensus.cc:697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:26.478951 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:26.458822 21943 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.463703 20088 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:26.425989 21895 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.464480 21823 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:26.467128 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 868) took 1227 ms. Trace:
I20260709 13:00:26.472488 21953 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.452544 21964 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.477665 20165 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:26.457549 21974 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.505311 21974 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.482029 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.483633 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.488823 20085 rpcz_store.cc:276] 0709 13:00:25.239891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.240183 (+   292us) service_pool.cc:224] Handling call
0709 13:00:26.466894 (+1226711us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b54005a1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:26.467036 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1645344256}
I20260709 13:00:26.504184 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.486586 21962 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:26.479167 21696 consensus_queue.cc:1059] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:00:26.501958 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 985) took 1107 ms. Trace:
I20260709 13:00:26.474742 21783 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:26.480819 21944 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:26.516776 20091 rpcz_store.cc:276] 0709 13:00:25.393910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.394349 (+   439us) service_pool.cc:224] Handling call
0709 13:00:25.414070 (+ 19721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.489958 (+ 75888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.490001 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.490057 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:25.490101 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.501079 (+1010978us) spinlock_profiling.cc:243] Waited 75.6 ms on lock 0x7b5400307354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:26.501132 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.501297 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173950208}
I20260709 13:00:26.475963 21831 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:00:26.540791 21831 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:26.468241 21906 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.541178 21831 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:26.519572 21944 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:26.542407 21900 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.654s	user 0.005s	sys 0.000s
I20260709 13:00:26.542119 21831 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.545046 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:26.517293 21964 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.517239 21962 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:26.527478 21852 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:26.527379 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.529847 21898 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:26.530499 21894 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.532248 21895 raft_consensus.cc:3055] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:26.484802 21884 consensus_queue.cc:1059] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:00:26.568500 20091 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:26.533833 21854 consensus_queue.cc:237] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.476125 21955 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.546864 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.547631 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:26.554563 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:26.556294 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:26.574126 20855 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:26.556532 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:26.517827 21721 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.516211 21832 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.573361 20853 raft_consensus.cc:2376] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5. Re-sending same reply.
I20260709 13:00:26.576980 20090 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:26.518332 21783 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.579263 20859 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:26.543687 21906 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.543648 21831 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.566295 21852 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.597890 21974 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:26.598212 21955 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:26.564982 21213 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:26.566262 21944 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:26.572537 21895 raft_consensus.cc:740] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:26.566051 21880 consensus_queue.cc:1059] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:00:26.620587 21213 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.626870 21213 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:26.620894 21696 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.630609 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.590751 21855 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:26.592052 21832 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.618367 21783 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.640276 20850 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:26.633625 21944 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:26.654929 21964 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.656487 21895 consensus_queue.cc:260] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.637392 21696 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.667014 21944 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.672633 21832 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:26.679805 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 821334us
I20260709 13:00:26.685211 21849 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:26.689935 21696 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:26.691286 20850 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:26.695364 21855 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.697356 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:26.698225 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.699134 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.699775 21820 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:26.700249 21820 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.700767 21820 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.701203 21964 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:26.683027 21901 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.704865 21901 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:26.711889 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:26.712539 20160 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:26.713260 21881 raft_consensus.cc:3055] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:26.713682 21881 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:26.714504 21881 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:26.716089 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.716503 21881 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.718607 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:26.720213 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:26.720873 20163 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.725014 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.726078 20743 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:26.727512 21943 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.729594 21880 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:26.730866 21987 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:00:26.732676 21856 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:00:26.732507 21777 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.697485 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:26.711697 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.736811 20852 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:26.737922 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:26.754472 20163 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:26.742475 21856 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:26.762311 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.741425 21852 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:26.765617 20055 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.767632 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.758487 21855 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.770555 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:26.771227 20088 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:26.773468 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.774263 21985 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.775383 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:26.775709 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:26.776098 20088 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:26.776574 20854 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:26.777137 20091 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:26.778932 20090 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:00:26.779222 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 540099us
I20260709 13:00:26.788023 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 604) took 1224 ms. Trace:
I20260709 13:00:26.789589 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:26.764514 21760 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:00:26.781023 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:26.782109 21902 log.cc:927] Time spent T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.890s	user 0.000s	sys 0.004s
W20260709 13:00:26.783833 21986 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.188s	user 0.002s	sys 0.000s
I20260709 13:00:26.809108 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:26.772092 21920 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:26.769166 21906 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:26.767030 21898 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:00:26.784205 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.772377 21903 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.784667 21982 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:26.785480 21924 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.779222 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:26.787890 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:26.822070 21820 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:26.823662 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.826354 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:26.826545 21820 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:26.827678 21820 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:26.828054 21820 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.828392 21820 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.830933 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 246) took 2337 ms. Trace:
I20260709 13:00:26.766747 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:26.832073 20156 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:26.792563 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:26.798780 20851 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:26.834196 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:26.834918 20851 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.802102 21944 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:26.803217 20160 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:26.804816 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:26.844933 20855 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.799837 21801 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.814572 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:26.809850 20858 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:26.769001 21855 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:26.817410 21852 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:26.831369 20014 rpcz_store.cc:276] 0709 13:00:24.493526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.493647 (+   121us) service_pool.cc:224] Handling call
0709 13:00:24.520722 (+ 27075us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:24.521243 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:24.521287 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:24.538658 (+ 17371us) log.cc:844] Serialized 52 byte log entry
0709 13:00:24.757729 (+219071us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:24.757784 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:24.757868 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.771841 (+2013973us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:26.830222 (+ 58381us) raft_consensus.cc:1685] finished
0709 13:00:26.830245 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.830834 (+   589us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2991,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":13404,"spinlock_wait_cycles":60441344,"thread_start_us":99384,"threads_started":1,"wal-append.queue_time_us":316219}
I20260709 13:00:26.846509 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:26.859395 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 68280.3us
I20260709 13:00:26.864408 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.865551 20010 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:26.869232 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.871232 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.876116 20015 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:26.879736 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.881891 20010 raft_consensus.cc:2410] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:26.884334 20089 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:26.886566 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 610) took 1249 ms. Trace:
I20260709 13:00:26.885212 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:26.890069 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 74531.4us
I20260709 13:00:26.890188 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.883118 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:26.858171 21849 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:26.893594 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:26.893890 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.833026 21972 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:26.847167 21923 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:26.846227 21801 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:26.858501 21721 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:26.894515 20013 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:26.906213 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59838.9us
I20260709 13:00:26.918841 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:26.855372 21855 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:26.906484 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 67230.4us
I20260709 13:00:26.894714 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:26.897310 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.921732 20008 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:26.908080 20089 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:26.911094 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:26.902812 20160 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:26.886968 20011 rpcz_store.cc:276] 0709 13:00:25.637195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.861229 (+224034us) service_pool.cc:224] Handling call
0709 13:00:25.890055 (+ 28826us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:25.890702 (+   647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.890729 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:26.216900 (+326171us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:26.217704 (+   804us) log.cc:844] Serialized 52 byte log entry
0709 13:00:26.359536 (+141832us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:26.382698 (+ 23162us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.797698 (+415000us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.886175 (+ 88477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.886271 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.886380 (+   109us) raft_consensus.cc:1685] finished
0709 13:00:26.886393 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.886514 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":491565,"prepare.queue_time_us":45380,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":422274,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":12450,"spinlock_wait_cycles":9710848,"thread_start_us":141370,"threads_started":1,"wal-append.queue_time_us":192087}
W20260709 13:00:26.926867 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.929391 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:26.930241 20010 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:26.931900 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:26.936205 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.937968 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.939502 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:26.914379 21801 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.961386 21899 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:26.877139 21878 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:00:26.956688 20156 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:00:26.941576 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:27.021307 20015 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:26.972934 20851 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:26.956594 21696 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:26.911732 21833 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.961026 21565 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:26.945739 20858 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:26.905490 21924 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
I20260709 13:00:26.960106 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 259) took 1562 ms. Trace:
I20260709 13:00:27.005163 20855 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:26.914831 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:27.008430 20008 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:27.043099 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 128388us
I20260709 13:00:26.877524 21874 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:00:26.915910 21984 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.960960 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1002) took 1233 ms. Trace:
W20260709 13:00:26.945627 21995 log.cc:927] Time spent T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:00:26.957810 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 46844.3us
I20260709 13:00:27.041509 21854 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:26.958351 21716 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.054080 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:27.024574 20156 pending_rounds.cc:85] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 2
I20260709 13:00:27.021168 21850 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.953055 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:26.888001 21845 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:00:26.943964 21943 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:26.946388 20014 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:26.943068 21849 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:27.009670 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:27.012701 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:26.979730 20129 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:26.913712 21765 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:27.000622 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:27.024868 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106357us
I20260709 13:00:27.040669 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:27.040849 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:27.048192 21903 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:27.077454 20015 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:27.057312 20156 pending_rounds.cc:107] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:00:27.046079 20009 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:27.047245 20086 rpcz_store.cc:276] 0709 13:00:25.727713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.730233 (+  2520us) service_pool.cc:224] Handling call
0709 13:00:25.746119 (+ 15886us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:25.746641 (+   522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.746684 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:25.747469 (+   785us) log.cc:844] Serialized 52 byte log entry
0709 13:00:25.906699 (+159230us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:26.960401 (+1053702us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.960659 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.960730 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.960776 (+    46us) raft_consensus.cc:1685] finished
0709 13:00:26.960788 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.960911 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1053159,"prepare.queue_time_us":1054679,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":6041,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":5032,"thread_start_us":105165,"threads_started":1,"wal-append.queue_time_us":211600}
W20260709 13:00:26.940869 21992 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:00:27.095999 20162 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:27.096593 20162 pending_rounds.cc:85] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 2
I20260709 13:00:27.096870 20162 pending_rounds.cc:107] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:27.097188 20162 raft_consensus.cc:2889] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:27.057123 19981 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.024387 21899 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:27.044754 21901 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:27.056537 21716 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.112375 20164 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:27.065923 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 86325.7us
I20260709 13:00:27.062566 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:27.118943 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:27.120021 20158 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:27.078205 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:26.978649 21783 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.085538 20851 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:27.089044 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:27.122264 20014 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:27.122650 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:27.126252 20014 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:27.126407 20011 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.071584 21895 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.084216 20156 raft_consensus.cc:2889] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:27.095057 20130 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:27.063676 21686 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.041393 21696 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.110810 21901 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.117239 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:27.072926 21974 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.076129 21796 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:27.060235 21850 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:27.080433 21903 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.083071 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:27.130453 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.118506 21716 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.065733 21822 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:00:27.096230 21963 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:27.119131 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 62093.4us
I20260709 13:00:27.084317 21765 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:27.112476 21972 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:27.120625 22002 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:27.142971 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48016.6us
I20260709 13:00:27.120126 20008 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:27.145722 20858 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:27.131680 21984 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:27.129951 21878 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.118616 21852 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
W20260709 13:00:27.198165 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50623.8us
I20260709 13:00:27.199884 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:27.200539 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:27.206934 21998 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:00:27.207615 21696 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:27.141350 21901 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.218133 20010 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:27.134932 21831 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:00:27.224292 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:27.226742 20852 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.146397 21903 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.147166 21881 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:27.200282 21880 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:27.237629 20852 raft_consensus.cc:1240] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:27.131675 21783 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.239835 20010 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.214682 21822 consensus_queue.cc:1243] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:27.250209 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:27.139036 21820 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.196780 21796 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:27.251508 21984 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:27.144529 21974 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.249604 21845 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:27.231266 21898 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:27.226867 21777 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.251156 20158 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:27.231413 21878 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:27.242614 21923 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.234025 21964 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:00:27.220189 21963 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:27.144922 21849 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:27.254801 20010 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.234202 21884 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:00:27.147977 21899 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.231963 22001 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:00:27.219225 22002 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:27.228271 21972 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:27.281652 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 151306us
I20260709 13:00:27.269794 21765 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:27.286623 22015 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:27.276758 21777 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:27.344934 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 118762us
I20260709 13:00:27.370687 20011 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
W20260709 13:00:27.346312 21998 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:27.338619 21963 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:26.788697 20017 rpcz_store.cc:276] 0709 13:00:25.565238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.742752 (+177514us) service_pool.cc:224] Handling call
0709 13:00:25.760837 (+ 18085us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:25.764871 (+  4034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.764915 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:25.786358 (+ 21443us) log.cc:844] Serialized 26 byte log entry
0709 13:00:25.897603 (+111245us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:26.372392 (+474789us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.787481 (+415089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.787577 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.787709 (+   132us) raft_consensus.cc:1685] finished
0709 13:00:26.787728 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.787937 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":438629,"prepare.queue_time_us":114767,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":421188,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":15804,"thread_start_us":501165,"threads_started":2,"wal-append.queue_time_us":175690}
I20260709 13:00:27.378777 20091 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:27.283025 21899 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.281098 20157 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.359010 22021 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.383725 20082 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.360066 21878 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.350309 21880 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:27.360961 21855 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.327877 21796 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:27.361948 21850 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:27.370203 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:27.338140 22002 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:27.378472 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:27.379269 20091 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:27.329620 21760 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:00:27.291399 21861 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:00:27.045943 20083 rpcz_store.cc:276] 0709 13:00:25.397846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.530206 (+132360us) service_pool.cc:224] Handling call
0709 13:00:25.530519 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.530732 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:25.692196 (+161464us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.959242 (+1267046us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18004618c0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 13:00:26.959474 (+   232us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.959686 (+   212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.959750 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:26.959808 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.959889 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.960038 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7580,"prepare.queue_time_us":1344589,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":91,"raft.queue_time_us":3906,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":6152,"spinlock_wait_cycles":25269632,"thread_start_us":262949,"threads_started":2,"wal-append.queue_time_us":212144}
I20260709 13:00:27.360867 21783 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:27.398142 20017 raft_consensus.cc:1275] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:27.420444 20017 pending_rounds.cc:85] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 1
W20260709 13:00:27.405794 22005 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:00:27.383392 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:27.415179 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 179389us
I20260709 13:00:27.443622 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.444427 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.444794 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:27.405987 20091 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:27.420742 20017 pending_rounds.cc:107] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:27.329468 22015 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.396768 22021 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:27.427650 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:27.428485 22004 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:00:27.429345 22008 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:00:27.432739 22012 consensus_queue.cc:1059] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:00:27.439042 22023 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:00:27.329262 21984 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.454893 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:27.455560 20856 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:27.445981 20091 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.455978 21855 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:27.448094 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.460285 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.449540 20082 raft_consensus.cc:2410] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:27.450273 21874 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.445873 20017 raft_consensus.cc:2889] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:27.462476 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:27.427657 22007 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:00:27.378428 21777 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:27.464596 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.467411 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:27.467630 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:27.473392 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:27.473204 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:27.474071 20854 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:27.451081 22001 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:27.452960 21696 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:27.485306 20011 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:27.492533 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:27.459115 20156 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:27.471778 21819 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:27.528185 21819 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:27.528710 21819 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.534668 22015 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:27.497300 21696 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.535634 21696 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.453785 21801 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.479158 21962 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:27.539090 22004 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:00:27.541445 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:27.464586 21944 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:27.501117 20091 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:27.483301 21833 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:27.471621 21811 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:27.480149 21874 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:27.509369 22019 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:27.518038 21972 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:27.518513 21989 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.894s	user 0.003s	sys 0.000s
I20260709 13:00:27.518839 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:27.524765 21958 raft_consensus.cc:3055] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.524577 21820 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:27.493929 21988 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.882s	user 0.005s	sys 0.000s
I20260709 13:00:27.494658 21850 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:27.448359 21943 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:00:27.546685 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:27.551116 20091 pending_rounds.cc:85] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 3
I20260709 13:00:27.555846 21855 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:27.479390 21880 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:27.468600 21963 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:27.471873 21881 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:27.607791 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 52602.4us
I20260709 13:00:27.606010 20017 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:27.680495 20091 pending_rounds.cc:107] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:27.539122 21801 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:27.719215 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:27.561452 21833 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:27.706374 20091 raft_consensus.cc:2889] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:27.711122 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 164509us
I20260709 13:00:27.531697 21901 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:27.728771 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 695) took 1172 ms. Trace:
I20260709 13:00:27.602847 21831 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.560803 21811 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:27.609622 21958 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:27.729380 20857 rpcz_store.cc:276] 0709 13:00:26.556529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.556680 (+   151us) service_pool.cc:224] Handling call
0709 13:00:26.576326 (+ 19646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.576875 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.576917 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.577809 (+   892us) log.cc:844] Serialized 26 byte log entry
0709 13:00:27.094888 (+517079us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b18003c1780. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:00:27.265778 (+170890us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:27.266957 (+  1179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.267056 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.706311 (+439255us) raft_consensus.cc:1685] finished
0709 13:00:27.706331 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.728672 (+ 22341us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":12914,"spinlock_wait_cycles":12580224,"thread_start_us":170681,"threads_started":1,"wal-append.queue_time_us":700262}
I20260709 13:00:27.742852 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:27.570767 21849 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.148s
I20260709 13:00:27.604214 21973 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.745277 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:27.722002 21721 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:27.614570 21982 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.117s
I20260709 13:00:27.744354 21688 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:27.737974 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 697) took 1176 ms. Trace:
I20260709 13:00:27.722402 21881 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.754812 21958 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:27.626076 22019 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.768712 21881 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:27.769855 22019 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:27.770953 21881 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:27.723276 21924 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:27.750834 21962 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:00:27.772435 21924 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:27.777259 21924 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:27.732429 21943 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:27.784977 21943 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:27.785313 21924 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.786326 21924 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.790203 21924 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:27.787245 21943 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.794162 21943 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.799360 21985 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.804242 21985 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:27.762315 21845 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.808902 21845 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.802810 20091 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:27.763904 20850 rpcz_store.cc:276] 0709 13:00:26.558510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.664378 (+105868us) service_pool.cc:224] Handling call
0709 13:00:26.688335 (+ 23957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.691736 (+  3401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.698402 (+1006666us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b5400240254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:27.734493 (+ 36091us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5857024}
I20260709 13:00:27.732826 21833 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:27.722270 21963 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.811700 21845 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:27.746927 21811 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:27.758009 21721 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.743806 22016 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:00:27.740916 21831 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:27.855571 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 708) took 1426 ms. Trace:
I20260709 13:00:27.856657 20165 rpcz_store.cc:276] 0709 13:00:26.429022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.429189 (+   167us) service_pool.cc:224] Handling call
0709 13:00:26.477446 (+ 48257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.503387 (+ 25941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.855257 (+1351870us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b5400573d50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:27.855455 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110176128}
I20260709 13:00:25.845719 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:27.839874 21721 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.877640 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:27.895449 20165 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:27.757105 21973 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:27.855527 21895 consensus_meta.cc:287] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.000s
I20260709 13:00:27.903183 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:27.861191 22016 consensus_queue.cc:1243] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:27.905851 20859 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:27.841661 21881 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.824518 21874 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:27.924340 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.924237 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:27.925468 20165 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:27.904245 21688 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:27.927518 19982 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:27.930362 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:27.915102 21686 consensus_meta.cc:287] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.003s
I20260709 13:00:27.934800 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:27.916193 21777 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:27.927012 21796 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.938177 21878 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.940429 21686 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:27.980304 20744 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.016s	sys 0.035s
W20260709 13:00:27.988504 20744 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.016s	sys 0.035s
W20260709 13:00:27.988885 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 514105us
I20260709 13:00:27.992226 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:27.997082 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:27.939656 21688 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:27.983086 21895 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.981693 21901 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:27.952554 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 617) took 2089 ms. Trace:
I20260709 13:00:27.982031 21878 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.008191 20158 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:28.021574 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:28.029527 21831 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.024545 21823 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:28.032514 22012 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:28.033311 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.040608 20014 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:28.042424 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 580) took 1245 ms. Trace:
I20260709 13:00:27.956404 21796 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.043788 21852 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:28.055032 20158 pending_rounds.cc:85] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 1
I20260709 13:00:28.062880 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:28.057073 20017 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:28.058560 20859 rpcz_store.cc:276] 0709 13:00:26.796892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.846824 (+ 49932us) service_pool.cc:224] Handling call
0709 13:00:28.042216 (+1195392us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400009bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:28.042348 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338432896}
I20260709 13:00:28.088044 20158 pending_rounds.cc:107] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:00:28.088725 20158 raft_consensus.cc:2889] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:28.055692 21962 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:28.056262 21823 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:28.072266 21852 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:28.092895 21831 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.060899 21878 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.116848 21823 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:28.132490 21852 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:28.133903 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 821) took 1040 ms. Trace:
W20260709 13:00:28.137192 22027 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 13:00:28.137770 21716 consensus_meta.cc:287] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.924s	user 0.008s	sys 0.006s
I20260709 13:00:28.150640 21973 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.108003 21963 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.107209 21962 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.180918 21962 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:28.181428 21696 consensus_meta.cc:287] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.645s	user 0.013s	sys 0.001s
I20260709 13:00:28.182188 21696 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.074162 21901 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.184448 21901 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:28.184947 21796 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.185659 21796 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.186177 21796 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.187171 21901 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:28.162032 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.188244 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.190943 21688 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.191728 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:28.192821 20082 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.206055 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.207026 20162 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.123288 22016 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:00:28.170616 21881 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.050809 21895 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:28.171325 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 235313us
W20260709 13:00:28.194607 21903 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.944s	user 0.013s	sys 0.001s
I20260709 13:00:28.211483 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:28.239050 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:28.240262 20163 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
W20260709 13:00:28.233713 21899 consensus_meta.cc:287] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.849s	user 0.009s	sys 0.008s
I20260709 13:00:28.166827 20013 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:28.212143 21852 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.246959 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.229789 21973 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.252691 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.178777 21716 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.259371 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.232524 21881 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:28.257205 20017 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:28.262269 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:28.263513 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.264129 21972 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:28.264379 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:28.266312 21688 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:28.271171 22014 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:28.262297 21565 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:28.233615 21903 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.263780 21881 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:28.282065 21895 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.048622 20012 rpcz_store.cc:276] 0709 13:00:25.862811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.045600 (+182789us) service_pool.cc:224] Handling call
0709 13:00:26.140919 (+ 95319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.359065 (+218146us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:26.410304 (+ 51239us) log.cc:844] Serialized 14 byte log entry
0709 13:00:27.033216 (+622912us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:27.136220 (+103004us) log.cc:844] Serialized 26 byte log entry
0709 13:00:27.143610 (+  7390us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:27.328990 (+185380us) log.cc:844] Serialized 14 byte log entry
0709 13:00:27.331306 (+  2316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.331410 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.493007 (+161597us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:27.938514 (+445507us) raft_consensus.cc:1685] finished
0709 13:00:27.941007 (+  2493us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.950611 (+  9604us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366337,"prepare.queue_time_us":165440,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":1409,"raft.queue_time_us":392841,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":6735,"spinlock_wait_cycles":38173952,"thread_start_us":512209,"threads_started":4,"wal-append.queue_time_us":149308}
I20260709 13:00:28.291007 21974 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.299513 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.263540 20086 raft_consensus.cc:2410] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:28.324990 21887 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.216522 21721 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:28.314314 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50009.2us
W20260709 13:00:28.316803 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 222681us
I20260709 13:00:28.334254 21796 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.329141 20082 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:28.347015 20016 raft_consensus.cc:2368] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5.
I20260709 13:00:28.312342 21973 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:28.324844 21565 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:28.365960 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:28.365689 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.364674 20744 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:28.325984 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 216258us
I20260709 13:00:28.367530 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.378734 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.256340 20162 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:28.172458 20159 rpcz_store.cc:276] 0709 13:00:27.093607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.095025 (+  1418us) service_pool.cc:224] Handling call
0709 13:00:27.143887 (+ 48862us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:27.279324 (+135437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.281312 (+  1988us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:27.364208 (+ 82896us) log.cc:844] Serialized 26 byte log entry
0709 13:00:27.377918 (+ 13710us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:27.377963 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.133423 (+755460us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540049be54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:28.133480 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.133531 (+    51us) raft_consensus.cc:1685] finished
0709 13:00:28.133548 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.133830 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81164,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":270087552,"thread_start_us":13161,"threads_started":1,"wal-append.queue_time_us":13474}
I20260709 13:00:28.348385 21856 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:28.325264 22014 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.353111 21899 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:28.352914 21982 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:28.384965 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:28.389312 20056 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 1.992s (SENT) [suppressed 46 similar messages]
I20260709 13:00:28.325578 21972 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.325358 21688 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:28.383665 21819 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.854s	user 0.010s	sys 0.007s
I20260709 13:00:28.398476 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:28.375133 21696 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:28.391907 21974 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:28.396083 22042 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:28.416571 21973 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.404119 21819 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.418831 20015 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:28.419803 21819 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.421451 22014 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.423399 21688 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.392894 21856 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:28.405722 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.422331 21878 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:28.442951 21716 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.443786 21856 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.444308 21721 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:28.447893 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.395108 21982 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.395499 21899 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:28.449359 20088 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:28.458916 20012 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:00:28.395260 21962 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.461668 21962 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:28.462242 20010 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:00:28.463819 21972 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.464088 22040 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:28.413208 22042 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:28.464843 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.464732 22040 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.465575 20159 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.465282 21823 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.466816 21881 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.467073 22040 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:28.467526 21823 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.425423 21901 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.465132 22042 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:28.468417 21901 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:28.459821 20088 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:28.492967 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.501706 20088 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:00:28.464744 21833 consensus_meta.cc:287] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.627s	user 0.004s	sys 0.009s
I20260709 13:00:28.414458 21974 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.461680 21878 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.526008 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.528321 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.503001 20083 pending_rounds.cc:85] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 1
I20260709 13:00:28.536072 20083 pending_rounds.cc:107] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:28.536448 20083 raft_consensus.cc:2889] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:28.536621 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 885) took 3275 ms. Trace:
I20260709 13:00:28.505818 22018 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:28.540360 22018 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.542438 22018 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.542733 20850 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:28.543725 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 906) took 1477 ms. Trace:
I20260709 13:00:28.529692 20084 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:28.530236 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.532351 21901 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.535092 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:28.537448 20087 rpcz_store.cc:276] 0709 13:00:25.260881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.355309 (+ 94428us) service_pool.cc:224] Handling call
0709 13:00:25.355655 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:25.512856 (+157201us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:26.543455 (+1030599us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b18002dcfa0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 13:00:26.600585 (+ 57130us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.639259 (+ 38674us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.644645 (+  5386us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.644786 (+   141us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:27.021569 (+376783us) log.cc:844] Serialized 14 byte log entry
0709 13:00:27.025210 (+  3641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.536192 (+1510982us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54000cdad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:28.536266 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.536326 (+    60us) raft_consensus.cc:1685] finished
0709 13:00:28.536342 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.536508 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":379871,"prepare.queue_time_us":646177,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":9820,"raft.queue_time_us":33844,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":14407,"spinlock_wait_cycles":404630016,"thread_start_us":74072,"threads_started":2,"wal-append.queue_time_us":124050,"wal-append.run_cpu_time_us":7012,"wal-append.run_wall_time_us":1466539}
I20260709 13:00:28.501376 21880 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:28.544329 20008 rpcz_store.cc:276] 0709 13:00:27.066642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.202924 (+136282us) service_pool.cc:224] Handling call
0709 13:00:27.234526 (+ 31602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.330405 (+ 95879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.330452 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.330512 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:27.330563 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.543525 (+1212962us) spinlock_profiling.cc:243] Waited 95.6 ms on lock 0x7b54005ebad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:28.543568 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.543667 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219834240}
W20260709 13:00:28.551265 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 161978us
I20260709 13:00:28.552830 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:28.557523 20855 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.504419 22014 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:00:28.564951 20858 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:28.565538 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.571118 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:28.582828 20855 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:28.575076 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:28.595247 21811 consensus_meta.cc:287] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.000s
I20260709 13:00:28.597914 21982 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:28.603529 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.603588 21972 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:28.605748 21811 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.608378 21811 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.611209 22021 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:28.612301 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.561477 21880 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.614775 21880 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.506651 21833 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.617458 21974 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.625854 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.627167 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.627543 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.628634 20856 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:28.561764 22044 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:28.628574 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.627962 20853 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:28.626605 20087 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:28.637213 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.641675 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.641664 20084 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:28.643206 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.643692 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.589398 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:28.638473 20855 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:28.645320 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56019.9us
I20260709 13:00:28.653201 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.649634 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.649659 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.570663 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.656796 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.657980 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.658212 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:28.664585 20089 raft_consensus.cc:2393] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:28.585930 21903 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:28.666815 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.660804 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.670733 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.671079 20859 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:28.648945 22021 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:28.668228 20088 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:28.552120 22042 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.675518 20850 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:28.677981 22021 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:28.681933 22048 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:00:28.683243 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:28.684262 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 826) took 1138 ms. Trace:
I20260709 13:00:28.684615 20157 rpcz_store.cc:276] 0709 13:00:27.545803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.550042 (+  4239us) service_pool.cc:224] Handling call
0709 13:00:28.683119 (+1133077us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:28.684185 (+  1066us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32998272}
I20260709 13:00:28.686399 21833 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:28.686651 21899 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.644070 22053 consensus_queue.cc:1059] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 13:00:28.655212 21852 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:28.688766 21852 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.689119 21852 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:28.693770 22042 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.694757 22042 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.696477 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:28.697109 20853 raft_consensus.cc:2376] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5. Re-sending same reply.
I20260709 13:00:28.697924 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.698093 21850 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.698603 21850 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.699324 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.631325 21721 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:00:28.701467 21721 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:28.701910 21721 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:28.701609 20055 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:28.702754 21721 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.703199 22052 consensus_queue.cc:1059] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:28.704692 21861 raft_consensus.cc:3055] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.704627 21903 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:28.705171 21861 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:28.705811 20013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 621) took 1777 ms (client timeout 1992 ms). Trace:
I20260709 13:00:28.706073 21861 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:28.706259 20013 rpcz_store.cc:269] 0709 13:00:26.928411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.928549 (+   138us) service_pool.cc:224] Handling call
0709 13:00:28.705597 (+1777048us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400091fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:28.705724 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2133314816}
I20260709 13:00:28.707273 21861 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.656916 22044 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.708704 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.709656 20859 raft_consensus.cc:2393] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:28.710438 22044 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.710952 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:28.712594 20850 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:28.716111 21880 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:28.718086 22021 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.719030 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 274) took 1264 ms. Trace:
I20260709 13:00:28.719507 21811 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:28.720191 21811 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.722436 21811 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.724943 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:28.725046 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 825) took 1262 ms. Trace:
I20260709 13:00:28.728044 21943 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.729039 21895 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:28.728722 21943 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.729604 21895 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:28.731432 22049 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:00:28.731792 21943 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:28.652302 21974 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:28.735407 21974 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.736532 20055 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:28.737466 21899 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.738251 21899 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:28.739005 21899 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:28.710500 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.740608 21982 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:28.742213 21855 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:28.742589 21855 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:28.743294 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.743378 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:28.744903 21688 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:28.745275 21688 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.745702 21688 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.743394 21855 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.746435 21855 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:28.733814 21721 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:28.691498 21565 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.748853 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:28.750691 21565 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:28.751036 22010 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:28.751523 22010 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.751430 21899 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:28.751999 22010 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:28.754951 20851 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:28.732643 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.738169 21974 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:28.719455 20015 rpcz_store.cc:276] 0709 13:00:27.461929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.467410 (+  5481us) service_pool.cc:224] Handling call
0709 13:00:28.718860 (+1251450us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540064f5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:28.718968 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175657216}
I20260709 13:00:28.758028 20859 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:28.765956 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:28.767318 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:28.768152 20090 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.769306 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.776472 20856 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:28.749709 20088 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:28.787750 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.795480 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.796262 20857 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:28.797567 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.801441 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:28.802510 20084 raft_consensus.cc:2393] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:28.806424 20853 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:28.756309 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:28.807317 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51060.4us
I20260709 13:00:28.807873 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:28.808651 20853 raft_consensus.cc:2368] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:28.814962 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.726810 20160 rpcz_store.cc:276] 0709 13:00:27.463092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.468366 (+  5274us) service_pool.cc:224] Handling call
0709 13:00:27.502769 (+ 34403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:28.011784 (+509015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.011815 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:28.016090 (+  4275us) log.cc:844] Serialized 52 byte log entry
0709 13:00:28.312437 (+296347us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:28.312473 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.724621 (+412148us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54004d6a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:28.724671 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.724719 (+    48us) raft_consensus.cc:1685] finished
0709 13:00:28.724732 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.724982 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136606,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":8445,"spinlock_wait_cycles":608783616,"thread_start_us":451233,"threads_started":2,"wal-append.queue_time_us":298119}
I20260709 13:00:28.817184 20855 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:00:28.824347 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 68448.6us
W20260709 13:00:28.827489 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.830124 20016 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:28.833616 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.835843 20853 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:28.834372 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:28.836706 20082 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:28.836620 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:28.842221 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:28.695457 22042 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.837913 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:28.846706 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:28.850597 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.852172 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:28.852810 20853 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.856654 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:28.869136 20855 raft_consensus.cc:2393] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:28.851651 20859 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:28.874389 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:28.875428 20011 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:28.805635 22052 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.795094 21895 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.747231 20159 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:28.806166 21944 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:28.879292 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 103068us
I20260709 13:00:28.878139 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:28.889264 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:28.891274 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44648.3us
I20260709 13:00:28.813967 21955 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:28.889086 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:28.898749 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:28.835793 22021 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.870149 22040 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:28.914968 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.927482 20163 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.923590 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:28.913357 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.922560 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.935386 20853 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:28.936659 20859 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:28.916930 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:28.925266 21955 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.945324 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.891950 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.946446 20160 raft_consensus.cc:2368] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:28.835740 21974 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:00:28.957857 20016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:28.913576 22018 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:28.966930 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50122.8us
I20260709 13:00:28.968750 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:28.959012 20016 raft_consensus.cc:606] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:28.881739 21856 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:00:28.877519 21777 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:00:28.921854 22021 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.963485 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:28.949563 22031 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:00:28.977437 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:28.989037 20859 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:28.976783 22026 log.cc:927] Time spent T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:00:28.931272 22040 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:28.982228 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:28.979437 22016 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:28.911643 21944 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:28.998502 20160 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:28.979161 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 80495.7us
I20260709 13:00:28.978884 21955 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:28.986994 22039 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:28.981220 21831 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:28.989208 21801 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:28.988878 22021 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.989111 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.003890 20011 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:29.016875 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 340) took 1468 ms. Trace:
I20260709 13:00:29.022869 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:29.023469 21831 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:28.999711 22040 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.034236 20090 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:29.059376 20163 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
W20260709 13:00:29.035772 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67248.6us
I20260709 13:00:29.020649 22039 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:29.022310 21850 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:29.000511 22016 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:29.039593 20165 rpcz_store.cc:276] 0709 13:00:27.548172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.940918 (+392746us) service_pool.cc:224] Handling call
0709 13:00:27.941302 (+   384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:28.260845 (+319543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.260876 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:28.274061 (+ 13185us) log.cc:844] Serialized 52 byte log entry
0709 13:00:28.368318 (+ 94257us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:28.372481 (+  4163us) log.cc:844] Serialized 14 byte log entry
0709 13:00:28.383481 (+ 11000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.383581 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.533723 (+150142us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b1800214180. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:28.730832 (+197109us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:29.016458 (+285626us) raft_consensus.cc:1685] finished
0709 13:00:29.016493 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:29.016780 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20957,"prepare.queue_time_us":18080,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":130251,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":33281,"spinlock_wait_cycles":18873856,"thread_start_us":405883,"threads_started":2,"wal-append.queue_time_us":157164}
I20260709 13:00:29.059435 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.092044 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:29.090544 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 67773.2us
I20260709 13:00:29.094374 20129 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:29.095373 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:29.100245 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:29.103184 20744 leader_election.cc:400] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5.
I20260709 13:00:29.105294 20090 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:29.106964 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:29.108317 20161 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:29.108220 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.026214 22024 consensus_queue.cc:1059] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:00:29.038656 21696 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.026510 22018 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:00:29.168931 22042 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:29.139063 21801 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:29.100435 22036 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.091553 22039 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.099900 21964 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:29.178591 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55225.9us
I20260709 13:00:29.112010 21850 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.200130 21944 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:29.117300 20012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:29.094990 20159 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:29.224135 21964 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:29.224709 22069 log.cc:927] Time spent T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:00:29.225188 21964 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:29.227391 22067 log.cc:927] Time spent T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:00:29.228580 22014 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:29.230167 22065 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:00:29.230770 21944 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.231359 21944 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.232211 21688 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.233610 21944 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:29.234627 21688 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:29.236591 22064 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:00:29.177479 20160 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:29.201642 21974 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:29.276855 22028 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.276s	user 0.001s	sys 0.002s
I20260709 13:00:29.277330 21974 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:29.278693 21974 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.222312 22076 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.265388 21831 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:29.271359 21852 consensus_meta.cc:287] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.002s
W20260709 13:00:29.310987 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 210816us
I20260709 13:00:29.314308 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.274998 21985 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.324653 21985 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:29.330519 21855 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 13:00:29.333719 21884 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.270222 20012 raft_consensus.cc:606] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.315387 21852 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:29.339779 21861 consensus_meta.cc:287] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.007s
I20260709 13:00:29.388665 20854 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:29.410434 22039 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.413336 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:29.366423 20010 raft_consensus.cc:2368] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7.
I20260709 13:00:29.367333 21852 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:29.414839 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.432597 20086 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:29.415688 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.433806 20083 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:29.368613 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 904) took 2320 ms. Trace:
I20260709 13:00:29.349066 20012 raft_consensus.cc:614] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:29.434686 20014 rpcz_store.cc:276] 0709 13:00:27.047747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.134151 (+ 86404us) service_pool.cc:224] Handling call
0709 13:00:27.134449 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.040327 (+905878us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400422320. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 13:00:28.057485 (+ 17158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.368282 (+1310797us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b5400422320. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b5a02 00007fa02e7b1dd6 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:29.368539 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279628032}
W20260709 13:00:29.359453 22010 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.001s
I20260709 13:00:29.416569 20160 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:29.435832 22039 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:29.436189 22010 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.438836 22010 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:29.439833 20158 raft_consensus.cc:2368] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7.
I20260709 13:00:29.417824 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:29.421797 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 527191us
I20260709 13:00:29.422431 21716 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:29.426134 21878 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:29.443995 21878 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.446280 21878 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:29.407686 21721 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.003s
W20260709 13:00:29.374033 22030 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 1.275s	user 0.000s	sys 0.004s
I20260709 13:00:29.434299 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:29.449709 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:29.449944 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.450471 20852 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:00:29.450729 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.450817 20014 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:29.429175 21958 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.434260 22036 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:29.451948 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.452625 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:29.466300 20161 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:29.453540 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.467672 20017 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:29.469733 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.454888 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:29.456954 20054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.008s	sys 0.010s
W20260709 13:00:29.475854 20054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.008s	sys 0.010s
I20260709 13:00:29.457535 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.345008 22040 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:29.413341 22021 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.461009 20016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:29.461367 22053 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:29.378237 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 827) took 1830 ms. Trace:
I20260709 13:00:29.464170 21338 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:29.490450 21338 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.464480 21801 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:29.464900 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.471578 20744 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:29.472837 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.453913 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:29.410090 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 95936.2us
I20260709 13:00:29.498778 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.475320 20015 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:29.459743 22052 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.480710 20016 raft_consensus.cc:606] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:29.500690 20016 raft_consensus.cc:614] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:29.493741 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.474295 22084 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:29.465217 22001 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.498948 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.515019 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:29.478264 20852 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:29.515728 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.518604 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:29.519151 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.521277 19981 leader_election.cc:400] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:29.521746 19981 leader_election.cc:403] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:29.526959 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.527315 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:29.528149 20016 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.525205 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.529418 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:29.531093 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59651.1us
I20260709 13:00:29.535601 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.492738 22036 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:29.537408 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:29.582651 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 46295.6us
I20260709 13:00:29.540189 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:29.584378 20085 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.585686 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:29.516047 22052 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.586081 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:29.589840 20084 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.587129 21338 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:29.513468 21721 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.513139 21982 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.522926 21855 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.475507 22010 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
W20260709 13:00:29.588654 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51324.7us
I20260709 13:00:29.517608 21716 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.595966 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:29.617210 20015 raft_consensus.cc:2410] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:29.526432 21971 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.608731 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:29.525759 21852 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.540756 21833 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:29.542320 22083 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.523447 21688 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:00:29.523640 22082 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.526834 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.582592 21796 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.534314 22084 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.624696 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.586453 22039 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.541905 22015 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.594954 21878 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:29.625528 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 92524.6us
I20260709 13:00:29.635977 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.634406 20084 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:29.512367 22053 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.627481 21855 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:29.616024 21881 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.533963 21899 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 13:00:29.652763 20854 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:29.527215 22001 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.642377 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:29.643291 22050 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.173s	user 0.000s	sys 0.005s
I20260709 13:00:29.643187 20012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:29.587724 22042 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:00:29.627990 21845 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:29.623334 21982 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.647439 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.582062 21760 consensus_queue.cc:1059] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:00:29.638396 20083 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.637230 21833 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:29.651201 21721 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:29.699887 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.704136 21878 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:29.652834 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:29.659708 21688 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:00:29.659619 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 286) took 1414 ms. Trace:
I20260709 13:00:29.719452 19838 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32774 (request call id 151) took 9457 ms. Trace:
I20260709 13:00:29.654528 22036 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:29.657653 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:29.718916 21721 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:29.720299 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 68040.5us
I20260709 13:00:29.726989 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:29.727146 21944 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:29.727581 21944 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:29.727643 20089 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.728067 21944 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:29.728832 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.730559 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:29.710371 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:29.661533 22053 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:29.700742 22010 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.732647 22010 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.733412 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 916) took 1353 ms. Trace:
I20260709 13:00:29.734368 20159 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:29.735021 22010 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:29.735682 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.736501 20163 raft_consensus.cc:2368] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4.
I20260709 13:00:29.736733 21974 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:29.739996 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 82583.1us
I20260709 13:00:29.739203 21721 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.745321 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.746291 20090 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:29.747330 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.748852 22001 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:29.749405 20743 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:29.750030 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.750336 22093 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:29.750809 22093 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.720448 21338 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:29.750809 20084 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:29.751700 21338 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.753026 20083 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:29.753813 21338 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:29.488660 20156 rpcz_store.cc:276] 0709 13:00:27.547322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.552046 (+  4724us) service_pool.cc:224] Handling call
0709 13:00:29.371751 (+1819705us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:29.377134 (+  5383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:29.377673 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.377714 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:29.377774 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:29.377828 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.377902 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:29.378156 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9070848}
I20260709 13:00:29.668768 22021 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:29.669109 21881 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.681864 22015 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:00:29.733814 20008 rpcz_store.cc:276] 0709 13:00:28.379894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.590368 (+210474us) service_pool.cc:224] Handling call
0709 13:00:28.655589 (+ 65221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.711604 (+ 56015us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:28.712007 (+   403us) log.cc:844] Serialized 14 byte log entry
0709 13:00:29.425748 (+713741us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:29.425803 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:29.425861 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.733189 (+307328us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b54003e6554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:29.733237 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:29.733357 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":433168,"prepare.queue_time_us":741,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":433465,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"spinlock_wait_cycles":127674752,"thread_start_us":280140,"threads_started":1,"wal-append.queue_time_us":377005}
I20260709 13:00:29.766620 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.767386 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.704702 20012 raft_consensus.cc:606] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:29.717301 21845 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:29.720813 22082 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:29.696517 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:29.757314 20850 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:29.773241 20851 raft_consensus.cc:2410] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:29.687856 22040 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:29.706445 21796 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:29.774770 21796 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.724038 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.751235 22093 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.758522 20011 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:29.721312 19839 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:29.662925 21855 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:29.767860 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.808152 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.719825 20091 rpcz_store.cc:276] 0709 13:00:28.245017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.245205 (+   188us) service_pool.cc:224] Handling call
0709 13:00:28.245668 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.246105 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.246133 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.246705 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:00:28.355332 (+108627us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:28.355386 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.355478 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:29.642755 (+1287277us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:29.659358 (+ 16603us) raft_consensus.cc:1685] finished
0709 13:00:29.659381 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:29.659566 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4326,"raft.queue_time_us":3003,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":6631,"spinlock_wait_cycles":6964096,"thread_start_us":108123,"threads_started":1,"wal-append.queue_time_us":165493}
I20260709 13:00:29.813750 20089 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:29.778226 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:29.737900 21944 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:29.765226 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.764258 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.770920 20008 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:29.771225 20164 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:29.771631 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 296) took 1048 ms. Trace:
I20260709 13:00:29.687956 20085 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:29.771858 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:29.786900 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 128765us
I20260709 13:00:29.767904 22052 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.788188 20015 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:29.772275 21845 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.772310 22082 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:29.762797 22016 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.760509 21854 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:29.754760 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:29.841917 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 87241.4us
I20260709 13:00:29.771246 21847 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:29.866144 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.772464 21878 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:29.671972 22084 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:29.761076 21982 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:29.760291 21955 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:29.797672 21852 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.800844 22083 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.872124 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.779850 19839 catalog_manager.cc:5697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 3, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:00:29.876974 20087 raft_consensus.cc:2368] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:29.878273 19839 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:29.806154 20016 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:29.786569 20090 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:29.820326 20088 rpcz_store.cc:276] 0709 13:00:28.723707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.723947 (+   240us) service_pool.cc:224] Handling call
0709 13:00:28.724393 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.806857 (+ 82464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.771388 (+964531us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b5400781254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:29.771571 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57739648}
I20260709 13:00:29.838258 20744 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:29.837639 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81fd99f42a47497f9d4372157320611b"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:29.845782 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:29.881680 20017 raft_consensus.cc:606] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:29.881776 20161 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.882076 20017 raft_consensus.cc:614] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:29.759302 21881 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:29.756531 21760 raft_consensus.cc:3055] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:29.777096 21796 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:29.874321 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:29.820127 21964 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.840867 21984 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.722987 19838 rpcz_store.cc:276] 0709 13:00:20.261790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.262026 (+   236us) service_pool.cc:224] Handling call
0709 13:00:20.583594 (+321568us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54008b1fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 13:00:29.719369 (+9135775us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:24.294288 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:24.336610 (+ 42322us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:24.336664 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:00:24.336691 (+    27us) tablet.cc:662] Decoding operations
0709 13:00:24.337500 (+   809us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:24.337577 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:24.337624 (+    47us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:00:24.338101 (+   477us) tablet.cc:701] Row locks acquired
0709 13:00:24.338128 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:00:24.338337 (+   209us) write_op.cc:270] Start()
0709 13:00:24.338484 (+   147us) write_op.cc:276] Timestamp: P: 1783602024338275 usec, L: 0
0709 13:00:24.338530 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:00:24.347248 (+  8718us) log.cc:844] Serialized 1307 byte log entry
0709 13:00:25.279571 (+932323us) op_driver.cc:464] REPLICATION: finished
0709 13:00:25.529605 (+250034us) write_op.cc:301] APPLY: starting
0709 13:00:25.529675 (+    70us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:25.529817 (+   142us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:25.529928 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:25.530375 (+   447us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:25.530540 (+   165us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:00:25.530587 (+    47us) write_op.cc:312] APPLY: finished
0709 13:00:25.530784 (+   197us) log.cc:844] Serialized 34 byte log entry
0709 13:00:25.530948 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:25.531034 (+    86us) write_op.cc:454] Released schema lock
0709 13:00:25.531078 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:25.531471 (+   393us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:25.531499 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10376,"reported_tablets":24,"spinlock_wait_cycles":641505664,"child_traces":[["op",{"apply.queue_time_us":245751,"apply.run_cpu_time_us":1886,"apply.run_wall_time_us":1886,"num_ops":3,"prepare.queue_time_us":184918,"prepare.run_cpu_time_us":76910,"prepare.run_wall_time_us":218456,"raft.queue_time_us":297146,"raft.run_cpu_time_us":42029,"raft.run_wall_time_us":237530,"replication_time_us":900377,"spinlock_wait_cycles":8960,"thread_start_us":473423,"threads_started":4,"wal-append.queue_time_us":348988,"wal-append.run_cpu_time_us":23852,"wal-append.run_wall_time_us":1816585}]]}
I20260709 13:00:29.803860 20163 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:29.899253 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:29.884466 20162 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:29.886288 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33788 (request call id 346) took 1522 ms. Trace:
I20260709 13:00:29.886322 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 288) took 1030 ms. Trace:
I20260709 13:00:29.902459 20009 rpcz_store.cc:276] 0709 13:00:28.855409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.856153 (+   744us) service_pool.cc:224] Handling call
0709 13:00:29.871902 (+1015749us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b5800867998. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68de9c 00007fa02b68da5b 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:29.886242 (+ 14340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51418496}
I20260709 13:00:29.839694 21688 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:29.907367 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.839124 21696 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:29.891516 20016 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:29.841198 21854 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:29.921195 22044 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:29.921942 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.922020 22042 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:29.904793 20855 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:29.894778 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:29.906636 22001 raft_consensus.cc:993] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:29.837617 21801 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.865252 21943 raft_consensus.cc:2764] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:29.904568 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:29.909336 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.901615 20164 rpcz_store.cc:276] 0709 13:00:28.364304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.365085 (+   781us) service_pool.cc:224] Handling call
0709 13:00:29.886087 (+1521002us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000cf3d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:29.886225 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3210387840}
I20260709 13:00:29.941839 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:29.942262 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:29.910386 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.913210 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:29.869544 22002 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.895896 21901 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.891892 21760 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:29.912171 21696 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.915719 22037 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:29.917779 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.956305 22037 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:29.956362 21944 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:29.957491 22001 raft_consensus.cc:1081] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:29.957569 22037 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.919533 21852 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:29.920370 22084 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.921763 22044 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.905325 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:29.923210 22016 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.923254 19982 leader_election.cc:400] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
W20260709 13:00:29.964373 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51235.3us
I20260709 13:00:29.930385 20008 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:00:29.889008 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.936022 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.935238 22035 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:29.939606 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:29.978890 21760 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.912478 21964 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:29.704753 21833 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.939357 21338 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:29.944891 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.915424 20162 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:29.922647 22042 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.942708 20014 raft_consensus.cc:2376] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6. Re-sending same reply.
I20260709 13:00:29.950089 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:29.955252 22002 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:29.982993 20088 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:29.955752 22052 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:29.908785 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:29.875792 21955 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:29.965647 19982 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:29.972816 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:29.973603 20161 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:29.959142 22037 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:29.965561 21765 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.977738 21855 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:30.031232 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 65966.2us
I20260709 13:00:29.978330 22035 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:29.948017 21801 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.960024 21696 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:29.938016 22024 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.009s	sys 0.009s
I20260709 13:00:30.004877 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 707) took 1380 ms. Trace:
W20260709 13:00:30.006150 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.028753 20085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:00:30.042089 20085 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:30.042627 20085 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.034669 20016 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:30.035614 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62911.4us
I20260709 13:00:30.046684 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:30.000535 22042 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.983343 21971 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:29.992357 20157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:30.040737 20858 rpcz_store.cc:276] 0709 13:00:28.624409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.626670 (+  2261us) service_pool.cc:224] Handling call
0709 13:00:28.660337 (+ 33667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.660610 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:28.667612 (+  7002us) log.cc:844] Serialized 14 byte log entry
0709 13:00:29.714735 (+1047123us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b1800002e00. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a 00007fa02e7bca2a
0709 13:00:30.004510 (+289775us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:30.004565 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:30.004618 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.004685 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.004791 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4255,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":24210,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":3637,"spinlock_wait_cycles":5094784,"thread_start_us":289531,"threads_started":1,"wal-append.queue_time_us":1336742}
I20260709 13:00:30.029531 21760 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.969964 22084 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:30.081184 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.989665 22001 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:00:29.964370 21901 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:29.980099 21984 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
W20260709 13:00:30.040500 22024 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.009s	sys 0.009s
I20260709 13:00:30.085448 21338 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:30.083112 20859 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:29.959941 22082 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.033366 22015 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:00:30.043642 22052 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:30.105533 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 267337us
I20260709 13:00:30.081543 22095 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:30.045012 20156 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:30.110149 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:30.097805 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:30.095144 20157 raft_consensus.cc:606] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:30.113215 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 269760us
I20260709 13:00:30.096462 21971 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:30.127123 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:30.115398 20085 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.128808 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:30.131711 20744 leader_election.cc:400] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7.
I20260709 13:00:30.080329 22093 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.136914 20085 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.138724 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:30.141433 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:30.144022 20056 leader_election.cc:400] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7.
I20260709 13:00:30.144469 20056 leader_election.cc:403] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:30.085803 21955 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.134512 21944 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:30.134378 21984 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:30.158645 20056 leader_election.cc:400] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4.
I20260709 13:00:30.191403 20056 leader_election.cc:403] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:30.127599 22052 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:30.136368 22095 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:30.143836 21899 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:30.130575 20157 raft_consensus.cc:614] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:30.130589 22019 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:30.202809 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 170705us
I20260709 13:00:30.134407 21943 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:30.142953 21855 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:30.199812 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 59328.3us
I20260709 13:00:30.145470 21852 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:30.204145 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45682.4us
I20260709 13:00:30.160497 21874 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:30.156602 21849 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:30.140628 21971 raft_consensus.cc:697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:30.189177 21974 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7.
I20260709 13:00:30.190819 21944 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:30.218619 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:30.220063 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 86924.1us
I20260709 13:00:30.223157 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:30.204152 21955 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.124423 22035 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.206358 21982 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 3. Current term is 4.
I20260709 13:00:30.243438 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:30.249501 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:30.249737 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:30.251248 20008 raft_consensus.cc:2410] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:30.222260 21874 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:30.230705 21849 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:30.267282 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:30.245059 21944 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.216718 22019 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.301659 21852 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.258503 21565 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.236537 22036 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:00:30.310083 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:30.308221 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 48686.3us
I20260709 13:00:30.322788 20854 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.281453 21849 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.315006 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:30.323777 20853 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:30.316834 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:30.332821 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:30.341178 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:30.341284 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:30.341890 20161 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.355698 20129 leader_election.cc:400] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:30.357537 20129 leader_election.cc:403] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:30.357234 20161 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:30.314321 22084 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:30.362275 22083 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:30.362403 22106 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:30.364326 22103 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 13:00:30.364987 22109 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:00:30.310781 21801 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:30.363714 22105 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:00:30.384423 22122 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:30.385136 22122 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.386037 21801 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:30.386629 21801 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.387540 22122 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:30.348706 22101 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:00:30.361431 20853 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:30.326316 21881 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:30.270740 22052 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.429134 22120 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:30.369307 20854 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:30.376618 20744 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:30.359671 21984 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.349913 21856 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:00:30.313141 21901 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.355899 21974 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:30.356878 22093 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:30.371101 22097 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
I20260709 13:00:30.397171 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 923) took 1588 ms. Trace:
I20260709 13:00:30.360113 22040 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:30.365199 22107 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:00:30.389942 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:30.404315 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 187924us
I20260709 13:00:30.475239 20853 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:30.349409 21688 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:00:30.293516 22095 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.318727 21971 consensus_queue.cc:237] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.401994 21796 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:30.349134 21874 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.422411 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:30.375520 22106 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:30.461314 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 84825.5us
I20260709 13:00:30.476246 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:30.476786 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:30.494685 20856 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:30.494794 20853 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:30.482285 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 92463.9us
I20260709 13:00:30.471043 20013 rpcz_store.cc:276] 0709 13:00:28.808324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.810168 (+  1844us) service_pool.cc:224] Handling call
0709 13:00:28.862253 (+ 52085us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:28.873504 (+ 11251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:29.449415 (+575911us) spinlock_profiling.cc:243] Waited 8.58 ms on lock 0x7b18002039e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 13:00:29.618387 (+168972us) log.cc:844] Serialized 14 byte log entry
0709 13:00:30.207579 (+589192us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:30.216117 (+  8538us) log.cc:844] Serialized 52 byte log entry
0709 13:00:30.218815 (+  2698us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:30.218857 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.219074 (+   217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.377756 (+158682us) raft_consensus.cc:1685] finished
0709 13:00:30.396831 (+ 19075us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.397068 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":541764,"prepare.queue_time_us":761199,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":3347,"raft.queue_time_us":101215,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":14985,"spinlock_wait_cycles":19723008,"thread_start_us":717881,"threads_started":3,"wal-append.queue_time_us":352527}
I20260709 13:00:30.477955 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:30.475615 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:30.479085 20854 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
W20260709 13:00:30.467897 22123 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
I20260709 13:00:30.480746 20852 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:00:30.463406 22122 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:30.513808 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:30.514691 20085 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.500538 20855 raft_consensus.cc:2368] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:30.499806 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:30.501268 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:30.502872 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:30.517486 20089 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:30.469156 21801 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:30.461535 21881 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.467626 21984 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.531617 21881 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.532315 21984 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.519146 22019 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:30.534353 22093 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:30.535049 22093 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:30.526499 22129 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:00:30.528481 22116 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:00:30.536053 22093 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.502786 21974 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:30.529683 22124 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:30.536123 22042 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.539954 22124 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.540405 22124 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.527217 22106 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.534444 22107 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:30.511538 20853 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:30.538044 21974 raft_consensus.cc:697] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:30.560312 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.563397 20856 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:30.479038 21849 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.473037 21696 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.572271 20085 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:30.503479 21796 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.582800 21849 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:30.591979 22042 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.567996 22107 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.593310 21944 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.594906 21696 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.554584 21874 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:30.596382 21845 consensus_meta.cc:287] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.731s	user 0.015s	sys 0.000s
I20260709 13:00:30.600296 21974 consensus_queue.cc:237] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.608317 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:30.603611 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 97871.7us
W20260709 13:00:30.605792 22037 consensus_meta.cc:287] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.001s
I20260709 13:00:30.523845 22122 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.616863 20856 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:30.619261 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:30.619771 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:30.620326 20086 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.620477 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:30.620471 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:30.621074 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:30.616061 22024 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:00:30.610890 21845 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.628620 22024 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:30.629068 22024 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:30.630333 21845 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:30.629973 22024 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.630681 21964 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:30.631403 22024 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.612916 22082 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.632048 22052 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:30.620359 22040 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:00:30.631453 20010 raft_consensus.cc:2410] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:30.633782 22044 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.635576 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:30.640726 21851 raft_consensus.cc:993] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:30.643441 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:30.638872 20743 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:30.646993 20086 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:30.647347 20084 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:30.649040 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:30.654253 22122 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.641821 22042 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:30.650852 21801 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:30.650169 22036 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:30.666654 22107 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.669314 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:30.670198 20084 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:30.658879 22042 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:30.667534 22002 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:30.673192 21854 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.671327 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:30.671042 22036 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:30.655627 21696 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:00:30.655270 22124 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.690718 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:30.691553 20084 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.702144 21943 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:30.702708 21943 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:30.703068 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.702905 22117 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:30.703707 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.703886 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 278411us
I20260709 13:00:30.706075 22111 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.685375 22133 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:30.706585 22111 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.707159 22133 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:30.707552 22133 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.702167 22121 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:30.710107 22111 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:30.712410 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:30.713223 20856 raft_consensus.cc:2393] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:30.713611 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:30.714588 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:30.715543 20082 raft_consensus.cc:2368] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:30.716208 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:30.716677 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:30.718538 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:30.724776 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:30.725951 20086 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:30.720386 21851 raft_consensus.cc:1081] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:00:30.742189 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:30.743908 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:30.675243 22042 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:30.675359 21801 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:30.751797 20082 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:30.756050 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 927) took 1050 ms. Trace:
I20260709 13:00:30.772495 22015 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:30.777163 21944 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.778909 21796 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.780540 21801 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.782401 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:30.783201 20082 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:30.783473 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 736) took 1018 ms. Trace:
I20260709 13:00:30.768388 22137 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:30.783950 20156 rpcz_store.cc:276] 0709 13:00:29.764629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.833883 (+ 69254us) service_pool.cc:224] Handling call
0709 13:00:30.783284 (+949401us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b54000cf3d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:30.783405 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148878720}
I20260709 13:00:30.747375 21971 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.785411 21971 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.788302 21796 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:30.788988 21796 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:30.792128 21796 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.792436 21943 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.794121 21943 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.802211 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:00:30.804515 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:30.809417 20084 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:30.811198 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:30.812127 22107 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:30.812659 22107 raft_consensus.cc:697] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:30.813555 22107 consensus_queue.cc:237] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.820640 22014 raft_consensus.cc:3055] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:30.821184 22014 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:30.821795 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:30.822068 22014 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.822665 21849 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.823190 22014 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:30.823505 21849 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:30.824231 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:30.842140 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:30.844439 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 72783.8us
I20260709 13:00:30.756917 20011 rpcz_store.cc:276] 0709 13:00:29.705933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.892292 (+186359us) service_pool.cc:224] Handling call
0709 13:00:29.934647 (+ 42355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:29.967460 (+ 32813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.967491 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:29.968197 (+   706us) log.cc:844] Serialized 52 byte log entry
0709 13:00:30.157494 (+189297us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:30.235152 (+ 77658us) log.cc:844] Serialized 14 byte log entry
0709 13:00:30.699315 (+464163us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800320200. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:00:30.699755 (+   440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.699896 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.712593 (+ 12697us) raft_consensus.cc:1685] finished
0709 13:00:30.755595 (+ 43002us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.755833 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70837,"prepare.queue_time_us":51689,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":9146,"raft.queue_time_us":499177,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":8667,"spinlock_wait_cycles":24092544,"thread_start_us":161580,"threads_started":1,"wal-append.queue_time_us":424151}
W20260709 13:00:30.854508 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.867597 19981 leader_election.cc:400] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:30.868202 19981 leader_election.cc:403] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:30.869032 21338 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:30.872376 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 649321us
I20260709 13:00:30.869143 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:30.884987 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:30.885898 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.887557 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45540.9us
I20260709 13:00:30.890782 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:30.894034 21338 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5.
I20260709 13:00:30.895112 21338 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:30.895843 21338 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.896313 21338 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:30.904202 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:30.897351 22132 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.905311 22132 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.905920 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:30.907464 22132 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.908344 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:30.864233 22124 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:30.887503 22140 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:00:30.703729 22117 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.871282 21796 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:30.894238 22002 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:30.916980 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:30.923470 20157 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:00:30.931025 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.941656 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:30.943336 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:30.947116 20010 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:30.915179 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:30.961383 20011 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:30.963014 21338 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.962908 21796 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:30.909428 22115 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:30.977494 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:30.969959 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:30.978461 20013 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:30.981143 22142 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:00:30.981619 22132 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:30.982630 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:30.983549 20014 raft_consensus.cc:2410] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:00:30.984818 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:30.979513 21971 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:30.941558 21765 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.988169 22037 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:30.988734 22037 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:30.990813 22037 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:30.994293 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:30.996281 22118 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:30.997447 22141 log.cc:927] Time spent T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.334s	user 0.000s	sys 0.000s
I20260709 13:00:31.000005 22082 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:30.947690 22002 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.001127 22002 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.003180 22117 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:30.902868 21878 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:00:30.942845 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.004693 21878 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:31.005122 21878 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:31.005963 21878 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:31.007272 21878 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:31.007982 22044 consensus_queue.cc:1175] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8b5b4e2a92b046d096f2661c54321d41" responder_term: 5 status { last_received { term: 5 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 5 index: 3 } }
I20260709 13:00:31.008877 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:31.009683 20010 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:31.009793 20856 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:31.010118 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:30.996940 22118 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:31.012861 22145 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
I20260709 13:00:30.971424 22124 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.017158 22124 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:30.995621 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:31.020884 20850 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:31.021082 22093 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.022100 21903 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.903196 22036 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:31.023239 22083 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:31.023806 22083 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:31.030383 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:31.034240 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:31.040264 20016 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:31.026309 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:31.042016 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:31.043845 20008 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:31.045620 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:31.046494 20084 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
W20260709 13:00:31.048401 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.064172 22042 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:31.066639 22042 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:31.069844 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:31.073915 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:31.074880 20859 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.048992 20084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:00:31.075740 20084 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:31.076161 20084 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.051390 21338 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:31.054596 20165 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.054737 20087 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:31.050465 22117 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:31.058314 22127 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:31.041940 21696 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.094659 20852 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:31.060019 22083 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:31.030201 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.082496 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:31.057344 22140 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:00:31.092032 20857 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:30.941807 22015 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 13:00:31.081351 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:31.129307 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48085.7us
I20260709 13:00:31.130605 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:31.102905 20854 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:31.132566 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:31.115785 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.090149 20013 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:31.121222 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.137421 20162 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:31.138576 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.138887 19982 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:31.139261 20162 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:31.102839 21849 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:31.143857 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.121563 20859 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:31.149317 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:31.135277 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:31.141057 21854 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:31.156087 21849 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:31.156631 22042 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:31.157253 22042 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:31.157505 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:31.157339 21849 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:31.158133 20160 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.139859 22120 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:00:31.164074 22149 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:31.165238 22149 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:31.165796 22149 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:31.166018 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.168201 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.121008 21696 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:31.132195 22127 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:31.134169 21850 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.055507 21903 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:31.179522 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:31.187266 20008 raft_consensus.cc:3055] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:31.187608 20008 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:31.188396 20008 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:31.189247 20008 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:31.180591 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:31.192682 20016 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:31.193948 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:31.152210 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 251503us
I20260709 13:00:31.140555 22126 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.185885 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:31.197155 22126 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:31.197364 20010 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:31.197647 22126 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.187456 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:31.148952 21874 consensus_meta.cc:287] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.003s
I20260709 13:00:31.200363 20014 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:31.152156 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:31.200521 22124 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:31.203859 22118 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:31.205005 22002 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:31.146173 22117 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:31.211241 20009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.015s	sys 0.022s
W20260709 13:00:31.213088 20009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.015s	sys 0.022s
W20260709 13:00:31.207685 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55620.1us
I20260709 13:00:31.236205 20161 raft_consensus.cc:1275] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:00:31.222111 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:31.274370 20164 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:31.221882 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:31.215893 20160 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:31.233959 20084 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:31.298332 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 76571.2us
I20260709 13:00:31.299888 22120 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.303330 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:31.240617 20012 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:31.304160 21696 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:31.220705 21972 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.304420 22124 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:31.304888 21972 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:31.305023 22124 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.305555 22124 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:31.306089 21903 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:31.309587 22002 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:31.216807 21796 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:31.305382 21972 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.311599 20164 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:31.314082 21801 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:00:31.320257 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 111125us
I20260709 13:00:31.320549 21851 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.321055 19981 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:31.324313 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:31.325109 20010 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:31.327788 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 50839.8us
I20260709 13:00:31.327895 21338 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:31.209583 22117 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:31.320598 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 655) took 1558 ms. Trace:
I20260709 13:00:31.344379 20009 rpcz_store.cc:276] 0709 13:00:29.762260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.903482 (+141222us) service_pool.cc:224] Handling call
0709 13:00:29.936237 (+ 32755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:29.936466 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:29.936734 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:00:30.658509 (+721775us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b18004068a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a 00007fa02e7bca2a
0709 13:00:31.320206 (+661697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:31.320267 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:31.320320 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.320414 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:31.320535 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104914,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":107816,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":5833472,"thread_start_us":556327,"threads_started":1}
I20260709 13:00:31.158146 22042 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:31.297619 22107 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:31.422784 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.315708 22127 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:31.297401 22131 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:31.423796 20084 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:31.335505 20016 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:31.323810 21851 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:31.433694 20008 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:31.434674 21851 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.434847 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:31.438983 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.440737 20009 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:31.424454 22044 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.448060 21851 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.464553 22044 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:31.466684 22044 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:31.470741 22014 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:31.471414 22014 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:31.473795 22014 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:31.449631 22133 consensus_meta.cc:287] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.730s	user 0.012s	sys 0.001s
W20260709 13:00:31.487639 21974 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.024s	sys 0.034s
W20260709 13:00:31.489761 20129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.010s	sys 0.047s
W20260709 13:00:31.491703 20129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.010s	sys 0.047s
I20260709 13:00:31.475035 22117 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:31.504271 20165 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:31.436946 22107 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:31.451547 22131 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:31.492394 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 878158us
I20260709 13:00:31.509400 20129 leader_election.cc:400] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:31.483698 22124 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:31.496263 22125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.007s	sys 0.020s
W20260709 13:00:31.491480 21974 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.024s	sys 0.034s
I20260709 13:00:31.525466 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:31.488056 22133 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:31.517151 22125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.101s	user 0.007s	sys 0.021s
W20260709 13:00:31.543525 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.517242 22117 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:31.537694 21964 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.023s	sys 0.045s
W20260709 13:00:31.582863 21964 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.024s	sys 0.046s
I20260709 13:00:31.602344 22133 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:31.604863 22155 log.cc:927] Time spent T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:00:31.610697 22131 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:31.612528 22107 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:31.616990 22124 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:31.615753 22151 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:00:31.635370 22149 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:31.643411 22149 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:31.693737 20015 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.012s	sys 0.008s
W20260709 13:00:31.694998 20054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.000s	sys 0.033s
W20260709 13:00:31.695441 20017 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.006s	sys 0.012s
W20260709 13:00:31.696133 20056 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.006s	sys 0.027s
W20260709 13:00:31.696547 20056 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.006s	sys 0.027s
W20260709 13:00:31.703157 20158 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.001s	sys 0.028s
W20260709 13:00:31.703866 20158 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.002s	sys 0.028s
W20260709 13:00:31.708206 20163 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.007s	sys 0.020s
W20260709 13:00:31.708866 20163 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.008s	sys 0.020s
W20260709 13:00:31.696085 20017 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.007s	sys 0.012s
W20260709 13:00:31.695194 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 181565us
W20260709 13:00:31.695540 20054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.000s	sys 0.033s
I20260709 13:00:31.726125 20858 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
W20260709 13:00:31.640697 22093 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.619s	user 0.014s	sys 0.001s
I20260709 13:00:31.639415 22126 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:31.694998 20015 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.013s	sys 0.009s
I20260709 13:00:31.693929 20008 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:00:31.708249 20055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.007s	sys 0.008s
I20260709 13:00:31.778160 22124 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:31.784229 22083 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.711558 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:31.593792 21943 consensus_meta.cc:287] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.798s	user 0.011s	sys 0.008s
W20260709 13:00:31.860225 20055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.008s	sys 0.008s
I20260709 13:00:31.885596 21856 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:31.886467 21854 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:31.888154 20156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.003s	sys 0.011s
I20260709 13:00:31.887848 21854 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:31.888702 20156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.004s	sys 0.012s
W20260709 13:00:31.890365 20057 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.010s	sys 0.036s
W20260709 13:00:31.891255 20057 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.010s	sys 0.036s
I20260709 13:00:31.885964 21881 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:31.877224 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:31.887320 21856 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:31.910377 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 816822us
W20260709 13:00:31.946523 20013 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.010s	sys 0.013s
I20260709 13:00:31.948140 22165 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.954430 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:31.956691 21903 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.913614 21833 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:31.992157 20012 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:00:31.996484 20013 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.011s	sys 0.014s
W20260709 13:00:31.951704 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.048957 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:32.052250 21849 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.008s	sys 0.021s
W20260709 13:00:32.052927 21849 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.009s	sys 0.021s
I20260709 13:00:32.053822 20164 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:32.055346 22133 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.056344 22133 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.060178 22124 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:32.068059 22083 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.069022 20856 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:32.069864 20856 pending_rounds.cc:85] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 2
I20260709 13:00:32.070374 20856 pending_rounds.cc:107] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:32.070804 20856 raft_consensus.cc:2889] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:32.072955 22093 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:32.074604 22093 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.052331 21903 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.080446 21801 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.081733 21801 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:32.085741 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:32.088524 20084 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:32.092628 22126 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:32.053172 21856 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:32.094887 22153 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:00:32.034608 22149 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.096658 22149 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.099939 21856 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:32.101513 21856 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.104537 21903 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:32.105463 21903 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.157996 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 940) took 1123 ms. Trace:
I20260709 13:00:32.161218 22014 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.059751 22133 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:32.107111 21903 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:32.225122 21972 consensus_meta.cc:287] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.913s	user 0.002s	sys 0.013s
I20260709 13:00:32.224552 22165 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:32.226168 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.158607 20017 rpcz_store.cc:276] 0709 13:00:31.034135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.034246 (+   111us) service_pool.cc:224] Handling call
0709 13:00:31.034509 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.034736 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:31.709763 (+675027us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.157101 (+447338us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.157348 (+   247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.157424 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:32.157546 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.157655 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.157891 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":230879,"prepare.queue_time_us":415986,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":66617,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":320,"thread_start_us":791564,"threads_started":2}
I20260709 13:00:32.193526 21854 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.238296 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:32.208940 22083 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:32.259528 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:32.257735 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 852) took 1215 ms. Trace:
I20260709 13:00:32.246390 22133 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.192539 21881 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:32.269467 20014 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.140s	sys 0.210s
W20260709 13:00:32.301432 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.228024 21833 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:32.256613 22082 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:32.297784 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 586612us
W20260709 13:00:32.334811 20014 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.141s	sys 0.211s
W20260709 13:00:32.284138 22094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.013s	sys 0.061s
W20260709 13:00:32.278611 22095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.010s	sys 0.015s
I20260709 13:00:32.251520 21972 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.304787 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:32.355676 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.365765 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 384) took 1231 ms. Trace:
I20260709 13:00:32.305773 22149 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:32.344478 20157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.049s	sys 0.070s
I20260709 13:00:32.306872 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:32.306052 22124 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:32.361713 20011 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:32.303642 21881 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:32.357834 20158 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:32.305178 21854 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:32.327652 21958 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.010s	sys 0.001s
W20260709 13:00:32.344874 22042 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.004s	sys 0.008s
I20260709 13:00:32.372875 20011 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:32.265151 20163 rpcz_store.cc:276] 0709 13:00:31.037477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.169991 (+132514us) service_pool.cc:224] Handling call
0709 13:00:31.170293 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.170460 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:31.709476 (+539016us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.163866 (+454390us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.177923 (+ 14057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.188471 (+ 10548us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:32.201757 (+ 13286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.218700 (+ 16943us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.252135 (+ 33435us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60008,"prepare.queue_time_us":1318087,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":53139,"raft.run_cpu_time_us":611,"raft.run_wall_time_us":54082,"spinlock_wait_cycles":9216,"thread_start_us":648577,"threads_started":2}
I20260709 13:00:32.301182 22133 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:32.344946 22093 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:32.371543 22095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.011s	sys 0.016s
W20260709 13:00:32.371860 22094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.309s	user 0.015s	sys 0.061s
I20260709 13:00:32.345398 22014 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.366410 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:32.367354 21856 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.367218 21801 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:32.389686 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 938) took 1356 ms. Trace:
I20260709 13:00:32.368666 20162 rpcz_store.cc:276] 0709 13:00:31.027732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.163267 (+135535us) service_pool.cc:224] Handling call
0709 13:00:31.163534 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.163900 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:31.517492 (+353592us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.186843 (+669351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.196107 (+  9264us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:32.219573 (+ 23466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.232100 (+ 12527us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.259021 (+ 26921us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30824,"prepare.queue_time_us":1062141,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":77571,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":170,"thread_start_us":800179,"threads_started":2}
I20260709 13:00:32.398149 20085 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:32.396399 21958 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.011s	sys 0.002s
I20260709 13:00:32.400079 22082 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:32.391855 22042 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.005s	sys 0.008s
I20260709 13:00:32.392186 22167 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:32.404963 20088 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:32.401238 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:32.389012 22172 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:32.405496 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 317) took 1082 ms. Trace:
I20260709 13:00:32.399168 21854 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.395316 22149 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:32.391582 22113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.042s	sys 0.077s
I20260709 13:00:32.419633 20008 rpcz_store.cc:276] 0709 13:00:31.323156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.442426 (+119270us) service_pool.cc:224] Handling call
0709 13:00:31.445403 (+  2977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.877756 (+432353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.405032 (+527276us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54000fa5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:32.405403 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568613248}
I20260709 13:00:32.430426 21903 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:32.432284 21801 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.432394 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:32.432719 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:32.433084 21801 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:32.434432 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:32.435274 20085 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:32.436595 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:32.438277 21851 consensus_meta.cc:287] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.975s	user 0.013s	sys 0.016s
I20260709 13:00:32.438951 21851 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.440778 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 850) took 1514 ms. Trace:
I20260709 13:00:32.441556 20158 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:32.444248 22149 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.407533 20015 rpcz_store.cc:276] 0709 13:00:31.033548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.035488 (+  1940us) service_pool.cc:224] Handling call
0709 13:00:31.035937 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.036282 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:31.888761 (+852479us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.273743 (+384982us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.282325 (+  8582us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:32.344378 (+ 62053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.356474 (+ 12096us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.389614 (+ 33140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":442765,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":13111,"raft.run_cpu_time_us":843,"raft.run_wall_time_us":65815,"thread_start_us":1175140,"threads_started":2,"wal-append.queue_time_us":492249}
I20260709 13:00:32.447857 20089 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:32.455762 22002 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:32.455503 22172 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.456740 22002 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.468792 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:32.474020 20011 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.475366 20011 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:32.399186 22014 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.425570 22165 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:32.477649 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 383) took 1450 ms. Trace:
I20260709 13:00:32.477869 22171 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.479501 20164 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:32.478101 20156 rpcz_store.cc:276] 0709 13:00:31.027476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.162179 (+134703us) service_pool.cc:224] Handling call
0709 13:00:31.162548 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:31.221514 (+ 58966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.223039 (+  1525us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:31.308846 (+ 85807us) log.cc:844] Serialized 52 byte log entry
0709 13:00:31.889515 (+580669us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:31.889623 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.889751 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:32.477281 (+587530us) raft_consensus.cc:1685] finished
0709 13:00:32.477300 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.477591 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9442,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":67029,"thread_start_us":579903,"threads_started":1,"wal-append.queue_time_us":1098023}
I20260709 13:00:32.500181 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:32.501259 22002 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.501992 22002 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:32.469038 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:32.478778 22171 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:32.498447 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 64108.5us
I20260709 13:00:32.471655 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:32.503732 22113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.043s	sys 0.078s
I20260709 13:00:32.543965 20090 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:32.544471 22172 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:32.547065 22166 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 13:00:32.548381 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 117428us
I20260709 13:00:32.553411 21854 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.555018 22149 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.555691 22149 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:32.556231 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:32.556447 22149 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.563441 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:32.441227 20157 rpcz_store.cc:276] 0709 13:00:30.926746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.950240 (+ 23494us) service_pool.cc:224] Handling call
0709 13:00:31.188430 (+238190us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:31.188606 (+   176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.188801 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:31.522214 (+333413us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.440061 (+917847us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.440207 (+   146us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:32.440358 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.440473 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.440718 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58207,"prepare.queue_time_us":1037711,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":23986,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"spinlock_wait_cycles":10421632,"thread_start_us":1155697,"threads_started":2}
I20260709 13:00:32.502014 22042 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.557380 20084 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:32.568607 20088 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:32.569502 20082 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:32.579380 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:32.582494 20089 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:32.561461 21851 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:32.583647 22002 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.556720 22014 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.618940 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 735) took 1549 ms. Trace:
I20260709 13:00:32.611706 22002 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.619607 20856 rpcz_store.cc:276] 0709 13:00:31.069271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.071089 (+  1818us) service_pool.cc:224] Handling call
0709 13:00:31.071420 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.071764 (+1000344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.618505 (+546741us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b5400004bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:32.618827 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181821184}
W20260709 13:00:32.627343 22180 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 13:00:32.627740 20163 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:32.592087 22171 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.527212 22167 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.579638 22082 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.596498 22149 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.596882 20086 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:32.579293 22133 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:32.634317 22117 consensus_meta.cc:287] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.062s	user 0.022s	sys 0.004s
W20260709 13:00:32.602028 20165 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33788 (request call id 372) took 1721 ms (client timeout 1939 ms). Trace:
I20260709 13:00:32.602959 20156 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:32.603075 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:32.579833 22042 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.638125 22042 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:32.635604 20165 rpcz_store.cc:269] 0709 13:00:30.880938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.886478 (+  5540us) service_pool.cc:224] Handling call
0709 13:00:32.601640 (+1715162us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400244d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:32.601827 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334506752}
I20260709 13:00:32.640391 21854 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:32.640911 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:32.642131 22095 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.642735 22095 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.643119 22095 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.643404 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:32.644760 20086 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:32.652463 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 54208.4us
I20260709 13:00:32.653853 20011 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:32.662390 22115 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.663306 22115 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.678781 22202 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.685448 22202 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.694671 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 734) took 1623 ms. Trace:
I20260709 13:00:32.696348 20858 rpcz_store.cc:276] 0709 13:00:31.068795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.069446 (+   651us) service_pool.cc:224] Handling call
0709 13:00:32.679320 (+1609874us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54000f46d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:32.692190 (+ 12870us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505359744}
I20260709 13:00:32.688917 22040 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:32.702445 21850 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.708794 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:32.710022 20859 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:32.711786 22185 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
I20260709 13:00:32.712064 22117 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.712270 22202 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:32.713074 22149 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.713507 20850 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:32.713765 22149 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.714692 22159 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.719372 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:32.715485 22159 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.712805 22117 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:32.723287 20082 raft_consensus.cc:1275] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:00:32.723526 22040 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.725294 21850 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:32.701325 22042 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.729328 20082 pending_rounds.cc:85] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 1
I20260709 13:00:32.733244 20082 pending_rounds.cc:107] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:00:32.733495 20082 raft_consensus.cc:2889] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:32.727813 22115 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:32.699784 22014 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.736924 20084 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:32.742501 20090 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:32.742151 22042 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:32.753533 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:32.758607 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:32.781080 22160 log.cc:927] Time spent T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.386s	user 0.000s	sys 0.002s
W20260709 13:00:32.786944 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 308330us
I20260709 13:00:32.786255 22120 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:32.783227 22040 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:32.791893 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 151183us
I20260709 13:00:32.789561 22149 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:32.840353 22190 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
W20260709 13:00:32.844169 22176 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.093s	user 0.000s	sys 0.004s
W20260709 13:00:32.863658 22186 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:00:32.790096 22014 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:32.898437 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.902136 21964 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:32.902912 22204 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:32.903033 21964 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.904161 22082 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.903889 22204 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.904735 22082 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.905956 21964 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:32.906332 22181 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.228s	user 0.001s	sys 0.001s
I20260709 13:00:32.906883 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 390) took 1579 ms. Trace:
I20260709 13:00:32.907616 20160 rpcz_store.cc:276] 0709 13:00:31.327474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.328106 (+   632us) service_pool.cc:224] Handling call
0709 13:00:31.724303 (+396197us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:31.724995 (+   692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.725951 (+   956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.726023 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:31.727075 (+  1052us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.231111 (+504036us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:32.231180 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.231283 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:32.906329 (+675046us) raft_consensus.cc:1685] finished
0709 13:00:32.906355 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.906610 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59331,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":391,"spinlock_wait_cycles":38360832,"thread_start_us":473294,"threads_started":1,"wal-append.queue_time_us":907628}
I20260709 13:00:32.911947 22202 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:32.912946 22202 consensus_queue.cc:1243] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:32.914042 22202 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:32.914829 22202 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:32.907038 22082 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:32.916815 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.912957 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 316) took 1590 ms. Trace:
I20260709 13:00:32.920372 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:32.915958 22002 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.834734 22120 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:32.921418 21850 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:32.928274 22204 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.916365 22083 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:32.928812 22200 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.934553 22082 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:32.974556 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.925637 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:32.930328 22002 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.985469 21796 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:32.986354 21796 consensus_queue.cc:1243] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:32.986851 21796 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:32.987375 21796 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:32.922263 20014 rpcz_store.cc:276] 0709 13:00:31.322876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.325841 (+  2965us) service_pool.cc:224] Handling call
0709 13:00:31.326202 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.326622 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:00:31.326654 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:31.327139 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.402734 (+1075595us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:32.402852 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.402991 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:32.912609 (+509618us) raft_consensus.cc:1685] finished
0709 13:00:32.912628 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.912869 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":977,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":72833,"thread_start_us":1022696,"threads_started":1,"wal-append.queue_time_us":1308647}
W20260709 13:00:32.991429 20158 consensus_meta.cc:287] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.549s	user 0.014s	sys 0.000s
I20260709 13:00:32.991755 20158 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:32.930193 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:32.989089 22204 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.999588 21964 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.001394 22082 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:32.987890 22002 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.999872 22040 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:33.009730 22120 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:32.994270 22083 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:32.989650 22159 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:32.992980 22094 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:33.014449 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 311426us
W20260709 13:00:33.015547 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 109725us
I20260709 13:00:32.993959 22167 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:33.019738 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 159884us
W20260709 13:00:33.009792 22205 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:00:33.017639 20086 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:32.930075 22200 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.021214 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:33.016278 22117 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.023630 22040 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.040314 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.041245 20017 raft_consensus.cc:2368] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:33.041596 22083 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.042332 22083 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:33.042503 22040 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.025125 22082 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.013855 21856 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:32.988564 21796 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.020503 22204 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.031059 22115 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:33.000519 22202 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.032951 22200 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.035302 22002 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:33.019583 22094 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.036187 22149 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:33.018064 21964 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.037174 21851 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.037789 22113 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.038987 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.066692 20011 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:33.039052 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.048559 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.068550 22113 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.057955 21796 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:33.057895 22002 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.058387 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.071444 22113 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:33.071560 21972 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.049181 21856 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.056530 22115 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:33.073622 20016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 665) took 1889 ms (client timeout 1670 ms). Trace:
I20260709 13:00:33.073870 22115 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:33.074235 20016 rpcz_store.cc:269] 0709 13:00:31.183807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.193607 (+  9800us) service_pool.cc:224] Handling call
0709 13:00:33.073019 (+1879412us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540097a254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:33.073202 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279175424}
I20260709 13:00:33.072345 21851 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.064034 22202 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:33.079746 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:33.060855 20084 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:33.036298 22117 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.072741 21856 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:33.060348 22149 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.061762 22094 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.090328 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.071647 22002 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.094310 22149 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.082741 22042 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.097095 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:33.089682 22014 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.101954 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.102595 20858 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:33.097719 22171 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:33.112035 22118 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:33.112226 22191 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.691s	user 0.001s	sys 0.012s
I20260709 13:00:33.112792 22118 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.100759 22117 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.115185 22118 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.100441 22002 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:33.110733 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.119727 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.122066 22133 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.122530 22117 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.122670 21964 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.122915 21850 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.122699 22133 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.124483 21851 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:33.124353 21850 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.125420 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.126005 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.126180 20850 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:33.126683 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.127027 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:33.127225 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.127338 20851 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:33.128296 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.129163 21850 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.130096 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.130259 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:33.131147 20853 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:32.279433 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:33.132529 20010 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:33.136520 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.137465 20165 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:33.138876 22052 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.139519 22052 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:33.140144 21881 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:33.140834 21881 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:33.142342 21854 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.143186 21881 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.143003 21854 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.144546 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.142350 22052 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.145390 20087 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:33.146965 20160 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:33.147506 20088 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:33.145569 21854 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.148237 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:33.149799 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.150570 20857 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:33.150642 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.151345 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.154685 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:33.155431 20164 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.156072 21850 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:33.157088 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.156282 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.158497 20156 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
W20260709 13:00:33.168260 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47711.9us
I20260709 13:00:33.168707 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:33.176656 20084 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:33.178359 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:33.117988 22002 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:33.183156 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:33.125479 22204 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:33.182533 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.143831 21881 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:33.185478 20851 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:33.186133 22107 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:33.186941 22178 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.871s	user 0.002s	sys 0.025s
W20260709 13:00:33.186795 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 103753us
I20260709 13:00:33.182073 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:33.185258 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88259.3us
I20260709 13:00:33.189352 22200 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.194146 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.194413 22200 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.185211 22171 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:00:33.196012 21854 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:33.196954 21854 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:33.185017 22002 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:33.198053 21881 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:33.197968 21854 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.199541 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:33.200397 21833 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.198438 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.202368 21881 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.196979 22200 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:33.211066 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 46414.3us
I20260709 13:00:33.214250 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:33.194207 22193 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.765s	user 0.010s	sys 0.007s
W20260709 13:00:33.220408 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.210208 22204 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:00:33.217726 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:33.210292 22117 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:33.232203 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.233078 20163 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:33.229492 21849 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:33.236361 21849 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.236758 21849 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:33.238091 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.241117 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.242406 21851 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:33.242802 21851 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.242913 19982 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.243198 21851 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:33.247221 21972 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:33.247761 22149 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.248205 21972 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:33.248641 21972 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:33.248391 22149 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.249127 22133 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.223232 21801 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:33.250697 22149 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:33.254608 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 50815.4us
I20260709 13:00:33.255321 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 659) took 2076 ms. Trace:
I20260709 13:00:33.203440 22107 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.256093 22118 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:33.236519 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.257912 22117 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:33.258126 22107 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.258908 20159 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:33.259150 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.255807 20013 rpcz_store.cc:276] 0709 13:00:31.178813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.178971 (+   158us) service_pool.cc:224] Handling call
0709 13:00:31.179433 (+   462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:31.179949 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.179987 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:31.180814 (+   827us) log.cc:844] Serialized 52 byte log entry
0709 13:00:32.277210 (+1096396us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:32.327319 (+ 50109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.351254 (+ 23935us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.054983 (+703729us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:33.254885 (+199902us) raft_consensus.cc:1685] finished
0709 13:00:33.254907 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.255259 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22396,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":10734,"spinlock_wait_cycles":16550144,"thread_start_us":733218,"threads_started":1,"wal-append.queue_time_us":1068912}
I20260709 13:00:33.229781 22042 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:33.262424 22042 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:33.259805 20156 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:33.263586 22042 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.266554 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.267230 20158 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
W20260709 13:00:33.230813 22095 consensus_meta.cc:287] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.587s	user 0.018s	sys 0.000s
I20260709 13:00:33.268451 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.269201 20162 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:33.268486 22095 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.270149 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.271277 20089 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:33.271696 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:33.259298 22117 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:33.273171 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62519.6us
I20260709 13:00:33.274694 22167 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:33.274904 22095 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:33.275389 22167 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:33.276628 22167 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.278615 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.279349 20162 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:33.283742 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.283648 20164 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:33.285279 20160 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:33.257941 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:33.300827 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.301524 20160 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:33.302495 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.305244 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:33.306589 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.306780 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.298190 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.314734 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.315228 20130 leader_election.cc:400] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:33.315616 20130 leader_election.cc:403] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:33.315691 20156 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:33.316414 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.316828 20130 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.317142 20157 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:33.297749 21854 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:33.316923 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.315529 20015 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:33.321090 20013 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:33.321883 21849 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.322983 21856 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.324651 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.325341 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.325763 19982 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.326017 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:33.326349 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 68547.9us
W20260709 13:00:33.363121 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66550.5us
I20260709 13:00:33.326663 22117 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:33.326949 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.327353 21972 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.327519 22002 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.329137 21964 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:33.332605 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.334101 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.336138 22115 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:33.337226 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.339437 22208 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:33.345870 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.373296 20159 raft_consensus.cc:2410] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:00:33.346169 21851 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:33.362434 20014 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:33.366410 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.323807 21849 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.370182 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:33.364971 22117 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.371909 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:33.371636 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45953.5us
I20260709 13:00:33.376840 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:33.380048 20008 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:33.378494 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:33.380960 20016 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:33.371716 20010 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:33.380215 20858 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:33.379546 20013 raft_consensus.cc:2393] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:33.382248 19981 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:33.368222 20163 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:33.384989 20854 raft_consensus.cc:2368] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9.
I20260709 13:00:33.380291 19982 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.385667 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.387475 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.388685 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.390470 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.372648 22208 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:33.394448 20015 raft_consensus.cc:2368] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
I20260709 13:00:33.378587 22204 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:33.400148 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.381091 22117 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:33.402132 20008 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:33.407588 20856 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:33.407946 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.408200 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.407612 20857 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:33.410343 20857 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:33.414655 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.416208 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.415835 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:33.420210 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.426290 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.426875 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.371721 22094 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:33.385119 22118 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:33.427264 20158 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:33.392488 22149 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:33.446079 22115 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:33.424947 19835 catalog_manager.cc:5697] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:33.430466 22002 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.428730 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.445778 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.452831 20014 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:33.397181 22208 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:33.454800 20165 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:33.451649 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:33.459102 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.460585 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:33.460943 20010 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:33.462566 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.463289 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:33.468307 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 52661.8us
I20260709 13:00:33.468633 20082 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:33.464501 20087 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:33.464807 21856 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:33.464128 19835 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 5 to 6, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:33.472776 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.474344 20088 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:33.475348 20016 raft_consensus.cc:2410] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:00:33.476423 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 94268.6us
I20260709 13:00:33.479969 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.433152 21849 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.469070 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.480432 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.484064 19835 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 4 to 6, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:33.485253 20089 raft_consensus.cc:2393] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:33.487126 19981 leader_election.cc:400] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9.
I20260709 13:00:33.447175 22118 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:33.464874 22204 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:33.512126 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 111748us
I20260709 13:00:33.511549 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.508492 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:33.525861 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 56864.5us
I20260709 13:00:33.488464 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:33.508574 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.552127 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.553064 20084 raft_consensus.cc:2368] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:33.553342 20082 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:33.554167 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.470391 22002 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:33.561771 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:33.585078 20090 raft_consensus.cc:2368] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:33.557986 20852 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
W20260709 13:00:33.587986 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 100950us
I20260709 13:00:33.588368 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.592060 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:33.508365 22118 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:33.605237 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:33.589277 22179 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:00:33.611177 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.610445 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.585564 22002 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.621551 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.596544 22149 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:33.565002 21849 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.652107 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.609469 21796 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:33.603935 21856 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.655622 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:33.656199 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51781.8us
I20260709 13:00:33.653077 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.657161 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.632784 22204 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:33.613999 21851 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.646301 22002 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:33.618433 22014 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:33.680346 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.612620 22213 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:33.677211 20852 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:33.678246 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.668872 19835 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 4 to 6, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:33.681782 19839 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32772 (request call id 150) took 9903 ms. Trace:
I20260709 13:00:33.682541 20088 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:33.687660 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.688269 20859 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:33.659518 21972 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:33.659791 21796 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.684849 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.700047 20089 raft_consensus.cc:2368] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:33.690258 22223 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:33.690634 22149 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:33.704002 22223 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.691460 20855 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:33.681864 21856 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:33.707794 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.692694 20083 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:33.693993 21851 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.694671 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.696981 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.687340 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.702200 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.703243 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:33.691226 22219 log.cc:927] Time spent T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:00:33.705989 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.708847 20090 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:33.682170 22124 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.689234 22207 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:33.755400 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.907121 20089 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:33.887025 20859 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:33.707713 22223 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:33.689510 22014 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.704694 21796 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:33.763883 22225 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:00:33.765473 21964 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:33.768478 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 79034.9us
I20260709 13:00:33.768739 22113 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:33.684733 19839 rpcz_store.cc:276] 0709 13:00:23.644191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.654360 (+ 10169us) service_pool.cc:224] Handling call
0709 13:00:33.479377 (+9825017us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:30.145091 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:30.279812 (+134721us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:30.286538 (+  6726us) write_op.cc:435] Acquired schema lock
0709 13:00:30.292878 (+  6340us) tablet.cc:662] Decoding operations
0709 13:00:30.388048 (+ 95170us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:30.388123 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:30.388155 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:00:30.388518 (+   363us) tablet.cc:701] Row locks acquired
0709 13:00:30.388530 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:00:30.401427 (+ 12897us) write_op.cc:270] Start()
0709 13:00:30.406965 (+  5538us) write_op.cc:276] Timestamp: P: 1783602030396419 usec, L: 0
0709 13:00:30.412335 (+  5370us) op_driver.cc:348] REPLICATION: starting
0709 13:00:30.437766 (+ 25431us) log.cc:844] Serialized 1307 byte log entry
0709 13:00:30.753557 (+315791us) op_driver.cc:464] REPLICATION: finished
0709 13:00:31.428903 (+675346us) write_op.cc:301] APPLY: starting
0709 13:00:31.437460 (+  8557us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:31.481868 (+ 44408us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:31.496377 (+ 14509us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:31.583279 (+ 86902us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:31.594155 (+ 10876us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:00:31.596853 (+  2698us) write_op.cc:312] APPLY: finished
0709 13:00:31.613459 (+ 16606us) log.cc:844] Serialized 34 byte log entry
0709 13:00:31.613879 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:31.614014 (+   135us) write_op.cc:454] Released schema lock
0709 13:00:31.614079 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:31.614661 (+   582us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:31.614703 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":344561,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":655537,"apply.run_cpu_time_us":4232,"apply.run_wall_time_us":192617,"num_ops":3,"prepare.queue_time_us":216847,"prepare.run_cpu_time_us":15027,"prepare.run_wall_time_us":437453,"raft.queue_time_us":69424,"raft.run_cpu_time_us":23806,"raft.run_wall_time_us":533018,"replication_time_us":272406,"thread_start_us":540175,"threads_started":4,"wal-append.queue_time_us":133076,"wal-append.run_cpu_time_us":18264,"wal-append.run_wall_time_us":2022002}]]}
I20260709 13:00:33.770249 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:33.772212 21972 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.763950 22229 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:00:33.782318 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:33.794631 21849 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:33.710731 22115 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.699443 22204 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:33.796357 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.797145 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.801580 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:33.958714 20856 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:33.799650 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:33.823523 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:33.867883 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 105438us
W20260709 13:00:33.885422 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:33.887001 20858 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.002s
I20260709 13:00:33.962813 20858 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:33.676381 22227 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:33.892657 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 189515us
I20260709 13:00:33.904495 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:33.909343 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.998742 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:33.906155 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 300927us
I20260709 13:00:33.927650 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.927752 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:33.939272 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.942957 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.946878 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.951483 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:33.910817 22207 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.908136 22213 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.031625 22213 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:34.008627 20091 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:34.033699 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:33.897485 21851 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.034777 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.011440 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:33.909940 22052 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:00:34.036235 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:34.066541 20089 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:34.040992 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.009974 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:34.009420 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:34.010005 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 105639us
I20260709 13:00:34.054275 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.010881 20090 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:33.957100 21972 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.073479 20856 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:34.021879 22224 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:33.910562 22124 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:34.021726 22115 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:33.996503 22113 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.023074 22204 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:33.958082 20085 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
W20260709 13:00:34.025497 22117 consensus_meta.cc:287] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.002s
I20260709 13:00:33.953717 22167 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:34.028352 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:34.030023 22094 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:34.030699 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:34.031136 22207 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:34.048993 20851 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:33.947865 21964 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:34.044644 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:34.116496 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.116865 20088 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:34.069298 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.052662 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.040445 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.016533 21856 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.036942 21851 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.068742 20857 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
W20260709 13:00:34.049345 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.967s (SENT) [suppressed 5 similar messages]
I20260709 13:00:34.016610 22014 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.095818 22124 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:34.098097 22115 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:34.084859 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 73479us
I20260709 13:00:34.099741 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.037434 20087 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:34.132915 20853 raft_consensus.cc:2368] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
W20260709 13:00:34.037585 22231 log.cc:927] Time spent T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:00:34.117450 20858 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.117904 20165 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:00:34.117878 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 48732.2us
I20260709 13:00:34.143002 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.121229 21972 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.049162 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.086140 22224 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.104373 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:34.077144 22194 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.101014 21849 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:34.053339 22229 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:00:34.149818 20160 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:34.148705 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.152611 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.156352 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.157984 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.154356 22095 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:34.117110 22167 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:34.163169 22167 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.108505 21850 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:00:34.164163 20082 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:34.164301 22235 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:34.165298 22235 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:34.165735 22235 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.165552 22233 consensus_queue.cc:1059] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:34.166664 22233 raft_consensus.cc:3055] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:34.167003 22233 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:34.167634 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.168313 21849 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:34.169305 21849 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.149008 22014 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:34.172552 22014 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:34.174608 22014 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.175079 21796 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.156925 20853 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:34.180949 20055 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:34.182760 21903 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.183300 21903 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.183712 21903 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.185377 22203 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:34.141322 21851 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:34.187764 22113 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.194619 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.197523 20858 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
W20260709 13:00:34.200724 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 243292us
I20260709 13:00:34.194839 20165 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:34.116851 22117 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.210144 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:34.210851 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 55441.9us
I20260709 13:00:34.148746 22224 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:34.224545 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 190909us
W20260709 13:00:34.208914 22118 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.004s
I20260709 13:00:34.192337 22052 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:34.134440 22222 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:00:34.230923 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:34.242305 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:34.202847 22203 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.242810 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.207796 22227 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:34.264930 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 218608us
I20260709 13:00:34.266458 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:34.271227 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.224426 21851 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:34.243821 22208 consensus_meta.cc:287] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.005s
I20260709 13:00:34.296018 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.300334 20160 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:34.234217 22171 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:34.167773 22233 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.309582 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:34.311760 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 69061.4us
W20260709 13:00:34.294449 22002 consensus_meta.cc:287] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.605s	user 0.014s	sys 0.002s
I20260709 13:00:34.275988 22094 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.301412 22194 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:34.293885 22243 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:00:34.308718 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:34.279840 22234 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:34.247447 22118 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.268322 22014 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:00:34.331022 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:34.331571 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 329057us
I20260709 13:00:34.294090 22208 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.299556 22213 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.355700 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:34.325376 22094 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:34.357676 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:34.338340 22171 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:34.348482 22242 log.cc:927] Time spent T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:00:34.343793 22240 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.376253 22240 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.378485 22240 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.380406 20130 leader_election.cc:400] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
I20260709 13:00:34.380800 20130 leader_election.cc:403] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:34.383424 22238 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
I20260709 13:00:34.344309 22234 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.384362 22234 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:34.363279 22233 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.359783 20744 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:34.390465 20157 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:00:34.367355 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 318192us
I20260709 13:00:34.370908 20161 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:34.372107 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:34.373637 21881 consensus_queue.cc:1059] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:34.353493 22203 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:34.329021 22002 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.368474 22094 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:34.400749 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45153us
I20260709 13:00:34.398515 19981 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:34.440737 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 140760us
I20260709 13:00:34.442529 20129 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:34.443699 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:34.457345 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 58943.8us
I20260709 13:00:34.460104 19981 leader_election.cc:400] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:34.460832 19981 leader_election.cc:403] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:34.388811 21964 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.399372 22115 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.389315 22117 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.388651 21796 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.473832 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.366255 21856 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:00:34.413357 22240 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:34.389147 22235 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.487672 22235 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.492623 21881 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:34.493224 21881 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:34.496984 22238 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:34.497689 21903 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:34.498551 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 774) took 1009 ms. Trace:
I20260709 13:00:34.498977 20161 rpcz_store.cc:276] 0709 13:00:33.489493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.489649 (+   156us) service_pool.cc:224] Handling call
0709 13:00:33.489935 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.399569 (+909634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.498317 (+ 98748us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b540001a9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:34.498492 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025615744}
I20260709 13:00:34.478367 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.500855 20013 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:34.440891 22014 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:34.479219 20016 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:34.475894 22115 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:34.503485 22115 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.406925 22203 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:34.505630 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.514796 20160 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:34.518030 20055 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:34.467293 22204 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:34.387840 22223 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.527901 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:34.372653 21851 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.507475 22203 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:34.497723 22236 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:00:34.561173 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:34.562299 20856 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.516444 21903 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.536566 22243 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:34.414880 22246 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.150s
W20260709 13:00:34.574074 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.519961 22118 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.577620 22208 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.391187 22227 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.593546 22223 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:34.537053 22167 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:34.576375 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.530807 22240 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:34.621676 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.506263 22207 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.566684 22202 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:34.639598 20016 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:34.598793 20856 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:34.641539 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.646338 20014 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.642490 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.647114 20015 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:34.646060 20055 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:34.595692 22213 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.650451 20161 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:34.652488 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:34.591701 22115 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:34.569649 22224 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.664630 22115 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:34.661396 22235 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:34.666637 22235 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:34.667187 22235 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:34.668884 21903 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:34.669618 21903 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:34.672055 22208 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:34.673451 22208 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:34.673815 22208 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:34.649302 22167 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:34.679457 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 279337us
I20260709 13:00:34.670792 21903 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:34.682091 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 222176us
I20260709 13:00:34.680004 22124 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.689504 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:34.707935 20744 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:34.731681 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.723280 21796 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.746071 21796 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.715736 22229 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:34.749449 22229 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.749938 22229 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.752689 20013 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:34.758523 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.759176 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.802608 20017 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:34.793489 20160 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:34.792425 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.806603 20158 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:00:34.812068 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.812861 20011 raft_consensus.cc:2368] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5.
W20260709 13:00:34.811997 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 148653us
W20260709 13:00:34.807658 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 161722us
I20260709 13:00:34.816452 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.722471 22149 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:34.819810 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:34.820469 20165 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:34.806672 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:34.769888 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.824894 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.825906 20011 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:34.752653 22213 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:34.827802 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 331704us
I20260709 13:00:34.829813 20162 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:34.830197 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.754477 22207 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.773403 22002 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:34.834892 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 95923us
I20260709 13:00:34.808238 22252 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:34.836081 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.840185 20008 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:34.759881 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.830026 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:34.825274 22258 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:00:34.841946 20853 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:34.717382 22124 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.842370 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.843116 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.751632 22234 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.830965 22213 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:34.874564 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47877.6us
I20260709 13:00:34.838567 22202 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:34.875720 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.875952 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.878021 20008 raft_consensus.cc:2368] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7.
I20260709 13:00:34.882006 20013 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:34.883419 22213 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.756376 22167 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.888904 20158 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:34.839908 22262 raft_consensus.cc:2764] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:00:34.902944 20165 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:34.778292 22240 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.906868 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.821317 22208 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:34.909200 19982 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:34.932714 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:34.934218 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:34.909783 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.836982 22203 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.880605 22207 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:34.912935 22094 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:34.914527 22202 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:34.849157 22265 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:00:34.917412 22124 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:34.918448 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:34.919220 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:34.950387 20082 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:34.950549 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.951848 20850 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:34.919852 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:34.922520 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 680) took 1447 ms. Trace:
I20260709 13:00:34.906623 22095 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:00:34.925905 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:34.926781 22236 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:34.928531 20852 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:34.929399 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.930822 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:34.974150 20859 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:34.975612 20858 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:34.846894 22246 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:00:34.942292 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.943286 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:34.945362 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:34.954483 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.938766 22223 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:34.972934 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:34.958137 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:34.849983 22252 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:34.998585 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 108930us
W20260709 13:00:35.027863 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54032.1us
I20260709 13:00:34.981312 20012 raft_consensus.cc:2376] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7. Re-sending same reply.
I20260709 13:00:34.999356 21851 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:34.971633 22207 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:34.972204 21964 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.028715 20743 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.027778 22223 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:35.059474 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.031642 21796 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.058496 20090 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:35.042352 21851 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:35.045794 22204 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.029259 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.002841 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.031229 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:35.031989 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.032228 20011 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:34.960872 20010 rpcz_store.cc:276] 0709 13:00:33.475367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.485565 (+ 10198us) service_pool.cc:224] Handling call
0709 13:00:33.485946 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.099098 (+613152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.099132 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.412621 (+313489us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:34.509148 (+ 96527us) log.cc:844] Serialized 26 byte log entry
0709 13:00:34.719124 (+209976us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:34.719170 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.719363 (+   193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.922219 (+202856us) raft_consensus.cc:1685] finished
0709 13:00:34.922237 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.922458 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116144,"raft.queue_time_us":201101,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"spinlock_wait_cycles":5135744,"thread_start_us":632975,"threads_started":2,"wal-append.queue_time_us":355082}
I20260709 13:00:34.997447 22002 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:35.041040 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.034008 20085 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:34.999142 21972 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.028184 22248 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:34.981601 22224 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:34.998778 20856 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:35.075865 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 43965.6us
I20260709 13:00:35.077524 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.078506 22124 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.079902 22124 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:35.034715 22270 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:00:35.074406 20089 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.060945 22265 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:35.082597 22118 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.087749 22223 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:35.088696 22203 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.111693 20013 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:35.111776 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 83141.4us
I20260709 13:00:35.057428 21964 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:35.114105 20089 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:35.117831 20856 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:35.076964 22248 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:35.112028 22234 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:35.129954 20014 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:35.117655 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:35.133445 20743 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.109016 22224 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:35.135697 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 58245.6us
I20260709 13:00:35.143560 20015 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:35.101011 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.073159 22204 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:35.147639 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 79215.2us
I20260709 13:00:35.148979 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.081488 21972 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:35.154385 22248 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:35.149701 22275 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.155771 21972 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:35.160199 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:35.148734 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:35.152192 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:35.206672 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 54571.1us
I20260709 13:00:35.207887 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.145733 22262 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:35.239375 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 94231.5us
I20260709 13:00:35.082571 21851 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:35.203186 22002 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:35.212477 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.156955 22234 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:00:35.155822 22248 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.113420 22265 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.212045 22118 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:35.276427 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:35.210677 22262 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:35.330884 20017 consensus_meta.cc:287] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.000s
I20260709 13:00:35.239308 21972 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.202567 22236 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:35.241569 22274 raft_consensus.cc:2764] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260709 13:00:35.303936 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 92247.1us
W20260709 13:00:35.319998 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 43671.3us
W20260709 13:00:35.246132 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 86025.4us
W20260709 13:00:35.324522 20056 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.043s	sys 0.055s
I20260709 13:00:35.203554 22235 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:35.242317 22280 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:35.333487 20056 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.043s	sys 0.055s
I20260709 13:00:35.331297 20017 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:35.248011 22094 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:35.210677 22281 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.131s
I20260709 13:00:35.334949 21903 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.337425 20013 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:35.338881 22284 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.337417 22204 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:35.342877 22279 log.cc:927] Time spent T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:00:35.255505 22263 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:35.345010 22263 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:35.346177 22263 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:35.346540 21881 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.347553 21881 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:35.340361 20015 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:00:35.341506 21903 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:35.341872 22276 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:00:35.356359 21903 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.357962 22243 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:35.358485 22243 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.332103 21972 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:35.364377 21972 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.375649 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.332782 22262 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:35.382236 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 49514.8us
I20260709 13:00:35.385926 22243 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.327301 21851 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:35.389163 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.391199 22243 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.394333 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:35.395771 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 1.04138e+06us
I20260709 13:00:35.397445 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.332357 22270 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.402573 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.405427 20856 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:35.407203 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.408178 20855 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:35.409531 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:35.413182 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.418138 20854 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:35.403223 21850 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:35.417470 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.404228 22284 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:35.331846 22275 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:00:35.392807 22252 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.412310 22236 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:35.409166 22270 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:35.394479 22115 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:00:35.414278 22094 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:35.415280 22002 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.449177 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.455183 20854 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:35.382964 22280 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:35.451691 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:35.452986 22270 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.436156 22234 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
W20260709 13:00:35.474623 22285 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:00:35.467121 20743 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:35.478368 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.456714 22227 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:35.480132 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 147303us
I20260709 13:00:35.461683 22207 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:35.497172 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:35.497936 20158 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.478986 22281 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:35.503082 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 93630.1us
I20260709 13:00:35.478633 22118 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:35.507467 22207 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.508184 22207 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.510593 22207 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.507586 21903 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:35.495857 20744 connection.cc:256] RPC call timeout handler was delayed by -1.2482825639999646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
W20260709 13:00:35.513585 20744 connection.cc:256] RPC call timeout handler was delayed by -1.1312825639997754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 13:00:35.513881 20744 connection.cc:256] RPC call timeout handler was delayed by -1.1240825639997638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
W20260709 13:00:35.514156 20744 connection.cc:256] RPC call timeout handler was delayed by -1.0739435350005806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29940000000000005s for a response
I20260709 13:00:35.496145 22227 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:35.517203 22263 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.516958 22227 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.507433 21850 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:35.518380 22263 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:35.525158 22280 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:35.529706 21850 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:35.504210 22094 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.506552 22281 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:35.540122 22280 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:35.563869 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49318.9us
I20260709 13:00:35.569799 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.570537 20158 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:35.571306 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.572136 20088 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:35.572606 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.512166 21903 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:35.573916 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.555624 22281 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.579820 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:35.580565 20010 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:35.579820 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.574429 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.581881 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.583297 20088 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:35.580127 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.584442 20087 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 2.
I20260709 13:00:35.584988 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.583166 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.573283 22095 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.590803 20013 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:00:35.588594 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.553294 22094 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:35.583141 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.586675 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.591325 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.591869 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.593681 20015 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:35.593847 20088 raft_consensus.cc:2410] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:35.595641 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.596557 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.597595 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.577972 22235 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:35.599517 22118 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:35.590797 22095 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:35.603442 22094 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:35.604983 22095 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.608690 22262 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:35.611356 22262 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:35.613183 20090 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:00:35.595129 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:35.615614 20083 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:35.573062 22280 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:35.618755 22289 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:00:35.657306 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.661378 20743 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.661388 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:35.591976 22194 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:00:35.623804 22296 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:35.668550 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.669113 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.670639 22115 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:35.683679 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:35.687031 20010 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:35.665174 22118 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.666615 22234 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:35.666489 22275 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.705453 22280 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:35.708349 20017 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.709697 20017 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:35.710238 22229 consensus_meta.cc:287] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.960s	user 0.010s	sys 0.003s
I20260709 13:00:35.710702 22229 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.712018 20016 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.712273 22263 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:35.712971 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:35.713685 20015 raft_consensus.cc:2368] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9.
I20260709 13:00:35.716130 21972 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.719065 20008 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:35.719853 20011 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.723609 22270 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:35.721194 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:35.726914 22213 consensus_meta.cc:287] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.817s	user 0.011s	sys 0.002s
I20260709 13:00:35.727429 22213 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:35.728838 20851 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.730341 22213 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:35.663873 22288 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:00:35.735512 22288 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:35.735929 22288 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:35.735994 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.736815 22288 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.737411 20017 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:35.738251 22288 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.742623 21972 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:35.759217 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 90748.9us
I20260709 13:00:35.762903 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:35.764112 22094 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:35.764510 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.769843 20017 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:35.772094 20013 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
W20260709 13:00:35.774792 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91241.6us
I20260709 13:00:35.776837 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:35.777025 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:35.778067 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:35.779531 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:35.780411 20013 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:35.781317 20008 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:35.786077 20010 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:35.788452 20744 leader_election.cc:400] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7.
I20260709 13:00:35.790040 20744 leader_election.cc:403] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:35.790681 22115 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.792589 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:35.793462 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.687716 22117 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.102s
I20260709 13:00:35.799700 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 332) took 1027 ms. Trace:
I20260709 13:00:35.800078 20009 rpcz_store.cc:276] 0709 13:00:34.771852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.063000 (+291148us) service_pool.cc:224] Handling call
0709 13:00:35.082737 (+ 19737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.381204 (+298467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.381238 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.381277 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:35.381305 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.799518 (+418213us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54009fa6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:35.799559 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.799657 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685920128}
I20260709 13:00:35.797936 20013 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:35.785964 22247 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:35.789098 22213 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:35.806258 20013 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:35.805491 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.805819 20857 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:35.816684 20010 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:35.779023 22094 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:35.796765 22270 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:35.803876 22247 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:35.832412 22247 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:35.835105 22263 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:35.819197 21850 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.812292 22213 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:35.855438 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:35.859637 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61651.3us
I20260709 13:00:35.860656 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.856325 22213 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.866438 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:35.867264 22290 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.885565 20083 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:00:35.892333 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.892627 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:35.893121 20858 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:35.893460 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:35.894170 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:35.894583 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.892333 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:35.894991 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.895071 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.893148 20853 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.896692 22280 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.897249 22280 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:35.898590 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.899209 20859 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:35.899407 22280 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:35.900036 22235 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.900137 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.900791 20859 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:35.900516 22235 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:35.877689 21964 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:35.902621 20856 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:35.908690 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:35.909387 20161 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:00:35.909704 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49135.4us
I20260709 13:00:35.909900 20162 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:35.910348 22229 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:35.910948 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:35.914012 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 322) took 1158 ms. Trace:
I20260709 13:00:35.919039 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:35.920260 20158 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:35.920051 20016 rpcz_store.cc:276] 0709 13:00:34.755553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.755748 (+   195us) service_pool.cc:224] Handling call
0709 13:00:34.756141 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.712383 (+956242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.913752 (+201369us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b5400567f54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:35.913944 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2197870976}
I20260709 13:00:35.920703 20162 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:35.922884 22281 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:35.928560 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:35.928665 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.930411 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:35.931172 20858 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:35.931342 20158 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:35.932045 20744 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:35.936076 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:35.934299 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:35.936784 20158 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:35.939162 20161 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:35.934751 22299 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.344s	user 0.000s	sys 0.009s
W20260709 13:00:35.946746 22286 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.694s	user 0.000s	sys 0.003s
I20260709 13:00:35.947028 22247 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:35.926014 22297 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.951948 20855 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:35.952733 20158 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:35.940536 22094 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:35.942406 22270 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:35.938380 22149 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:35.962136 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:35.998622 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.009249 22149 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:36.009850 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88961.1us
I20260709 13:00:36.010649 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:36.013643 22281 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:36.013942 22270 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:36.015372 20158 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:36.015305 22270 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:36.016764 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:36.020329 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:36.025650 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:36.030066 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:36.030539 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.043984 20130 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:36.046576 20744 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:36.024966 22115 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7.
I20260709 13:00:36.050598 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:36.055326 20056 leader_election.cc:400] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9.
I20260709 13:00:36.062749 20091 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:36.058566 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:36.075716 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:36.082345 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.064247 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:36.076807 20162 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:36.085070 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:36.088498 20165 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:36.085767 20853 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:36.088433 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:36.090426 20089 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:36.093055 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:36.097934 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.098626 20162 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:00:36.099141 20162 pending_rounds.cc:85] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 3
I20260709 13:00:36.099498 20162 pending_rounds.cc:107] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:36.099957 20162 raft_consensus.cc:2889] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:36.101254 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.101784 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:36.102530 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:36.062268 22273 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:36.118829 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.103129 20156 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:36.152350 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:36.160177 22288 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:36.177752 22235 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:36.089219 21856 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:00:36.164268 22316 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:36.084476 21964 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:36.091310 22297 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.100262 22315 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:36.196029 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107051us
I20260709 13:00:36.138226 22281 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
W20260709 13:00:36.131498 22313 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:00:36.198048 20083 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:00:36.199628 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 141119us
I20260709 13:00:36.135752 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:36.084584 22167 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260709 13:00:36.137017 22207 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:36.133121 22247 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:36.249150 22316 consensus_queue.cc:1243] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:36.249599 22208 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:36.254117 22263 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:36.254280 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:36.255216 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:36.276475 20859 raft_consensus.cc:2368] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
I20260709 13:00:36.209022 22094 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:00:36.275633 20853 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:36.244829 22315 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.291033 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 667) took 1476 ms. Trace:
I20260709 13:00:36.247277 21964 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:36.285281 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:36.286728 22275 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:36.284739 21856 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:36.296098 21856 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:36.296299 21850 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.297115 21850 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:36.297812 21850 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:36.298643 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 162999us
I20260709 13:00:36.299998 20853 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:36.301023 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:36.297180 21856 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.302739 22262 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:36.302829 21856 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:36.303469 22262 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:36.304615 22262 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:36.291467 20851 rpcz_store.cc:276] 0709 13:00:34.815153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.815329 (+   176us) service_pool.cc:224] Handling call
0709 13:00:34.830337 (+ 15008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.729353 (+899016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.290784 (+561431us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b54000041d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:36.290966 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065744768}
I20260709 13:00:36.279767 22263 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:36.312155 20853 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:36.315630 22235 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.336571 22235 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:36.339278 22235 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:36.269783 22320 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:00:36.294309 22207 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:36.293617 22315 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:36.328948 21964 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.294850 22203 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:00:36.278667 22115 raft_consensus.cc:2764] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:00:36.345611 22247 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:00:36.352499 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107266us
I20260709 13:00:36.353441 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:36.355432 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46437.7us
I20260709 13:00:36.354028 22235 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.356890 22235 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:36.357506 22320 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:36.358482 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.359335 22235 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:36.358080 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:36.359875 22235 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:00:36.360721 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 50008.7us
I20260709 13:00:36.365875 21856 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:36.367769 22094 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:36.369088 22331 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:36.369516 22331 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:36.370428 22331 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.370986 22235 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:36.371443 22235 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:36.371802 22235 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.374524 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:36.376077 22323 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.376629 22323 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:36.377071 22335 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:36.378952 22323 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:36.360600 22324 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:00:36.394619 22324 raft_consensus.cc:3055] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:36.395054 22324 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:36.395561 22095 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:36.395946 22324 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:36.398566 22324 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.401070 22270 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:36.401706 22270 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:36.402417 21850 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:36.365494 22273 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:36.383059 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:36.387616 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.384045 22207 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:36.408715 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.409986 22326 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:36.410604 22326 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:36.412386 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:36.413758 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 60400.2us
W20260709 13:00:36.415763 20890 heartbeater.cc:646] Failed to heartbeat to 127.12.30.126:46523 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.30.126:46523 timed out after 15.000s (SENT)
I20260709 13:00:36.417400 22280 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.421149 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.381834 22320 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:36.411124 22326 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:36.429837 22323 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:36.403080 22270 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:36.396126 22095 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:36.431603 22320 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.364513 22213 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:36.435139 22208 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:00:36.498485 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 63047.4us
I20260709 13:00:36.503983 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:36.418103 22280 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:36.522897 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.530391 20089 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:36.530930 20089 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:36.531886 20089 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:36.546154 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.552601 22194 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:36.555835 22194 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:36.558733 22194 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:36.559633 22203 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:36.532853 22298 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:36.568452 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 64572.4us
W20260709 13:00:36.570837 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 208505us
I20260709 13:00:36.571672 20859 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.500794 22325 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:36.577560 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 158504us
I20260709 13:00:36.521179 22320 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:36.581264 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 54808.6us
I20260709 13:00:36.579093 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.601881 20055 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:36.601717 22095 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.602557 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:36.602849 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:36.603242 22095 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.603956 22113 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:36.604197 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:36.604579 22113 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:36.605525 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:36.606659 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.607489 20017 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:36.609035 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.609803 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:36.612721 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:36.607407 22113 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:36.603340 20010 raft_consensus.cc:2368] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5.
I20260709 13:00:36.568846 22298 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:36.621209 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:36.576086 22323 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:36.633738 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.636677 21903 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:36.637403 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 968) took 1690 ms. Trace:
I20260709 13:00:36.637820 20011 rpcz_store.cc:276] 0709 13:00:34.947018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.244924 (+297906us) service_pool.cc:224] Handling call
0709 13:00:35.245165 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.723083 (+477918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.636886 (+913803us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54005f2154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:36.637334 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090936576}
I20260709 13:00:36.638234 21903 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:36.639191 22315 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:36.640308 21903 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.641090 22265 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:36.640903 21903 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.641630 22315 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:35.915258 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:36.643647 22324 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:36.644258 22324 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.644644 21903 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:36.646294 22324 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:36.639191 22235 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.633839 20744 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:36.527607 22149 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.618217 22167 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:36.619426 22270 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:36.667169 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.622978 21850 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:36.668576 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:36.688800 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:36.670117 20855 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:36.651067 22235 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:36.671792 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 102255us
I20260709 13:00:36.693041 22298 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:36.693878 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.694578 22298 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:36.635035 22323 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:36.695348 20850 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:36.695811 22323 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:36.696252 22235 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:36.696602 22117 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:36.697022 22117 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.697350 22117 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:36.698932 20014 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:36.633667 22207 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.688140 22274 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:36.708802 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:36.709632 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 109252us
I20260709 13:00:36.704375 20853 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:36.711649 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:36.643222 22315 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:00:36.700539 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:36.668081 22227 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:36.731648 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 97858.6us
I20260709 13:00:36.735325 20850 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:36.738588 20129 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:36.747370 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:36.748335 20853 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.750720 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:36.757157 20850 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:36.757694 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:36.729272 22234 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:36.732590 22227 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:36.753621 20129 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:36.743963 22235 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.770417 20853 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:36.759795 20744 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:36.742923 22274 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:36.757241 22208 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:36.760685 22290 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.804351 22290 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:36.804029 22244 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.105s
W20260709 13:00:36.788247 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 167177us
I20260709 13:00:36.807399 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.808204 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.808841 20744 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.808864 20008 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:36.788734 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:36.754315 22002 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:00:36.790356 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:36.793231 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:36.795230 22334 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:36.797070 22149 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:36.766377 22227 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.815984 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.774394 22234 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:36.667830 22240 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
W20260709 13:00:36.817632 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.819389 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.764104 22118 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:00:36.824116 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:36.816282 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:36.745425 21964 consensus_queue.cc:1059] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:00:36.766222 22235 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:36.829777 20850 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:36.833158 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.846280 20744 leader_election.cc:400] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5.
I20260709 13:00:36.846585 20008 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:36.850687 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.851967 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.823089 22320 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:36.858471 20008 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:36.805639 22244 consensus_queue.cc:1243] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:36.830870 22213 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:36.861155 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.813319 22246 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:00:36.790961 22284 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.059s
W20260709 13:00:36.868292 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 79623.9us
I20260709 13:00:36.868832 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.844298 21972 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:36.849123 22235 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.855481 22208 raft_consensus.cc:3055] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:36.873870 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:36.874986 22229 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:36.873548 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 255212us
I20260709 13:00:36.881819 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:36.885336 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:36.890026 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:36.876449 22275 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.894649 20162 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:36.878378 22208 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:36.900991 22208 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:36.902621 22208 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.884806 22213 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:36.899190 22283 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:36.898892 22234 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.883255 22229 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:36.896521 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:36.872357 22296 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:00:36.909699 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:36.882009 22348 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.911677 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:36.922406 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.933835 22334 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:36.935791 22320 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:36.938191 22290 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:36.939427 22113 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:36.972754 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 78293.3us
I20260709 13:00:36.904448 22284 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.872269 22274 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:36.985703 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:36.984035 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:36.989291 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:36.990809 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:36.995286 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:36.897260 22118 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:00:36.951292 22337 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:36.984443 21856 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:36.905918 21972 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:00:37.000414 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:36.999671 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:37.003409 20090 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:00:37.011444 19838 catalog_manager.cc:5697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:37.013298 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.014731 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.012305 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:37.013547 22229 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:37.015890 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.016271 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.026911 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:37.030267 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:37.002182 22337 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.022362 20091 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:37.038249 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:36.985420 22113 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:37.040330 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.042448 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:36.987504 22284 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:37.044718 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.044801 20857 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:37.047654 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:37.036765 22263 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.000s
I20260709 13:00:37.054239 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.049017 20014 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:37.055860 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:37.058799 20010 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:37.060441 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:37.061249 20159 raft_consensus.cc:2410] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:00:37.062062 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.062346 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:37.067124 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.056473 20160 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:37.068639 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 152286us
I20260709 13:00:37.069367 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:37.040576 22337 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.069849 20158 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:37.069859 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.075191 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:37.076002 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:37.078013 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.078644 20091 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:37.079443 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.080041 20091 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:37.070778 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.080745 20160 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:37.084759 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:37.090617 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:37.091980 22320 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:37.093036 22320 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:37.093616 22350 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:37.094383 22350 consensus_queue.cc:1243] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:37.095957 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:37.096796 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.097985 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.099494 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:37.101513 20158 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:37.103276 20158 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:37.096112 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:37.105584 20013 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:37.112274 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 358701us
I20260709 13:00:37.114028 20129 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:37.120690 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.068425 22343 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:37.077613 22342 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.241s	user 0.002s	sys 0.000s
I20260709 13:00:37.077504 22247 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:37.138907 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:37.144759 20857 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:37.077396 22240 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:37.138494 22334 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:37.134984 22353 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:37.055300 20162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:55614
W20260709 13:00:37.150305 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 44931.4us
I20260709 13:00:37.173228 20162 raft_consensus.cc:606] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:37.173662 20162 raft_consensus.cc:614] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:37.096634 22325 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:37.087422 22095 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:37.167548 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 53590.6us
I20260709 13:00:37.186817 20859 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:37.168876 22369 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:37.069621 22263 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:37.173410 22334 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.187170 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:37.175668 20013 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:37.135707 22236 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.145787 22247 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:37.191051 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 170020us
I20260709 13:00:37.124895 22262 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:00:37.200639 22236 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:37.202852 22247 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:37.203066 22369 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.205804 22369 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:37.206284 22369 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.157087 22280 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:37.208225 22236 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:37.108139 22124 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:00:37.209616 22113 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:37.194307 22334 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:37.161904 22240 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:37.212502 20129 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:37.217010 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:37.231429 20129 leader_election.cc:400] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
I20260709 13:00:37.233110 20129 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:37.235411 20011 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:37.251371 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:37.222081 22298 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.527s	user 0.013s	sys 0.000s
I20260709 13:00:37.208278 22263 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:37.258106 20854 raft_consensus.cc:2410] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:00:37.153013 22366 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:00:37.255802 20009 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:00:37.260033 20009 pending_rounds.cc:85] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 5
I20260709 13:00:37.259703 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:37.260310 20009 pending_rounds.cc:107] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:00:37.260550 20009 raft_consensus.cc:2889] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:37.184904 22353 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:37.191804 22095 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:37.247462 22323 consensus_meta.cc:287] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:00:37.261390 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:37.212168 22357 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:00:37.272161 20850 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:37.279438 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:37.284530 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51425.9us
I20260709 13:00:37.280691 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:37.291229 20011 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:37.298735 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:37.271346 22320 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:37.302966 20859 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:37.274533 22378 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:37.328451 20856 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:37.327965 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.296964 22298 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.290369 22202 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:00:37.282600 22323 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:37.291505 22325 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:00:37.370368 20011 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:37.373067 22383 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:37.373632 22383 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:37.373425 22382 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:37.374069 22383 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.300230 22343 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:00:37.326047 22350 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:00:37.341306 22320 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:37.467242 22331 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.341115 22297 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:00:37.500335 20162 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:37.470449 22357 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:37.382241 22298 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:37.526001 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.494076 22283 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.597082 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:37.507807 22227 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.665s	user 0.013s	sys 0.000s
W20260709 13:00:37.512567 22376 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.229s	user 0.004s	sys 0.006s
I20260709 13:00:37.604413 22365 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:37.604883 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:37.506078 22207 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.003s
W20260709 13:00:37.598640 22117 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.901s	user 0.009s	sys 0.005s
W20260709 13:00:37.690899 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86102.9us
I20260709 13:00:37.541425 22124 consensus_queue.cc:1059] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.177s
I20260709 13:00:37.715343 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:37.780746 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 65476.6us
I20260709 13:00:37.787247 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:37.608508 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 347) took 2029 ms. Trace:
I20260709 13:00:37.712858 22207 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:37.690913 22366 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.713336 22365 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:37.851078 22386 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:00:37.861690 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 79914.2us
I20260709 13:00:37.864377 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:37.782931 22095 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:37.789142 22280 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:37.691438 22227 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:37.760794 22117 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:37.835736 22353 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:37.603967 22357 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:37.898600 22382 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:37.915623 22379 log.cc:927] Time spent T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.227s	user 0.001s	sys 0.001s
W20260709 13:00:37.965574 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 101277us
I20260709 13:00:37.965829 20016 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:37.712791 22283 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:37.972854 20014 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:37.976336 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:37.848841 22366 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:37.968434 22375 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.589s	user 0.002s	sys 0.000s
W20260709 13:00:38.011904 20055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.017s	sys 0.008s
W20260709 13:00:38.012338 20055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.017s	sys 0.008s
W20260709 13:00:38.012619 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 664800us
I20260709 13:00:37.973519 22280 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:37.971779 22323 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:38.029505 20057 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.013s	sys 0.024s
W20260709 13:00:38.030169 20057 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.013s	sys 0.024s
I20260709 13:00:37.978296 22343 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:38.041817 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 71552.9us
W20260709 13:00:38.018208 22369 consensus_meta.cc:287] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.807s	user 0.000s	sys 0.020s
W20260709 13:00:38.043720 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.043778 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:38.050786 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.052668 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.055819 20162 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
W20260709 13:00:38.042577 22350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.024s	sys 0.031s
I20260709 13:00:38.073684 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.075452 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:38.054522 22320 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.059103 22382 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.062207 22323 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:38.077574 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 795) took 1019 ms. Trace:
I20260709 13:00:38.081046 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:38.081979 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:38.081822 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 336928us
W20260709 13:00:38.072503 22341 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 1.362s	user 0.000s	sys 0.016s
I20260709 13:00:38.063864 22298 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.087473 20165 rpcz_store.cc:276] 0709 13:00:37.057971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.058486 (+   515us) service_pool.cc:224] Handling call
0709 13:00:37.058779 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:37.073872 (+ 15093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.073906 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:37.110228 (+ 36322us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.245079 (+134851us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:37.245142 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.245234 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:38.077285 (+832051us) raft_consensus.cc:1685] finished
0709 13:00:38.077316 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.077495 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107986,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":187,"thread_start_us":80834,"threads_started":1,"wal-append.queue_time_us":182506}
W20260709 13:00:38.089187 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.048460 22343 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:38.078624 22270 consensus_meta.cc:287] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.366s	user 0.012s	sys 0.003s
W20260709 13:00:38.070887 22350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.025s	sys 0.031s
I20260709 13:00:38.051116 22369 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.099161 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.100140 20129 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:38.081424 22280 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.104113 20855 raft_consensus.cc:2410] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:38.082924 22365 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.085952 22323 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.068958 22095 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:00:38.104790 22382 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:38.113061 22235 consensus_meta.cc:287] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 1.228s	user 0.013s	sys 0.000s
W20260709 13:00:38.110224 22385 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.252s	user 0.002s	sys 0.009s
I20260709 13:00:38.109935 22389 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:38.068013 22202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.006s	sys 0.022s
I20260709 13:00:38.112277 22298 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.117897 22323 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:38.127126 22326 consensus_meta.cc:287] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 1.608s	user 0.013s	sys 0.005s
W20260709 13:00:38.133745 22387 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
W20260709 13:00:38.155058 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 55016.1us
I20260709 13:00:38.156297 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.157939 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:38.144788 22202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.007s	sys 0.022s
I20260709 13:00:38.116110 22388 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:00:38.133760 22280 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.164702 20089 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:38.154774 22334 consensus_meta.cc:287] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.944s	user 0.011s	sys 0.001s
W20260709 13:00:38.159258 22391 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:00:38.159720 22389 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.154477 22227 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.131313 22235 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.180827 20089 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
W20260709 13:00:38.174170 22208 consensus_meta.cc:287] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 1.271s	user 0.012s	sys 0.003s
I20260709 13:00:38.189625 20089 pending_rounds.cc:85] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 3
I20260709 13:00:38.151115 22298 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.202744 20089 pending_rounds.cc:107] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:38.152810 22326 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.203135 20089 raft_consensus.cc:2889] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:38.206468 20156 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:00:38.207042 20156 pending_rounds.cc:85] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 2
I20260709 13:00:38.207279 20156 pending_rounds.cc:107] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:38.207512 20156 raft_consensus.cc:2889] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:38.183795 22117 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:38.211800 20164 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:38.211953 20089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1103) took 2106 ms. Trace:
I20260709 13:00:38.212745 20089 rpcz_store.cc:276] 0709 13:00:36.100597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.211694 (+111097us) service_pool.cc:224] Handling call
0709 13:00:36.212031 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.530255 (+318224us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b540076d020. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 13:00:38.163893 (+1633638us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b58008288d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:00:38.206228 (+ 42335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.206893 (+   665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197453312}
I20260709 13:00:38.187258 22280 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.220275 20008 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:38.220165 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 370) took 1005 ms. Trace:
I20260709 13:00:38.221593 20088 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:38.211144 22316 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:00:38.230679 22326 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.208992 22357 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:38.232602 22343 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.232626 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 909) took 1158 ms. Trace:
I20260709 13:00:38.233407 20160 rpcz_store.cc:276] 0709 13:00:37.074538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.086605 (+ 12067us) service_pool.cc:224] Handling call
0709 13:00:37.086906 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:37.296936 (+210030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.296972 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:37.329328 (+ 32356us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.519290 (+189962us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:37.519328 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.519393 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:38.232337 (+712944us) raft_consensus.cc:1685] finished
0709 13:00:38.232367 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.232544 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81359,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":383669,"threads_started":2,"wal-append.queue_time_us":495778}
I20260709 13:00:37.826761 20012 rpcz_store.cc:276] 0709 13:00:35.578840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.582364 (+  3524us) service_pool.cc:224] Handling call
0709 13:00:35.615985 (+ 33621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.857097 (+241112us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:35.868517 (+ 11420us) log.cc:844] Serialized 14 byte log entry
0709 13:00:35.943903 (+ 75386us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.411204 (+467301us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800427660. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:36.568079 (+156875us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b1800427660. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:00:36.576612 (+  8533us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.580628 (+  4016us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:36.586634 (+  6006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.514497 (+927863us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b540064f5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:37.538506 (+ 24009us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.602633 (+ 64127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":625370,"prepare.queue_time_us":10295,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":1521,"raft.queue_time_us":637446,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":17856,"spinlock_wait_cycles":553338112,"thread_start_us":57148,"threads_started":1,"wal-append.queue_time_us":221450,"wal-append.run_cpu_time_us":1959,"wal-append.run_wall_time_us":1077175}
W20260709 13:00:38.236907 22247 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.033s	user 0.007s	sys 0.007s
I20260709 13:00:38.237592 22235 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.237440 22247 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.206542 22366 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.175714 22334 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.225981 22408 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:38.205267 22388 consensus_queue.cc:1243] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:38.221060 20013 rpcz_store.cc:276] 0709 13:00:37.214557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.233784 (+ 19227us) service_pool.cc:224] Handling call
0709 13:00:37.234247 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:37.250253 (+ 16006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.250282 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:37.269190 (+ 18908us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.392877 (+123687us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:37.392920 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.393018 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:38.219932 (+826914us) raft_consensus.cc:1685] finished
0709 13:00:38.219954 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.220108 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13618,"raft.queue_time_us":22444,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":23811,"thread_start_us":65397,"threads_started":1,"wal-append.queue_time_us":314326}
W20260709 13:00:38.239768 22337 consensus_meta.cc:287] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 1.140s	user 0.011s	sys 0.003s
W20260709 13:00:38.239993 22383 consensus_meta.cc:287] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.865s	user 0.013s	sys 0.000s
I20260709 13:00:38.246460 22369 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.256309 22383 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.272055 22334 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.273033 22346 raft_consensus.cc:3055] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:38.244451 22408 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.258296 22247 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.274072 22346 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:38.278374 22247 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.278872 22247 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.281903 22247 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.282215 22302 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:38.282588 22388 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.282765 22302 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.283141 22388 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.284363 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:38.285223 20083 raft_consensus.cc:2410] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:38.286257 22323 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.284296 22408 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:38.288388 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:38.289374 20089 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:38.288509 22298 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:38.262440 22384 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.768s	user 0.002s	sys 0.000s
I20260709 13:00:38.293391 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:38.294003 20089 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.294502 20091 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:38.294780 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:38.283044 22346 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.301784 22346 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.301913 22149 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:38.302305 20850 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:38.302420 22149 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:38.303532 22149 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.304555 22149 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.304993 22388 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.305826 22315 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:38.308382 20083 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:00:38.308877 22203 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:38.309332 22203 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:38.309646 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:38.310192 22203 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.311936 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:38.313505 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:38.304636 20086 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:38.315249 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.312772 22394 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:38.319983 22394 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:38.252964 22337 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.334308 20159 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:38.320936 22394 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.337077 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:38.314325 22390 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 13:00:38.323076 21856 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:38.311658 22203 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.336622 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.331072 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.321836 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:38.321779 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.335592 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.320834 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:38.387562 20858 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:38.379132 20850 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:00:38.383281 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46359.4us
I20260709 13:00:38.383783 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.388989 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.319262 20089 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:38.389601 20163 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:38.385924 20162 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:38.386631 20157 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:38.387343 20852 raft_consensus.cc:2376] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6. Re-sending same reply.
I20260709 13:00:38.393532 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 797) took 1333 ms. Trace:
I20260709 13:00:38.394372 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.394413 20157 rpcz_store.cc:276] 0709 13:00:37.059803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.060837 (+  1034us) service_pool.cc:224] Handling call
0709 13:00:38.330940 (+1270103us) spinlock_profiling.cc:243] Waited 988 us on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68deee 0000000000375939 00007fa0337011e0 00007fa0337ca0c4 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:38.393475 (+ 62535us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299520}
I20260709 13:00:38.395161 20160 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:38.397825 20161 raft_consensus.cc:2368] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:38.398766 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:38.390985 22378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.015s	sys 0.020s
W20260709 13:00:38.401715 22378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.016s	sys 0.020s
I20260709 13:00:38.405552 20158 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:38.403321 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.403333 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:38.400216 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:38.407083 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:38.409618 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:38.415709 20008 raft_consensus.cc:2393] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:38.391002 22388 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:38.405284 21856 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.417559 22400 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.417647 21856 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.418325 22400 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.420532 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.421471 20163 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:38.422573 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:38.423664 22411 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:38.423978 22280 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.424396 22411 raft_consensus.cc:697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:38.424598 22280 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.425424 22411 consensus_queue.cc:237] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.427004 22280 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:38.427306 22302 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.428896 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 687) took 1149 ms. Trace:
I20260709 13:00:38.430377 22360 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:38.431845 22323 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:38.433634 22263 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.434311 22167 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:38.436733 22315 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:38.408258 22331 consensus_meta.cc:287] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.907s	user 0.015s	sys 0.000s
I20260709 13:00:38.465883 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 704) took 1408 ms. Trace:
I20260709 13:00:38.405140 22378 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:38.402861 22334 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.407836 22316 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.405438 22208 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:00:38.448004 22167 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:38.448719 22263 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.488832 22316 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.440629 20851 rpcz_store.cc:276] 0709 13:00:37.279741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.279894 (+   153us) service_pool.cc:224] Handling call
0709 13:00:37.294496 (+ 14602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:37.334919 (+ 40423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.334967 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:37.345302 (+ 10335us) log.cc:844] Serialized 26 byte log entry
0709 13:00:37.503093 (+157791us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:37.503140 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.503230 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:38.307451 (+804221us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b240051a688. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b70409d 00007fa02e33610e 00007fa02e34d88c 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221
0709 13:00:38.428575 (+121124us) raft_consensus.cc:1685] finished
0709 13:00:38.428598 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.428820 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39463,"raft.queue_time_us":44819,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":8638,"spinlock_wait_cycles":5621504,"thread_start_us":133245,"threads_started":1,"wal-append.queue_time_us":475675}
I20260709 13:00:38.457283 22388 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:38.499428 22388 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.497759 22208 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.500453 22316 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:38.468351 22337 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.458369 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:38.459018 22095 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:38.460429 22400 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.412611 22369 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:00:38.443178 22360 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:38.466462 20014 rpcz_store.cc:276] 0709 13:00:37.057188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.064310 (+  7122us) service_pool.cc:224] Handling call
0709 13:00:37.086634 (+ 22324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:37.973417 (+886783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.465583 (+492166us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b540064ebd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:38.465813 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037340928}
I20260709 13:00:38.440999 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:38.475666 20853 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:38.476106 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.486936 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 787) took 1398 ms. Trace:
I20260709 13:00:38.489001 22149 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:00:38.488932 22117 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:00:38.397953 22343 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:00:38.473235 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:38.498509 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:38.503252 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45052us
I20260709 13:00:38.514690 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.516160 22400 consensus_queue.cc:1059] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:38.516709 22398 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:38.500589 22208 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.517436 22398 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.518112 22398 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:38.517875 22315 raft_consensus.cc:3055] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:38.518956 22315 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:38.519721 22208 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:38.519907 22315 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.521771 22315 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.501335 22378 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.522433 22263 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.509157 20857 rpcz_store.cc:276] 0709 13:00:37.088739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.088879 (+   140us) service_pool.cc:224] Handling call
0709 13:00:37.119754 (+ 30875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.145300 (+ 25546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.486641 (+1341341us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b54000046d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:38.486867 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56469248}
I20260709 13:00:38.506738 22331 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.520618 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:38.526286 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 53150.1us
I20260709 13:00:38.527335 22149 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.527859 22149 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:38.528293 22149 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.506297 22316 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.528961 22316 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.529354 22316 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.526471 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:38.531807 20085 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:38.532166 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:38.537461 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:38.542428 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 775) took 1874 ms. Trace:
I20260709 13:00:38.542325 22360 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.543025 20854 rpcz_store.cc:276] 0709 13:00:36.667844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.669096 (+  1252us) service_pool.cc:224] Handling call
0709 13:00:38.542213 (+1873117us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b540036ced4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:38.542370 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307094912}
I20260709 13:00:38.543591 22360 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.545325 22113 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:38.543186 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.547955 20159 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:38.539511 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.552521 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:38.555961 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:38.557986 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.558349 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.527335 22400 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.568464 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.570415 22397 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:38.572129 22393 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.572917 22393 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.574077 22227 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.574553 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:38.575350 22411 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.575445 20084 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:38.575546 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:38.576112 22411 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:38.578737 22411 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.580156 22355 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.580823 22355 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.581312 22355 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.583222 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:38.525141 22331 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.576512 20090 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:38.566690 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:38.540747 22095 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.593277 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.584367 22411 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.597277 20857 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:38.599059 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:38.535979 22002 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:00:38.603471 20084 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:38.558982 21851 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.607173 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:38.610077 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.610950 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.567324 22400 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.617046 22400 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.620191 22326 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:00:38.620354 22349 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:38.607468 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:38.621268 22349 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:38.622186 22378 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:38.621963 22149 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.622589 22378 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.622997 22378 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:38.622627 22349 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.625598 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:38.626363 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.629315 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.630898 20857 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:38.595332 22095 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.632261 22095 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.605938 22227 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:38.625418 22117 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:38.636509 22117 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.621992 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:38.644017 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.600303 22113 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:38.653846 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.659901 20854 raft_consensus.cc:2410] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:00:38.699630 19982 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.652863 22325 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.604622 22411 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:38.707326 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 81101us
I20260709 13:00:38.670356 22337 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:38.625845 21851 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:38.714954 22374 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:00:38.715370 22326 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:38.715974 22326 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.718225 22326 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.714751 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:38.720994 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:38.728907 20010 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:38.702248 22149 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.732290 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:38.705610 22325 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.737812 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:38.736191 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:38.744525 20015 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:38.676492 22113 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:38.710709 22343 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.706440 22202 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:38.706669 22350 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:38.732098 22302 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:38.800669 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 242361us
I20260709 13:00:38.791288 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:38.803037 20010 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.747643 22325 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.737071 22337 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:38.789938 22375 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:00:38.818676 22410 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:38.766449 22395 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:38.820425 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:38.820995 22095 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:38.819675 22425 log.cc:927] Time spent T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:00:38.826643 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.732609 22326 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:00:38.853708 20009 raft_consensus.cc:2410] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:00:38.857941 20015 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:00:38.863714 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:38.862426 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:38.864544 20851 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.874765 20856 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:38.890697 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 169780us
I20260709 13:00:38.801028 22202 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.807731 22350 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:38.899108 20010 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:38.902117 20856 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:38.805069 22378 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:38.863199 22410 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:38.893689 22418 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:00:38.824877 22149 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:00:38.879449 22117 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:38.858616 22316 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:38.960850 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 102180us
I20260709 13:00:38.964150 22430 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:38.919601 22410 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:38.970147 22350 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:38.974989 20856 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:00:38.904187 22295 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.985548 22295 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.014959 22316 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:38.985971 22287 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:39.031281 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.012907 22350 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:39.029892 20012 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.032785 22287 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.016011 22295 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:39.029052 21851 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.040014 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:38.931358 22337 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:39.030701 22433 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:00:39.017792 22360 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:00:39.034006 22411 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.036502 22095 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.066280 22392 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:39.072965 22392 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:39.073416 22431 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:39.073875 22392 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.074354 22113 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.075207 22392 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:39.069171 22315 consensus_meta.cc:287] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.003s
I20260709 13:00:39.070263 22283 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:39.034902 22410 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:39.085202 22445 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:00:39.087034 20856 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.074954 22400 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:39.087320 22202 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:39.069574 22325 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:39.089773 22410 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:39.087121 20161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 907) took 2198 ms (client timeout 1916 ms). Trace:
I20260709 13:00:39.091379 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:39.091137 20161 rpcz_store.cc:269] 0709 13:00:36.888946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.889581 (+   635us) service_pool.cc:224] Handling call
0709 13:00:39.086924 (+2197343us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54000bfc54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:39.087054 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3310913280}
I20260709 13:00:39.093410 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 800) took 2032 ms. Trace:
I20260709 13:00:39.093842 20164 rpcz_store.cc:276] 0709 13:00:37.060695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.060810 (+   115us) service_pool.cc:224] Handling call
0709 13:00:37.071703 (+ 10893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.212522 (+1140819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.093189 (+880667us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54004d6554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:39.093350 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2621395328}
I20260709 13:00:39.078073 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.084898 22284 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:39.088819 22378 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:39.095535 22202 consensus_queue.cc:1059] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:39.097011 22202 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.097453 22202 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:39.098310 22202 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:39.099192 22229 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:39.099516 22202 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:39.084420 22437 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:00:39.102232 20851 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:39.099592 22229 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:39.083724 22398 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.001s
I20260709 13:00:39.077814 21851 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:39.106823 21851 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:39.106892 22229 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.107694 22229 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.107667 21851 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.108971 21851 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.121498 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 371) took 2131 ms. Trace:
I20260709 13:00:39.122016 20088 rpcz_store.cc:276] 0709 13:00:36.990136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.009137 (+ 19001us) service_pool.cc:224] Handling call
0709 13:00:37.009579 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.223849 (+1214270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.121064 (+897215us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54005a1254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:39.121394 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2786944256}
W20260709 13:00:39.124171 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.124665 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.126052 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.126493 22398 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:39.127715 22383 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.129462 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.130813 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:39.131246 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.133016 22094 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.133433 22094 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:39.133829 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.134212 22094 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.135219 22094 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.045955 22350 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:39.136543 22284 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:39.135150 22395 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.140061 20084 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:00:39.140625 22095 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:39.141149 22095 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:39.142707 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:39.143285 22095 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.139659 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.147333 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:39.147851 19981 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.086s (SENT) [suppressed 33 similar messages]
I20260709 13:00:39.148484 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 794) took 2090 ms. Trace:
I20260709 13:00:39.148808 22398 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.148938 20156 rpcz_store.cc:276] 0709 13:00:37.057642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.058414 (+   772us) service_pool.cc:224] Handling call
0709 13:00:37.072978 (+ 14564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.210905 (+1137927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.148260 (+937355us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540009b354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:39.148424 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2603394816}
W20260709 13:00:39.126174 22355 consensus_meta.cc:287] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.006s
I20260709 13:00:39.151429 20857 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:39.152074 22398 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.152676 22362 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.153105 22362 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:39.153290 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.153930 22362 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.138520 22350 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.155220 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.150597 22355 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.156304 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.157132 20857 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:39.158157 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:39.159566 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.159725 20091 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:39.159862 20083 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:39.160964 20091 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:39.161036 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:39.161530 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:39.161932 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:39.162883 20086 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:39.163265 20086 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:39.164141 20011 raft_consensus.cc:2368] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:39.164541 20858 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:39.164237 20086 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.167179 22283 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.167972 20008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:39.168551 20008 raft_consensus.cc:606] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:39.168627 20854 raft_consensus.cc:2393] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:39.168905 20008 raft_consensus.cc:614] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:39.169858 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.170507 20857 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.174791 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:39.176581 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.177313 20012 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:39.178274 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.180097 22355 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.182574 20165 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:39.185575 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.186395 20161 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:39.187192 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.188329 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.188843 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.189488 20156 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.192348 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.193197 20011 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:39.193995 22331 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.194196 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:39.163764 20091 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.197188 22203 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:39.197705 22203 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:39.198479 22203 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:39.199414 22203 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:39.201103 22434 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.171s	user 0.000s	sys 0.003s
I20260709 13:00:39.202179 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.187405 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.205063 22315 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.199417 20088 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:00:39.167750 22283 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:39.206727 20855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 783) took 2450 ms (client timeout 2400 ms). Trace:
W20260709 13:00:39.207166 20855 rpcz_store.cc:269] 0709 13:00:36.756635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.757463 (+   828us) service_pool.cc:224] Handling call
0709 13:00:39.206454 (+2448991us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400008cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:39.206602 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3093630208}
W20260709 13:00:39.202562 22446 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:00:39.212900 22167 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:39.214003 22167 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.196997 22395 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.192397 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.217460 22240 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.218964 22240 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.221457 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 363) took 2192 ms. Trace:
I20260709 13:00:39.222208 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.223011 20163 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:39.224169 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.224336 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.224931 22393 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:39.224994 20162 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:39.225996 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.226277 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.228992 22393 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:39.229835 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.221899 20016 rpcz_store.cc:276] 0709 13:00:37.028993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.029103 (+   110us) service_pool.cc:224] Handling call
0709 13:00:37.044657 (+ 15554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.974483 (+929826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.221203 (+1246720us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b54009fa6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:39.221373 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113215232}
I20260709 13:00:39.226179 20159 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:39.232973 22095 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:39.235440 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55704.2us
I20260709 13:00:39.238042 20857 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:39.236338 22348 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:39.233530 22095 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:39.240463 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:39.241014 20157 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:39.218997 22439 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:00:39.243839 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:39.244544 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:39.244192 20091 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.244952 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:39.245321 20090 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:39.247767 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.251770 22223 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:39.254683 22348 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:39.255245 22348 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:39.255393 22394 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.257053 20088 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:39.260329 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.239622 22287 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:39.261039 20743 connection.cc:256] RPC call timeout handler was delayed by -1.3230779830009851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260709 13:00:39.265064 20743 connection.cc:256] RPC call timeout handler was delayed by -1.1439298940003937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:00:39.266974 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.267836 20856 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:39.267778 20855 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:39.208268 22283 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.268294 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.269946 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.259629 22235 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:39.270917 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.254163 22315 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:39.264516 22275 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.275368 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.276257 20856 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:39.277089 22275 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.277550 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.278362 20851 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:39.278172 22223 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:39.279017 22223 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:39.279138 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.279381 22223 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:39.279657 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.280100 22235 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.280495 22366 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:39.280933 22366 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:39.281808 22366 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.282948 22235 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.283422 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:39.283740 22366 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.289163 20090 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.289722 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.259644 22094 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.290967 22094 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:39.284658 20859 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.273396 21856 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.293058 22094 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:39.288314 22348 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.284238 20858 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.295135 20008 raft_consensus.cc:2410] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:00:39.295396 20851 raft_consensus.cc:2368] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:39.300892 20856 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:00:39.277575 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:39.301617 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.302215 20855 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.302759 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.303329 20857 raft_consensus.cc:2376] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11. Re-sending same reply.
I20260709 13:00:39.303906 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.301259 20156 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:39.305588 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:39.306568 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.306833 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.308131 20161 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:00:39.307189 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:39.307452 20156 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:39.305030 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:39.311633 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:39.313423 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.314406 20156 raft_consensus.cc:2368] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:39.279196 22275 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.317741 20130 leader_election.cc:400] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:39.318037 20130 leader_election.cc:403] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:39.318643 22335 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:39.318845 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.319221 22335 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.319693 22335 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:39.321813 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.322613 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.328008 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.308732 22244 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:39.328336 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.309896 22287 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.329027 20156 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:39.319286 22275 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:00:39.370597 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.375833 20090 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:39.369334 20011 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:39.376605 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:39.380928 20857 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.383805 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.381860 20014 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:39.388377 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.389444 20016 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:39.385540 20852 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
W20260709 13:00:39.357303 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.392689 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.308060 22235 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:00:39.284426 22223 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:39.309417 22365 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:00:39.328202 22378 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:39.403726 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88602.2us
I20260709 13:00:39.482472 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:39.401360 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.315653 22337 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:39.400939 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.487844 20857 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:39.490841 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.433287 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.420909 20855 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:39.471060 22325 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.472338 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:39.407164 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.495090 20014 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:39.475075 20087 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:39.480266 22095 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:39.318749 22208 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:39.487275 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:39.496960 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.497903 20855 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:39.435429 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:39.492019 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 167682us
I20260709 13:00:39.311223 22207 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:00:39.393474 21851 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:39.511039 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.468350 20156 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:39.506218 22223 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:39.492798 20158 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.488917 20012 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:39.483465 20011 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:39.389899 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:39.386083 22287 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:39.524293 22287 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.501469 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.528805 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.401597 22284 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.107s
W20260709 13:00:39.502652 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67347.7us
I20260709 13:00:39.503530 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.448565 22115 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.454770 22363 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:39.505257 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.506313 22208 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:39.508992 22167 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:39.328328 22383 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.225s
W20260709 13:00:39.531770 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 142009us
I20260709 13:00:39.537364 20854 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:39.551872 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.392251 22355 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.498843 22281 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:39.494374 22262 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:39.536698 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:39.545567 20085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 1107) took 2590 ms (client timeout 2081 ms). Trace:
I20260709 13:00:39.532348 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:39.555263 19982 leader_election.cc:400] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:39.556425 20852 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:39.549618 20158 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
W20260709 13:00:39.398115 22443 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:00:39.556730 19982 leader_election.cc:403] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:39.576236 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.513266 22302 consensus_meta.cc:287] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.000s
W20260709 13:00:39.555703 20085 rpcz_store.cc:269] 0709 13:00:36.955265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.955539 (+   274us) service_pool.cc:224] Handling call
0709 13:00:39.545355 (+2589816us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54005a1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:39.545468 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3400668544}
I20260709 13:00:39.577796 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:39.620467 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57685.4us
I20260709 13:00:39.580826 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:39.537498 22115 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:39.537626 22363 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:39.591586 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.685420 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:39.637032 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.653465 20859 raft_consensus.cc:1217] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Deduplicated request from leader. Original: 6.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 13:00:39.709983 20859 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:39.555778 22223 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.558776 22281 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:39.542164 22444 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.507s	user 0.002s	sys 0.001s
W20260709 13:00:39.714468 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 136752us
I20260709 13:00:39.717569 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:39.544431 22335 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.541038 22208 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.721757 22362 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.547060 22383 consensus_queue.cc:1243] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:39.724645 22383 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.725154 22383 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.727375 22383 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.729454 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.730365 20161 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:39.731398 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.732402 22094 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:39.541962 22167 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:39.580574 22095 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:39.735246 20161 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 3: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:39.631211 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:39.736191 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 155582us
I20260709 13:00:39.736833 20164 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:39.738734 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:39.739365 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.752936 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.613557 22355 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:39.704825 22302 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.784381 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 361) took 3185 ms. Trace:
I20260709 13:00:39.782931 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.574715 22262 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:39.716847 22223 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:39.748095 22281 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:39.798334 22378 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:39.750599 22167 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.806993 20160 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:39.963406 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.964007 20160 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.808523 22363 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.810369 22115 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:39.973263 22115 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.810988 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.785358 21856 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:39.752393 22284 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:39.812642 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:39.814018 22240 consensus_meta.cc:287] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.595s	user 0.010s	sys 0.001s
I20260709 13:00:39.817628 22400 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:39.819687 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.020272 20086 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:40.020694 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:39.829561 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 802) took 1003 ms. Trace:
I20260709 13:00:40.022538 20853 rpcz_store.cc:276] 0709 13:00:38.825674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.825813 (+   139us) service_pool.cc:224] Handling call
0709 13:00:38.871249 (+ 45436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:38.871729 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:38.871769 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:38.886962 (+ 15193us) log.cc:844] Serialized 26 byte log entry
0709 13:00:38.962062 (+ 75100us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:38.962120 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.962198 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:39.541559 (+579361us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:39.829270 (+287711us) raft_consensus.cc:1685] finished
0709 13:00:39.829290 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.829499 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8182,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4533,"spinlock_wait_cycles":12072832,"thread_start_us":74683,"threads_started":1,"wal-append.queue_time_us":145904}
I20260709 13:00:39.776079 22203 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:39.841269 22335 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:39.751255 22095 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.545745 22287 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:39.852906 20854 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:39.853902 22459 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.854272 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:40.037591 20008 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:00:40.037739 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 183560us
I20260709 13:00:39.854763 22207 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:39.861322 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.039623 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:00:39.865790 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.041451 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.867727 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:40.012209 21856 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:39.870940 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 363161us
I20260709 13:00:39.871218 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:40.048906 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 181290us
I20260709 13:00:40.053858 20085 raft_consensus.cc:2410] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:39.875494 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:40.054627 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:39.558425 21851 consensus_queue.cc:1059] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.190s
W20260709 13:00:39.876927 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 377820us
I20260709 13:00:39.881572 20017 rpcz_store.cc:276] 0709 13:00:36.599317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.621904 (+ 22587us) service_pool.cc:224] Handling call
0709 13:00:36.622414 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:36.693928 (+ 71514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:36.693959 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:36.875284 (+181325us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b48000801d8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 13:00:38.845714 (+1970430us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:38.870021 (+ 24307us) log.cc:844] Serialized 26 byte log entry
0709 13:00:38.985743 (+115722us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:38.985778 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.985860 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:39.397457 (+411597us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:39.784116 (+386659us) raft_consensus.cc:1685] finished
0709 13:00:39.784141 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:39.784307 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71058,"raft.queue_time_us":71711,"raft.run_cpu_time_us":2493,"raft.run_wall_time_us":180886,"spinlock_wait_cycles":40918656,"thread_start_us":107725,"threads_started":1,"wal-append.queue_time_us":193028}
I20260709 13:00:39.639087 22366 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:39.884627 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.060101 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:39.879819 20164 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:39.961311 22223 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.970858 22363 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.984489 20083 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:39.986135 19838 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 7, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:40.015106 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:40.018276 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:39.888146 22463 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:40.029621 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.030287 20162 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:40.032140 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.045799 20015 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
W20260709 13:00:39.870340 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 338799us
I20260709 13:00:40.057746 20086 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:00:40.058990 20156 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:40.064256 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.058344 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.060796 20161 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:40.055229 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.068176 20160 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:40.028648 22203 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.028951 22095 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:40.057493 21856 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:40.092572 20087 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:40.085273 19838 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:40.094072 20013 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:40.052794 22378 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.083210 20158 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.095240 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:40.094451 19838 catalog_manager.cc:5697] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 4, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:40.100688 19838 catalog_manager.cc:5697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 7, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:40.065907 22406 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:40.086149 19982 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:40.144548 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 58460.2us
I20260709 13:00:40.147107 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:40.088433 19835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35376 (request call id 96) took 19321 ms (client timeout 15000 ms). Trace:
I20260709 13:00:40.087131 20086 pending_rounds.cc:85] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 4
I20260709 13:00:40.152266 20086 pending_rounds.cc:107] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:00:40.152675 20086 raft_consensus.cc:2889] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:40.153661 22320 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:00:40.097065 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.086092 20165 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:00:40.159192 22362 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:40.160028 22362 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:40.161389 22362 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.161984 22353 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:40.162961 22353 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.165804 22353 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:40.165622 22368 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:40.064496 22284 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:40.046383 22302 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:40.168700 22302 raft_consensus.cc:697] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:40.170168 22302 consensus_queue.cc:237] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.102322 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:40.172873 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 70602.2us
I20260709 13:00:40.173846 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:40.052690 22453 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:40.102504 20743 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:40.103869 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45593.9us
I20260709 13:00:40.104935 19838 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 4 to 5, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 5 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:40.082145 22395 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.106947 22207 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.098282 22343 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:40.085238 22369 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:40.088161 22366 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.109727 22325 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.110203 22315 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:40.092284 22462 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:40.085304 22333 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:00:40.089975 21851 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:40.111676 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.082293 22337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.039s	sys 0.039s
I20260709 13:00:40.117161 22348 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:40.211198 22337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.039s	sys 0.040s
I20260709 13:00:40.211735 22315 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:40.213268 22315 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.118336 22326 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.215262 22326 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.216686 21851 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.218261 22326 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:40.218494 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 117793us
I20260709 13:00:40.219151 21851 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.129678 22287 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.130805 22262 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:40.047612 22458 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:40.175799 22094 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.093724 22463 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.239470 20016 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:40.179718 19838 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:40.181433 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:40.148136 19835 rpcz_store.cc:269] 0709 13:00:20.767490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.785665 (+ 18175us) service_pool.cc:224] Handling call
0709 13:00:21.061789 (+276124us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54008b1fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 13:00:40.088364 (+19026575us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:34.201462 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:34.369994 (+168532us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:34.384335 (+ 14341us) write_op.cc:435] Acquired schema lock
0709 13:00:34.384356 (+    21us) tablet.cc:662] Decoding operations
0709 13:00:34.385200 (+   844us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:34.385285 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:34.385320 (+    35us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:00:34.385872 (+   552us) tablet.cc:701] Row locks acquired
0709 13:00:34.385900 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:00:34.386083 (+   183us) write_op.cc:270] Start()
0709 13:00:34.386220 (+   137us) write_op.cc:276] Timestamp: P: 1783602034386004 usec, L: 0
0709 13:00:34.386245 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:00:34.386949 (+   704us) log.cc:844] Serialized 1693 byte log entry
0709 13:00:35.066230 (+679281us) op_driver.cc:464] REPLICATION: finished
0709 13:00:35.522458 (+456228us) write_op.cc:301] APPLY: starting
0709 13:00:35.522562 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:35.522725 (+   163us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:35.522857 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:35.523522 (+   665us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:35.523746 (+   224us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:00:35.523794 (+    48us) write_op.cc:312] APPLY: finished
0709 13:00:35.524114 (+   320us) log.cc:844] Serialized 40 byte log entry
0709 13:00:35.524315 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:35.524438 (+   123us) write_op.cc:454] Released schema lock
0709 13:00:35.524485 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:35.524866 (+   381us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:35.524894 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":89189,"reported_tablets":31,"spinlock_wait_cycles":386379520,"child_traces":[["op",{"apply.queue_time_us":448728,"apply.run_cpu_time_us":2382,"apply.run_wall_time_us":2382,"num_ops":4,"prepare.queue_time_us":320885,"prepare.run_cpu_time_us":17519,"prepare.run_wall_time_us":417961,"raft.queue_time_us":259232,"raft.run_cpu_time_us":27835,"raft.run_wall_time_us":445214,"replication_time_us":670657,"thread_start_us":723153,"threads_started":4,"wal-append.queue_time_us":333389,"wal-append.run_cpu_time_us":70001,"wal-append.run_wall_time_us":1838173}]]}
I20260709 13:00:40.241523 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 379) took 1071 ms. Trace:
I20260709 13:00:40.126986 22167 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:40.183488 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.184911 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:40.094755 22203 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:40.188025 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 85620.9us
I20260709 13:00:40.252750 20163 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:40.252913 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:40.190359 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:40.191395 20012 raft_consensus.cc:3055] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:40.099361 22359 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:40.176102 22320 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:40.207261 20014 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:40.119061 22223 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.120252 21856 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.048187 22459 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.198208 22275 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:40.220281 22235 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:40.220554 22462 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:40.221259 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.196715 22240 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.176425 22378 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.275384 22223 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:40.260977 22359 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:40.277972 22359 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:40.279573 22462 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.269878 22235 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:40.284092 22325 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:40.284677 22325 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.284454 22235 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.285411 22459 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.286796 22325 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:40.288026 22167 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:40.288834 22281 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.230358 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.232982 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 379) took 1057 ms. Trace:
W20260709 13:00:40.293936 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 64759.1us
I20260709 13:00:40.293645 20013 rpcz_store.cc:276] 0709 13:00:39.175815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.177876 (+  2061us) service_pool.cc:224] Handling call
0709 13:00:40.232757 (+1054881us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b54004ea054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:40.232910 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2104206208}
I20260709 13:00:40.106380 22452 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.186837 22453 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.199254 22284 consensus_queue.cc:1059] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:00:40.199707 22002 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:00:40.193392 22343 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.324594 22366 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.325487 22343 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:40.241958 20083 rpcz_store.cc:276] 0709 13:00:39.160508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.160767 (+   259us) service_pool.cc:224] Handling call
0709 13:00:40.231725 (+1070958us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54007a76d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:40.231871 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891847552}
W20260709 13:00:40.248338 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 63526.2us
I20260709 13:00:40.243674 19838 catalog_manager.cc:5697] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 4, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:40.198755 22369 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.327448 19838 catalog_manager.cc:5697] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 5, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 5 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:40.328392 22235 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.328742 22369 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:40.251791 20086 raft_consensus.cc:1275] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:40.254065 20012 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:40.330790 22275 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.331000 22002 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:40.331370 22275 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.333431 22275 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.335955 20163 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:40.254491 19982 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:40.261024 20743 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:40.237351 22458 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:40.266319 20085 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:40.263283 22115 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.271741 22320 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:40.343815 20157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.032s	sys 0.060s
I20260709 13:00:40.277318 22378 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.231475 20158 raft_consensus.cc:1240] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:40.338500 22273 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.019s	sys 0.051s
I20260709 13:00:40.259960 22262 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:40.259496 22436 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.316655 19981 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.365499 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.328995 19838 catalog_manager.cc:5697] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
W20260709 13:00:40.337837 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 83408.5us
I20260709 13:00:40.344249 22463 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:40.260038 22353 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:00:40.331617 22002 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:40.404819 22002 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.405988 22002 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:40.273653 21856 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:40.413026 22095 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.414351 20082 raft_consensus.cc:2368] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:40.361940 22320 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.422894 22320 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:40.401887 22281 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:40.424875 22203 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.426682 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:40.428282 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:40.429175 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.429186 20016 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 3: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:40.432161 22223 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:40.344527 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.436061 22262 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:40.436616 20165 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:00:40.437310 22262 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.333427 22453 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.313738 22287 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:40.359216 20157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.033s	sys 0.060s
I20260709 13:00:40.440166 22115 consensus_queue.cc:1059] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:40.441443 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 924) took 1219 ms. Trace:
I20260709 13:00:40.441550 22115 raft_consensus.cc:3055] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:40.343511 22284 consensus_queue.cc:1243] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:00:40.366233 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 49700us
I20260709 13:00:40.345067 22366 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:40.339179 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 78278us
I20260709 13:00:40.466704 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:40.369489 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:40.355180 22452 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.320425 22240 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:00:40.344134 22458 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.371280 19838 catalog_manager.cc:5697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 6, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:40.401790 22378 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:40.402601 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:40.493844 20158 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.402869 20851 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:00:40.494212 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:40.494944 20851 pending_rounds.cc:85] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 4
I20260709 13:00:40.495285 20851 pending_rounds.cc:107] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:40.495523 20851 raft_consensus.cc:2889] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:40.376482 22368 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.304740 20162 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:40.501308 19838 catalog_manager.cc:5697] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 4, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:40.445029 20163 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:40.415343 22095 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:40.449343 22115 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:40.367769 22273 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.019s	sys 0.052s
W20260709 13:00:40.468637 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.567580 19838 catalog_manager.cc:5697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 3, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:40.466617 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.490284 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:40.495975 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:40.584918 20016 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:40.471671 22464 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:00:40.503734 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:40.587949 20009 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.491990 22203 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:40.604839 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138338us
I20260709 13:00:40.605933 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.607196 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:40.608014 20013 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:40.441471 22002 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.478778 22406 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:40.573211 19838 catalog_manager.cc:5697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:40.577387 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:40.487530 22275 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.611104 19838 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:40.581375 20129 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.003s	sys 0.017s
W20260709 13:00:40.613508 20129 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.007s	sys 0.017s
I20260709 13:00:40.450477 22320 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.377995 22436 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.567062 22452 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:00:40.449523 22287 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:40.614177 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 551433us
I20260709 13:00:40.578361 22353 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:40.621063 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 43789.3us
I20260709 13:00:40.671172 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:40.624276 19838 catalog_manager.cc:5697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:40.606285 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:40.501245 22333 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:00:40.642079 22348 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:00:40.642231 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.642735 22459 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:40.643240 22453 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:40.566627 22368 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:40.657469 22471 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:00:40.660154 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.731441 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.441900 20157 rpcz_store.cc:276] 0709 13:00:39.222369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.833274 (+610905us) service_pool.cc:224] Handling call
0709 13:00:39.833746 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:39.834022 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:39.834407 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.440926 (+606519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:40.441164 (+   238us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:40.441212 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.441276 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.441387 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":184,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":1203,"thread_start_us":606138,"threads_started":1}
I20260709 13:00:40.603551 20158 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:40.715570 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 815) took 1335 ms. Trace:
I20260709 13:00:40.738488 20162 rpcz_store.cc:276] 0709 13:00:39.487821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.087362 (+599541us) service_pool.cc:224] Handling call
0709 13:00:40.087615 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.502625 (+415010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.697361 (+194736us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400573854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:40.715463 (+ 18102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479049600}
I20260709 13:00:40.702234 20012 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:40.719424 20159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.012s	sys 0.037s
I20260709 13:00:40.647578 22283 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:40.648092 22281 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:40.722702 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 720) took 1239 ms. Trace:
I20260709 13:00:40.748799 20851 rpcz_store.cc:276] 0709 13:00:39.483466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.486097 (+  2631us) service_pool.cc:224] Handling call
0709 13:00:39.503455 (+ 17358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.496086 (+992631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.722369 (+226283us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b540027e2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:40.722614 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067667328}
W20260709 13:00:40.740447 20159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.012s	sys 0.037s
W20260709 13:00:40.741142 22362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.018s	sys 0.044s
W20260709 13:00:40.747175 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54205.7us
I20260709 13:00:40.620111 22002 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.714756 22275 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.655368 22320 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.783176 22436 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:40.727474 22453 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.748469 22283 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.762125 20016 raft_consensus.cc:2410] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:40.644294 22465 consensus_queue.cc:1059] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
I20260709 13:00:40.764082 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 928) took 1540 ms. Trace:
I20260709 13:00:40.699745 22353 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.801486 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1122) took 1560 ms. Trace:
W20260709 13:00:40.762637 22362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.018s	sys 0.044s
I20260709 13:00:40.759074 22378 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:40.800179 22283 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.748229 22281 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:40.801286 22349 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.000s	sys 0.067s
I20260709 13:00:40.808362 22452 raft_consensus.cc:3055] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:40.807776 20742 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.013s	sys 0.008s
W20260709 13:00:40.818096 20055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.000s	sys 0.028s
I20260709 13:00:40.691341 22115 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:40.758713 22459 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:00:40.817058 20014 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:40.787739 22453 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:40.818516 20055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.000s	sys 0.028s
I20260709 13:00:40.802163 20084 rpcz_store.cc:276] 0709 13:00:39.241008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.242826 (+  1818us) service_pool.cc:224] Handling call
0709 13:00:39.546978 (+304152us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:39.547198 (+   220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.575390 (+ 28192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.575419 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.575883 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.847682 (+271799us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18000277c0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:00:40.178409 (+330727us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:40.727234 (+548825us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.750371 (+ 23137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.750481 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:40.800444 (+ 49963us) raft_consensus.cc:1685] finished
0709 13:00:40.800530 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.801391 (+   861us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55047,"raft.queue_time_us":50743,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":83781,"spinlock_wait_cycles":9710208,"thread_start_us":827594,"threads_started":2,"wal-append.queue_time_us":912884}
W20260709 13:00:40.821672 20164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.020s	sys 0.009s
W20260709 13:00:40.840229 20164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.020s	sys 0.010s
W20260709 13:00:40.823480 20742 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.013s	sys 0.008s
W20260709 13:00:40.843341 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.798822 20054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.020s	sys 0.047s
W20260709 13:00:40.844502 20054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.021s	sys 0.047s
W20260709 13:00:40.821354 22349 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.000s	sys 0.069s
I20260709 13:00:40.795926 22464 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:00:40.866952 20017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.026s	sys 0.033s
W20260709 13:00:40.867453 20017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.026s	sys 0.033s
I20260709 13:00:40.845837 22406 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:40.819604 22378 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.870038 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:40.871496 20085 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:40.852999 22459 raft_consensus.cc:3055] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:40.860080 22359 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.001s
I20260709 13:00:40.856338 22203 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.814584 22287 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:40.866530 22095 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.865348 22115 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:40.881271 20160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.018s	sys 0.059s
I20260709 13:00:40.823227 22452 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:40.881980 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:40.873768 20011 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.034s	sys 0.042s
W20260709 13:00:40.903975 20011 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.034s	sys 0.042s
I20260709 13:00:40.874645 22459 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:40.836863 22462 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.010s	sys 0.020s
W20260709 13:00:40.894512 20129 leader_election.cc:336] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Timed out: RequestConsensusVote RPC to 127.12.30.65:38607 timed out after 1.569s (SENT)
W20260709 13:00:40.897166 20056 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.007s	sys 0.029s
W20260709 13:00:40.914366 20056 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.007s	sys 0.029s
I20260709 13:00:40.898658 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:40.915185 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 653174us
W20260709 13:00:40.898160 20160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.019s	sys 0.059s
I20260709 13:00:40.914275 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:40.910501 20087 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.024s	sys 0.046s
W20260709 13:00:40.918308 20087 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.024s	sys 0.046s
I20260709 13:00:40.874271 22406 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.869277 22281 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:40.913962 22462 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.010s	sys 0.021s
I20260709 13:00:40.903939 22095 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.935850 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.930409 22368 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:40.937407 22095 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:40.937616 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.889607 22203 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:40.938392 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.945765 20009 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:40.958640 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.871343 22302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.021s	sys 0.054s
W20260709 13:00:40.871511 22382 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.017s	sys 0.013s
I20260709 13:00:40.895418 22359 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:40.999432 20129 leader_election.cc:336] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Timed out: RequestConsensusVote RPC to 127.12.30.65:38607 timed out after 1.707s (SENT)
I20260709 13:00:40.999946 20129 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:41.001291 22359 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:40.974875 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:40.975219 20085 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:41.003648 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:40.996682 22382 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.017s	sys 0.013s
I20260709 13:00:41.010906 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:41.014153 20163 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:40.946916 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:40.924777 22208 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:40.995771 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:41.048051 20165 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:40.971473 22095 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.999245 22353 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:40.994518 22302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.021s	sys 0.054s
I20260709 13:00:40.983219 20084 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:41.028745 22094 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.035233 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.039637 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.041345 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.069217 20086 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
W20260709 13:00:41.067812 20015 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.015s
W20260709 13:00:41.070816 20015 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.000s	sys 0.015s
W20260709 13:00:40.977088 20156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.015s	sys 0.018s
I20260709 13:00:41.038419 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:40.973973 22470 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:00:41.048813 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 942) took 1264 ms. Trace:
I20260709 13:00:41.075816 20164 rpcz_store.cc:276] 0709 13:00:39.784930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.225223 (+440293us) service_pool.cc:224] Handling call
0709 13:00:40.225567 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:40.226013 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:40.226042 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:40.258142 (+ 32100us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b2c001e7048. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:40.264929 (+  6787us) log.cc:844] Serialized 52 byte log entry
0709 13:00:40.840697 (+575768us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:40.840734 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.840802 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.031527 (+190725us) raft_consensus.cc:1685] finished
0709 13:00:41.031566 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.048748 (+ 17182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1932,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":8244992,"thread_start_us":549632,"threads_started":1,"wal-append.queue_time_us":738652}
W20260709 13:00:40.946419 22207 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.012s	sys 0.023s
I20260709 13:00:40.947369 22369 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:40.949146 22002 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:41.056587 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.057127 22406 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.082867 20090 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3.
I20260709 13:00:41.053622 22487 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:41.052126 22235 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.063961 22348 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:41.090792 19982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.021s	sys 0.033s
W20260709 13:00:41.091202 19982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.021s	sys 0.033s
I20260709 13:00:41.064025 22095 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.004187 22320 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:40.928135 22457 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 13:00:40.928747 22453 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
W20260709 13:00:40.976888 22315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.005s	sys 0.028s
W20260709 13:00:41.073761 20156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.015s	sys 0.018s
W20260709 13:00:41.074972 19983 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.002s	sys 0.014s
I20260709 13:00:41.073040 20088 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:41.079404 20089 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:41.069269 20091 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:41.080248 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:41.083266 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:41.094400 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:41.081367 22207 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.013s	sys 0.024s
W20260709 13:00:41.130486 19983 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.014s	sys 0.016s
I20260709 13:00:41.131444 20163 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:41.133399 20157 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:41.110078 19980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.020s	sys 0.040s
I20260709 13:00:41.138703 20165 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:41.074486 22482 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.090481 22348 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:41.089524 22369 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:41.153554 19980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.022s	sys 0.041s
W20260709 13:00:41.158280 20744 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.003s	sys 0.017s
W20260709 13:00:41.160000 20744 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.004s	sys 0.017s
W20260709 13:00:41.160547 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 554695us
W20260709 13:00:41.130285 22315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.006s	sys 0.029s
I20260709 13:00:40.990074 22452 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:41.080938 22094 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.036092 22459 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:41.174427 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 409249us
I20260709 13:00:41.130981 22487 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:41.156391 22355 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:41.152513 22002 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:41.184571 19981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.008s	sys 0.014s
W20260709 13:00:41.185002 19981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.008s	sys 0.014s
I20260709 13:00:41.188550 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:41.189507 20009 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:41.186471 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:41.163132 22207 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.162484 22368 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:41.199106 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.195838 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.199826 20089 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:41.201510 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.176647 22359 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.180305 22203 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.213827 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:41.201031 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:41.221717 20009 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:41.195664 22459 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.207342 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:41.227226 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:41.230087 20009 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:41.234290 20161 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:41.237087 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:41.237540 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:41.237828 20164 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:41.238158 20088 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:41.240077 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 240649us
I20260709 13:00:41.199038 22349 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.244441 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1002) took 1054 ms. Trace:
I20260709 13:00:41.250079 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.250135 20011 rpcz_store.cc:276] 0709 13:00:40.190358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.190486 (+   128us) service_pool.cc:224] Handling call
0709 13:00:40.206769 (+ 16283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.207017 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:40.904569 (+697552us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.243561 (+338992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:41.243662 (+   101us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:41.243782 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.244034 (+   252us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.244310 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21246,"prepare.queue_time_us":813592,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":993200,"threads_started":2}
W20260709 13:00:41.225373 22262 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.001s	sys 0.016s
I20260709 13:00:41.248124 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:41.257280 20161 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:41.229476 20744 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:41.161399 22482 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:41.255311 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.208601 22335 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:41.211942 22378 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.223196 22353 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.213449 22315 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.213479 22284 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.194401 22002 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.242503 21856 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:41.184852 22355 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.261996 20129 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:41.278700 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.245002 22333 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:41.230937 22452 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.206805 22368 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.257339 22320 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.204160 22207 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:41.296463 20010 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:41.299304 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.299947 20011 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:41.300274 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.214048 22359 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.291126 19981 leader_election.cc:400] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:41.303016 19981 leader_election.cc:403] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:00:41.304517 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:41.296105 22493 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.296830 20088 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:41.249634 21851 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.286936 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:41.202965 22348 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:41.308542 20015 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.258942 22349 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.307703 20089 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:41.309000 22496 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.286226 21856 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:41.317524 22491 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:00:41.402446 20014 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:41.284698 22315 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:41.318413 22488 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:00:41.319151 22475 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:00:41.413616 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 823) took 1912 ms. Trace:
I20260709 13:00:41.414021 20160 rpcz_store.cc:276] 0709 13:00:39.501142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.113795 (+612653us) service_pool.cc:224] Handling call
0709 13:00:40.114148 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:40.114515 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:40.114559 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:40.115060 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:00:40.917078 (+802018us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:40.974195 (+ 57117us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.976925 (+  2730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.977048 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.412859 (+435811us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b18002ed680. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:41.413306 (+   447us) raft_consensus.cc:1685] finished
0709 13:00:41.413406 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.413549 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35309,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":1888,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":43570,"spinlock_wait_cycles":10947328,"thread_start_us":801594,"threads_started":1,"wal-append.queue_time_us":1036551}
I20260709 13:00:41.319769 22281 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:41.284417 22284 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:40.799401 20159 rpcz_store.cc:276] 0709 13:00:39.223413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.027754 (+804341us) service_pool.cc:224] Handling call
0709 13:00:40.052750 (+ 24996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.052973 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:40.085040 (+ 32067us) log.cc:844] Serialized 14 byte log entry
0709 13:00:40.763345 (+678305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:40.763420 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:40.763491 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:40.763809 (+   318us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:40.764005 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9617,"prepare.queue_time_us":36961,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":21867,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":14993,"thread_start_us":668799,"threads_started":1}
I20260709 13:00:41.327864 22501 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:41.420331 22501 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.420771 22501 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:41.421432 20016 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:00:41.278379 22487 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:41.336015 22499 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:41.421962 20016 pending_rounds.cc:85] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:41.422210 22499 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:41.422369 20016 pending_rounds.cc:107] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:00:41.422711 22499 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:41.422758 20016 raft_consensus.cc:2889] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:41.336891 22369 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:41.337781 22492 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:00:41.275727 22382 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:00:41.425201 22382 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:41.425541 22382 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:41.426367 22382 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:41.428261 22382 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:41.428856 22094 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.429750 22094 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.430500 22094 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:41.343571 22469 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.615s	user 0.001s	sys 0.005s
I20260709 13:00:41.307183 22235 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:41.437800 22369 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.438481 22369 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:41.358312 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:41.367838 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:41.372426 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 66643.9us
I20260709 13:00:41.389629 22496 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:41.341432 22203 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:00:41.520164 22203 consensus_queue.cc:1243] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:41.506101 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:41.538010 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 67828.9us
I20260709 13:00:41.539860 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:41.542398 20012 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.541129 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:41.567101 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:00:41.570923 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 341480us
I20260709 13:00:41.576346 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:41.577492 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 390) took 1102 ms. Trace:
W20260709 13:00:41.578698 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 72709.3us
I20260709 13:00:41.578178 20015 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
W20260709 13:00:41.585506 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 227320us
I20260709 13:00:41.303884 22320 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:41.590960 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:41.575786 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:41.577039 22366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.004s	sys 0.017s
I20260709 13:00:41.583621 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:41.591935 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:41.310864 22493 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:41.640740 20011 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:41.641403 22462 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:41.583266 22507 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:41.609261 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.650215 20010 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:41.582700 22348 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.578068 20008 rpcz_store.cc:276] 0709 13:00:40.475546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.594024 (+118478us) service_pool.cc:224] Handling call
0709 13:00:40.594415 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.594674 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:41.134934 (+540260us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.577093 (+442159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:41.577145 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:41.577206 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.577289 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.577424 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9202,"prepare.queue_time_us":426465,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":34,"raft.queue_time_us":10750,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":915624,"threads_started":3,"wal-append.queue_time_us":385475}
W20260709 13:00:41.637421 22458 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 1.059s	user 0.012s	sys 0.010s
I20260709 13:00:41.624109 20009 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:41.588299 22326 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:41.570919 22483 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.667213 22467 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:41.667270 22483 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.669785 22483 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.583611 22353 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:00:41.630550 22366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.005s	sys 0.017s
W20260709 13:00:41.652940 22283 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.812s	user 0.025s	sys 0.015s
I20260709 13:00:41.671656 22283 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:41.673225 22517 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:41.673913 22283 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.673815 22517 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:41.674489 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 82641.9us
I20260709 13:00:41.675223 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:41.592525 22302 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:41.628568 22465 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:41.643867 22207 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.633862 22095 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.601785 22203 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:41.683763 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 82575.7us
I20260709 13:00:41.695760 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.642704 22462 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.602648 21851 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:41.592516 22223 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:41.642146 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 58637.7us
I20260709 13:00:41.600955 22235 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:00:41.642704 22262 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:41.741271 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 150349us
I20260709 13:00:41.778056 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.779181 20010 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:41.653825 22497 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.690486 20012 raft_consensus.cc:1275] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:41.788347 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1003) took 1597 ms. Trace:
I20260709 13:00:41.621464 22002 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:41.601166 22411 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
W20260709 13:00:41.703874 20157 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.011s
I20260709 13:00:41.789426 20157 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:41.702145 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:41.772917 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.773307 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:41.701934 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:41.699283 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:41.678818 22517 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:41.784684 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.682650 22302 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.657027 22275 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:00:41.788918 20012 rpcz_store.cc:276] 0709 13:00:40.190709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.190855 (+   146us) service_pool.cc:224] Handling call
0709 13:00:40.191256 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.701791 (+510535us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400469c20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e73006f 00007fa02e7a8f60 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62
0709 13:00:41.345073 (+643282us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b58000a58d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:00:41.787607 (+442534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.788253 (+   646us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333226752}
I20260709 13:00:41.744520 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.792500 20082 raft_consensus.cc:2410] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:41.816213 20009 raft_consensus.cc:2410] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:41.676518 22366 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:41.774785 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:41.690645 22315 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:41.681160 21856 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:41.790807 20011 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:41.804761 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:41.679322 22458 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:41.828699 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 126869us
I20260709 13:00:41.844115 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:41.688659 22353 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:41.795686 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:41.693522 22273 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.852623 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 57040.4us
W20260709 13:00:41.828553 22244 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:00:41.720135 22462 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:41.725718 22359 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:41.787985 22287 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:41.834702 22207 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:41.712616 22369 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:41.691434 22368 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:41.706719 22520 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:41.853565 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:41.788883 22355 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:41.794516 22411 consensus_queue.cc:1243] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:41.795840 22482 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:41.858996 22284 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:41.866598 22483 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:41.871753 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 95197.4us
I20260709 13:00:41.875753 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.929888 20854 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:41.876448 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.931226 20853 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:41.877033 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:41.932667 20857 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:41.820355 22496 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:41.877614 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:41.857607 22115 consensus_meta.cc:287] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.953s	user 0.018s	sys 0.000s
I20260709 13:00:41.662814 22507 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:41.834911 22524 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:41.880842 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.798635 22262 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:41.917872 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 64409.6us
I20260709 13:00:41.829665 22302 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:41.920576 20858 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:41.917372 22359 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.923163 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 289306us
I20260709 13:00:41.867780 22378 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:41.706046 22465 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:41.871640 22287 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.945096 22463 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:41.995569 20161 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:41.938230 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.997735 20856 raft_consensus.cc:2410] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:41.941179 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.878472 22207 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:41.879581 22223 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:42.006892 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.856707 21851 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:41.865937 22514 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:42.015125 22320 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.014899 22514 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.973949 22482 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.016242 22482 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:41.915027 22520 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.017144 22262 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.017216 22520 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.018257 22482 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:41.969616 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.808779 22497 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.956400 20859 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:41.974530 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:41.927126 22208 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:41.976148 20853 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:41.977582 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.045068 20857 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:42.010689 22207 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:41.928483 22483 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.946305 22275 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.987373 20162 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:41.935655 22349 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:41.979624 22359 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:41.959875 20858 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:42.008399 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 158374us
I20260709 13:00:41.934705 22355 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:00:42.010052 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:42.010936 20157 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:41.973119 22115 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.013618 20743 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:41.978615 22353 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:41.985473 22516 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.023967 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.073096 20160 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:41.986557 22378 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:41.986392 22368 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:41.992810 22287 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.045226 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.083788 20859 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:42.084231 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.046444 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.086681 20857 raft_consensus.cc:2376] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11. Re-sending same reply.
I20260709 13:00:41.866248 22273 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:41.980980 20158 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
W20260709 13:00:42.009650 22513 log.cc:927] Time spent T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
W20260709 13:00:42.024977 22500 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.366s	user 0.001s	sys 0.001s
I20260709 13:00:42.048537 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.052632 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.093385 20855 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
W20260709 13:00:42.053908 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 85421.5us
I20260709 13:00:42.055069 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.093963 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.094507 20853 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:42.061869 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.095781 20162 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
W20260709 13:00:42.049890 22511 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.366s	user 0.001s	sys 0.001s
I20260709 13:00:41.988106 22366 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.044476 22208 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:42.050987 22515 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 13:00:42.061853 20858 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:42.070139 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56618.2us
I20260709 13:00:42.048647 22315 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:42.059887 22275 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.053766 22483 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.020316 22463 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.083248 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.080696 22262 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:42.097784 22333 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:42.098335 22458 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.098734 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.057336 22493 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.101688 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.092741 20854 raft_consensus.cc:2376] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6. Re-sending same reply.
I20260709 13:00:42.079792 22378 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.109107 20859 raft_consensus.cc:2376] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 3. Re-sending same reply.
I20260709 13:00:42.048231 22524 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:42.128127 20857 raft_consensus.cc:2410] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:42.112185 20858 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.112540 22514 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.117199 22273 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.014916 22465 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.074373 22115 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.108296 22208 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.118968 22335 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.096483 22464 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:00:42.131394 22359 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.085981 22516 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.095957 22368 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:42.137948 22519 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:00:42.138226 22526 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:00:42.138758 22528 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 13:00:42.139372 22518 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:00:42.139770 22527 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:42.141256 22499 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.006s
W20260709 13:00:42.141753 22382 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.713s	user 0.005s	sys 0.009s
I20260709 13:00:42.137506 22326 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:42.128801 22333 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:42.145825 22521 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 13:00:42.162237 22501 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.005s
I20260709 13:00:42.162761 22501 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.164973 22501 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.150556 22520 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.168382 22520 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:42.169356 22459 consensus_meta.cc:287] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.920s	user 0.017s	sys 0.001s
I20260709 13:00:42.159634 22507 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.159754 22499 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.176921 22497 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.178900 22499 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.152611 22527 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.179695 22527 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.169945 22524 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.175462 22516 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.185869 22463 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.186163 22462 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.191870 22527 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:41.937311 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 262180us
I20260709 13:00:42.079119 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:42.057336 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:42.229944 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1009) took 1060 ms. Trace:
I20260709 13:00:42.237854 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 746) took 1017 ms. Trace:
I20260709 13:00:42.238381 20013 rpcz_store.cc:276] 0709 13:00:41.220324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.220455 (+   131us) service_pool.cc:224] Handling call
0709 13:00:41.247856 (+ 27401us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:00:41.280582 (+ 32726us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:41.280616 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:00:41.286774 (+  6158us) log.cc:844] Serialized 78 byte log entry
0709 13:00:41.604360 (+317586us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:41.647896 (+ 43536us) log.cc:844] Serialized 14 byte log entry
0709 13:00:41.835520 (+187624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.835626 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:42.228280 (+392654us) raft_consensus.cc:1685] finished
0709 13:00:42.228348 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.237670 (+  9322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167687,"prepare.queue_time_us":43313,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":214360,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":28991,"thread_start_us":432883,"threads_started":2,"wal-append.queue_time_us":389456}
W20260709 13:00:42.229638 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 150407us
W20260709 13:00:42.240731 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 183383us
I20260709 13:00:42.241993 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:42.242267 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:42.230561 20017 rpcz_store.cc:276] 0709 13:00:41.169705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.169831 (+   126us) service_pool.cc:224] Handling call
0709 13:00:41.170142 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:41.171350 (+  1208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:41.171378 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:41.171787 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 13:00:41.288716 (+116929us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:41.288752 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.288818 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:41.410181 (+121363us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b48000801d8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa0336dbaa1 00007fa0336d2b37 00007fa0336d29ca 00007fa0337ca26f 00007fa0337c9f37 00007fa0337de55c 00007fa0337de508 00007fa0337de488 00007fa0337de443 00007fa0337dd6ec 00007fa032707a96
0709 13:00:42.229223 (+819042us) raft_consensus.cc:1685] finished
0709 13:00:42.229268 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.229709 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112661,"raft.queue_time_us":9015,"raft.run_cpu_time_us":2537,"raft.run_wall_time_us":227894,"spinlock_wait_cycles":20772224,"thread_start_us":116638,"threads_started":1,"wal-append.queue_time_us":385590}
I20260709 13:00:42.227672 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:42.250299 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.260972 20162 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:42.254722 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.254870 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:42.255822 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.139430 22115 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:42.302325 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 75193.3us
I20260709 13:00:42.132085 22275 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.271955 21856 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.273180 20851 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:42.189571 22462 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.303927 22499 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.304646 21856 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.306013 22499 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:42.306260 21856 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.306512 22499 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.305392 22320 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:42.308827 22320 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:42.309907 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.310609 22320 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.292447 22458 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:00:42.273272 22203 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.275324 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.277676 20852 raft_consensus.cc:2393] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:42.278685 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.147688 22335 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.102600 22223 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:42.292133 22450 negotiation.cc:336] Failed RPC negotiation. Trace:
0709 13:00:39.179204 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.12.30.65:38607 (local address 127.0.0.1:46670)
0709 13:00:39.254584 (+ 75380us) negotiation.cc:107] Waiting for socket to connect
0709 13:00:39.254675 (+    91us) client_negotiation.cc:175] Beginning negotiation
0709 13:00:39.255255 (+   580us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0709 13:00:41.023140 (+1767885us) client_negotiation.cc:281] Received NEGOTIATE NegotiatePB response
0709 13:00:41.023201 (+    61us) client_negotiation.cc:404] Received NEGOTIATE response from server
0709 13:00:41.023346 (+   145us) client_negotiation.cc:190] Negotiated authn=SASL
0709 13:00:41.156526 (+133180us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0709 13:00:41.170068 (+ 13542us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0709 13:00:41.918393 (+748325us) client_negotiation.cc:281] Received TLS_HANDSHAKE NegotiatePB response
0709 13:00:41.929157 (+ 10764us) client_negotiation.cc:543] Received TLS_HANDSHAKE response from server
0709 13:00:42.083290 (+154133us) client_negotiation.cc:530] Sending TLS_HANDSHAKE message to server
0709 13:00:42.092327 (+  9037us) client_negotiation.cc:262] Sending TLS_HANDSHAKE NegotiatePB request
0709 13:00:42.125954 (+ 33627us) client_negotiation.cc:564] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0709 13:00:42.157267 (+ 31313us) client_negotiation.cc:639] Initiating SASL PLAIN handshake
0709 13:00:42.165935 (+  8668us) client_negotiation.cc:672] calling sasl_client_start()
0709 13:00:42.275969 (+110034us) client_negotiation.cc:830] callback for SASL_CB_AUTHNAME
0709 13:00:42.277151 (+  1182us) client_negotiation.cc:262] Sending SASL_INITIATE NegotiatePB request
0709 13:00:42.288232 (+ 11081us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.12.30.65:38607: sent 0 of 16 requested bytes
Metrics: {"client-negotiator.queue_time_us":74380,"thread_start_us":72177,"threads_started":1}
I20260709 13:00:42.292449 22095 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:42.287627 22472 negotiation.cc:336] Failed RPC negotiation. Trace:
0709 13:00:39.878369 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.12.30.65:38607)
0709 13:00:41.021890 (+1143521us) server_negotiation.cc:207] Beginning negotiation
0709 13:00:41.021942 (+    52us) server_negotiation.cc:400] Waiting for connection header
0709 13:00:41.022033 (+    91us) server_negotiation.cc:408] Connection header received
0709 13:00:41.022255 (+   222us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0709 13:00:41.022296 (+    41us) server_negotiation.cc:462] Received NEGOTIATE request from client
0709 13:00:41.022739 (+   443us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0709 13:00:41.023059 (+   320us) server_negotiation.cc:227] Negotiated authn=SASL
0709 13:00:41.195403 (+172344us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0709 13:00:41.869743 (+674340us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0709 13:00:42.107703 (+237960us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0709 13:00:42.146348 (+ 38645us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0709 13:00:42.284776 (+138428us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46670: BlockingRecv error: recv got EOF from 127.0.0.1:46670 (error 108)
Metrics: {"server-negotiator.queue_time_us":972977,"thread_start_us":824875,"threads_started":1}
I20260709 13:00:42.136327 22094 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.110185 21851 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.296384 22463 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.297586 22482 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.298585 22507 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.300552 22487 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:42.348227 22463 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.300897 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.301573 22359 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.302040 20856 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:42.320984 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.326927 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:42.329583 22095 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.331976 22465 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.347196 22507 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:42.329984 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 75425.2us
I20260709 13:00:42.343163 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:42.341275 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 76146.8us
W20260709 13:00:42.348263 12409 auto_leader_rebalancer.cc:438] leader transfer for tablet 45dd40d980c94ccdab3d167b33984c44 from e5eb27db496b47129df3e689e546d94a to 48da76e971b84219a6ba80d1ba83b7eb failed: Timed out: Client connection negotiation failed: client connection to 127.12.30.65:38607: sent 0 of 16 requested bytes
I20260709 13:00:42.353912 20854 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:42.352429 22095 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.355336 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.356760 22482 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.274403 22326 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.268137 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:42.368486 22465 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.370539 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:42.371908 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.374387 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:42.371734 20011 raft_consensus.cc:606] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:42.364873 22333 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.381922 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.382364 22482 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.382894 20852 raft_consensus.cc:2376] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11. Re-sending same reply.
I20260709 13:00:42.384101 22359 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.384786 20160 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:42.379999 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.386575 20011 raft_consensus.cc:614] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:42.394707 20857 raft_consensus.cc:2376] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5. Re-sending same reply.
I20260709 13:00:42.400450 22463 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.402370 20013 pending_rounds.cc:85] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 2
I20260709 13:00:42.402823 20013 pending_rounds.cc:107] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:00:42.403132 20013 raft_consensus.cc:2889] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:42.403697 22508 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.404574 22508 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.405862 20744 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.404986 22507 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:42.406840 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 95117us
I20260709 13:00:42.408548 22333 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:42.408815 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.409586 22508 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.410043 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.411072 20853 raft_consensus.cc:2368] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:42.414288 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:42.414922 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.414260 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.416217 20158 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:42.409096 22333 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.410105 20160 raft_consensus.cc:2410] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:00:42.420679 22333 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:42.431063 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 127482us
I20260709 13:00:42.418191 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.436623 20854 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:42.438243 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.438838 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.439951 20859 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:42.441686 20859 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:00:42.441836 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 174400us
I20260709 13:00:42.443039 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.444008 20857 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:42.446365 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a42b8897c4224305af65104f541be1fc"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:42.447182 20011 raft_consensus.cc:606] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:42.447407 20856 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:42.449040 22335 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.449626 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:42.450992 22541 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:42.452099 22368 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.452385 22115 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.452122 22507 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:42.452122 22458 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.452992 22115 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.453018 22507 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:42.454921 22507 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.455518 22115 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:42.441705 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52588.7us
I20260709 13:00:42.456853 22499 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.456739 22507 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.460615 22546 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:42.461203 22546 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:42.461614 20055 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.440117 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.446404 20857 pending_rounds.cc:85] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 3
I20260709 13:00:42.467021 20857 pending_rounds.cc:107] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:42.435194 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:42.467368 20857 raft_consensus.cc:2889] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:42.473908 22283 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:42.474421 22283 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:42.476346 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.475975 22283 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.477111 20855 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.477816 22283 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.473203 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
W20260709 13:00:42.482457 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:42.483054 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:42.483296 22502 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:00:42.486470 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 110384us
I20260709 13:00:42.498118 22315 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.498663 22315 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.499045 22315 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.462517 22546 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.502944 22483 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.504096 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.505268 22483 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.505513 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.506021 20855 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:42.506044 22508 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:42.506492 22275 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.494900 20011 raft_consensus.cc:606] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:42.503729 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:42.506701 22508 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:42.513300 22508 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.497370 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:42.516090 20011 raft_consensus.cc:614] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:42.519781 20014 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.508049 22275 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.510720 22115 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.522631 22275 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.534036 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.526273 22115 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.530277 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:42.538560 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:42.551398 22482 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:42.552064 22482 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.552595 22482 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.552729 22333 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.540295 22115 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.541922 22463 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:42.540012 22522 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.560s	user 0.003s	sys 0.003s
I20260709 13:00:42.535231 22517 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:42.561419 22497 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.562141 22497 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.566629 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:42.560233 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:42.568248 22537 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:42.569795 22094 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.570078 22537 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.557497 22499 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.570693 22537 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.570613 22094 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.574210 22094 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:42.575412 22558 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:00:42.576020 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.576438 21856 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.578208 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 949) took 1346 ms. Trace:
I20260709 13:00:42.582939 20156 rpcz_store.cc:276] 0709 13:00:41.231971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.241265 (+  9294us) service_pool.cc:224] Handling call
0709 13:00:41.241591 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:41.313378 (+ 71787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:41.313417 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:41.350868 (+ 37451us) log.cc:844] Serialized 26 byte log entry
0709 13:00:41.690259 (+339391us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:41.690322 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.775050 (+ 84728us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b5400573354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:41.775101 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:42.419827 (+644726us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:42.577692 (+157865us) raft_consensus.cc:1685] finished
0709 13:00:42.577727 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.577955 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26113,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":171641216,"thread_start_us":256229,"threads_started":1,"wal-append.queue_time_us":434822}
I20260709 13:00:42.541467 22378 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.584769 22094 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.585512 22094 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.585996 22094 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.586160 20160 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:42.587430 21856 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.581552 22095 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:42.588712 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 137145us
W20260709 13:00:42.590047 22452 consensus_meta.cc:287] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.291s	user 0.006s	sys 0.018s
I20260709 13:00:42.590451 22095 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.590744 20852 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:42.591941 22382 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:42.592628 22556 log.cc:927] Time spent T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:00:42.592811 22382 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.594871 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:42.596359 22382 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.596659 22378 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.599977 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:42.601034 22483 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:42.605427 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:42.609504 22546 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.616582 22497 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.621337 22527 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.622484 20009 pending_rounds.cc:85] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 4
I20260709 13:00:42.623270 20009 pending_rounds.cc:107] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:42.623687 20009 raft_consensus.cc:2889] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:42.624914 22359 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.625664 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.625943 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.626744 20160 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:42.596248 22287 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.606671 22483 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:42.634395 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:42.644467 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 171621us
I20260709 13:00:42.646364 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.646850 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.645695 20014 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:42.647578 20858 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:42.648541 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.641572 22203 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:42.644697 22333 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.660852 22463 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:42.664355 22207 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.594s	user 0.016s	sys 0.003s
I20260709 13:00:42.666046 22463 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.667520 22369 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.672298 20161 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:42.673240 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.674152 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.681001 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.682121 22287 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.684231 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.687160 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.679244 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.700179 22203 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.701066 22203 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.701329 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.703635 22203 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.704654 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:42.706324 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.706512 22203 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:42.707403 22203 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.707901 22483 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.708721 22483 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.709864 22203 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.710887 22094 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:42.712515 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 107235us
I20260709 13:00:42.713198 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:42.695405 20851 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:42.726130 22462 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:42.726890 22462 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.728179 22537 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.730479 22537 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.731262 22462 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.731513 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:42.732327 22366 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:00:42.734383 22463 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:42.692809 22207 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.736568 20055 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:42.748909 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:42.749692 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.722947 22369 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:42.750692 20085 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:42.752146 22369 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.738853 22527 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:00:42.753724 22527 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:42.754135 22527 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:42.758018 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 74144.1us
I20260709 13:00:42.737882 22465 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.755251 22527 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.779004 20159 raft_consensus.cc:2410] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:42.780539 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:42.782099 20158 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.790951 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.798804 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.800290 20089 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:42.783160 22463 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:42.802556 20091 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:42.784677 22369 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:42.816946 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 78142.9us
I20260709 13:00:42.806766 22463 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:42.818312 22463 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:42.797263 22281 consensus_meta.cc:287] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.237s	user 0.013s	sys 0.013s
I20260709 13:00:42.829162 22094 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:42.811746 22378 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.830550 22378 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.834414 22378 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.839323 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:42.804800 22499 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:42.826364 22359 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 13:00:42.842844 22208 consensus_meta.cc:287] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.695s	user 0.016s	sys 0.001s
I20260709 13:00:42.850579 20850 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:42.851356 22335 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.843528 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:42.852198 22335 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.850611 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.854221 20082 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:42.854667 22207 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.857652 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.862260 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.861570 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.852917 20089 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:42.860882 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:42.866145 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.855865 22335 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.867198 20163 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:42.868304 22499 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:42.871356 22499 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:42.873805 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.797595 22507 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:42.850430 22208 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:42.796706 22483 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:42.849064 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 133438us
I20260709 13:00:42.881812 20089 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
W20260709 13:00:42.901300 22302 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.916s	user 0.017s	sys 0.004s
I20260709 13:00:42.901926 22302 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.904206 22302 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:42.848666 22527 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.911197 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.912312 20854 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.912424 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:42.913494 20859 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:42.914285 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.915907 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.917029 20855 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.918588 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:42.919480 20853 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.878297 22094 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:42.943205 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:42.940560 20162 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:42.848670 22369 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.931293 22094 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.987459 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:42.987746 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:42.996443 20163 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.939802 22302 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:42.996762 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.924167 22507 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:42.949630 20859 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:00:42.999691 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:42.997839 22562 log.cc:927] Time spent T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:00:43.014940 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.003808 22359 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:42.997788 22302 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:43.045876 20855 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
W20260709 13:00:43.047801 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.056943 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.006800 22094 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:43.062446 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:43.062599 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 242696us
I20260709 13:00:43.066121 20855 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:43.066507 20851 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:43.082216 20088 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.084024 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.087831 22302 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:43.088842 22565 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:00:43.092527 22208 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:43.093885 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.094107 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:43.094786 20082 raft_consensus.cc:2393] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:43.098747 20851 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:00:43.063764 22283 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.100631 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.100947 22283 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:43.108108 22546 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.109015 22546 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.112447 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.113732 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.093921 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.072616 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.121348 20084 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 3.
I20260709 13:00:43.137305 20855 pending_rounds.cc:85] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 3
I20260709 13:00:43.137703 20855 pending_rounds.cc:107] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:43.137854 20854 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:43.137949 20855 raft_consensus.cc:2889] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:43.122924 20851 pending_rounds.cc:85] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 5
I20260709 13:00:43.139644 20851 pending_rounds.cc:107] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:00:43.140081 20851 raft_consensus.cc:2889] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:43.141469 20158 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:43.144361 20852 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:43.154462 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:43.155457 20161 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.160347 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:43.161417 20157 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.143740 20853 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:43.173545 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 749) took 1204 ms. Trace:
I20260709 13:00:43.177928 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:43.179697 20854 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.179232 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.186185 20859 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:43.066004 22359 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.202747 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.177713 20857 rpcz_store.cc:276] 0709 13:00:41.973366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.444668 (+471302us) service_pool.cc:224] Handling call
0709 13:00:42.446047 (+  1379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:42.527208 (+ 81161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.527249 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:42.542171 (+ 14922us) log.cc:844] Serialized 52 byte log entry
0709 13:00:43.010206 (+468035us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:43.010269 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.010371 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:43.168565 (+158194us) raft_consensus.cc:1685] finished
0709 13:00:43.168599 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.173468 (+  4869us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8748,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":7466,"thread_start_us":352416,"threads_started":1,"wal-append.queue_time_us":578618}
I20260709 13:00:43.206070 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.209475 20084 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:43.215494 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:43.216379 20084 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:43.216574 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.218125 20165 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:43.242508 20055 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:43.251502 20854 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:43.253166 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:43.255666 20162 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:43.261848 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.262912 20857 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:43.277420 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.281778 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.290257 20091 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:43.299518 20086 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:43.269250 22326 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:43.303890 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.307374 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.276677 22281 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:43.315560 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 81253us
I20260709 13:00:43.318619 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:43.320006 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 143778us
I20260709 13:00:43.320900 22497 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:43.321560 22567 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.471s	user 0.004s	sys 0.002s
I20260709 13:00:43.321960 22320 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:43.322692 22281 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.323712 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:43.325133 22546 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:43.325896 22281 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:43.328307 20744 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:43.335726 20856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.034s	sys 0.084s
I20260709 13:00:43.297856 22366 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:00:43.336802 20856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.035s	sys 0.084s
I20260709 13:00:43.335528 20744 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:43.340920 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.343135 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:43.350989 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 117202us
I20260709 13:00:43.344173 22546 raft_consensus.cc:2764] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:00:43.315132 22369 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:43.334102 20165 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:43.331401 22326 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:43.355366 20159 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:43.338955 22366 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:43.379005 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:43.374060 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:43.355826 22281 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:43.397867 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 144761us
I20260709 13:00:43.400830 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.401319 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:43.401486 20852 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:43.409241 20163 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:43.400347 20161 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
W20260709 13:00:43.415063 22563 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.274s	user 0.002s	sys 0.000s
W20260709 13:00:43.421487 22500 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
I20260709 13:00:43.403182 22281 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:43.412827 22369 raft_consensus.cc:2764] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:00:43.431365 20129 leader_election.cc:400] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 4 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 4. Current term is 5.
I20260709 13:00:43.429527 22326 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:43.437132 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:43.439146 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66744.2us
I20260709 13:00:43.420230 22207 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:43.423197 22499 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:43.431734 22574 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:00:43.448225 20084 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:43.411340 22483 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:00:43.449967 22315 consensus_meta.cc:287] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.950s	user 0.010s	sys 0.006s
I20260709 13:00:43.441498 22359 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:43.473515 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.472527 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.464910 20157 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:43.473868 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.479686 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.480476 20853 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
W20260709 13:00:43.467680 22482 consensus_meta.cc:287] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.905s	user 0.012s	sys 0.003s
I20260709 13:00:43.485488 22482 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:43.489619 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 89478.7us
I20260709 13:00:43.456245 22207 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:43.472030 22571 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:00:43.489620 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 277975us
W20260709 13:00:43.470765 22115 consensus_meta.cc:287] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.858s	user 0.011s	sys 0.003s
W20260709 13:00:43.472702 22577 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 13:00:43.496876 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46187.9us
I20260709 13:00:43.499009 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 821) took 1396 ms. Trace:
I20260709 13:00:43.455245 22499 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:43.524555 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 88711.3us
I20260709 13:00:43.506502 22359 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:43.528872 22359 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.504895 22115 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.529317 22359 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.529241 22207 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.522734 22281 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:43.495038 22320 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:43.535609 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.534616 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.542037 20086 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:43.538724 20090 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:43.546129 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.547091 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:43.476714 22315 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:43.515878 22546 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.500835 20856 rpcz_store.cc:276] 0709 13:00:42.102228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.472856 (+370628us) service_pool.cc:224] Handling call
0709 13:00:42.518943 (+ 46087us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:42.540592 (+ 21649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.540622 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:42.629507 (+ 88885us) log.cc:844] Serialized 52 byte log entry
0709 13:00:43.355446 (+725939us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.369589 (+ 14143us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.369887 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.370019 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:43.480711 (+110692us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b18002cb300. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:43.491109 (+ 10398us) raft_consensus.cc:1685] finished
0709 13:00:43.494771 (+  3662us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.498914 (+  4143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13937,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":15098,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":15913,"spinlock_wait_cycles":10580736,"thread_start_us":677502,"threads_started":1,"wal-append.queue_time_us":780598}
W20260709 13:00:43.559976 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 374312us
I20260709 13:00:43.561434 20858 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:43.568881 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:43.548062 22275 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.015s	user 0.012s	sys 0.006s
I20260709 13:00:43.527235 22581 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:43.569072 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.531833 22283 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:43.536841 22507 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:43.576771 20089 raft_consensus.cc:2368] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:43.569604 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.582793 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.584813 20091 raft_consensus.cc:2368] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9.
I20260709 13:00:43.587891 20085 raft_consensus.cc:2410] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:00:43.592135 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 76602.2us
W20260709 13:00:43.593163 22572 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 13:00:43.593240 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:43.597790 22315 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:43.614526 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49274.1us
W20260709 13:00:43.616786 22579 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:00:43.618036 22320 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:43.624050 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:43.576948 22582 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.634550 22462 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:43.582774 22275 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:43.656948 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.657508 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.658545 20854 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:43.662199 22527 consensus_meta.cc:287] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.005s
I20260709 13:00:43.590895 22581 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:43.571233 22546 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:43.638711 22463 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.820s	user 0.016s	sys 0.002s
I20260709 13:00:43.587129 22542 raft_consensus.cc:2764] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:00:43.665082 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:43.631286 22507 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:43.648717 22582 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:43.670367 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:43.675074 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.675856 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:43.686636 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:43.679386 22582 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:43.688899 22499 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:43.681138 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:43.690249 20086 raft_consensus.cc:2410] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
W20260709 13:00:43.691187 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67318.4us
W20260709 13:00:43.693931 22094 consensus_meta.cc:287] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.003s
I20260709 13:00:43.694270 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 811) took 1038 ms. Trace:
I20260709 13:00:43.694456 22094 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:43.694895 20858 rpcz_store.cc:276] 0709 13:00:42.661431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.664847 (+  3416us) service_pool.cc:224] Handling call
0709 13:00:43.694010 (+1029163us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b540000a5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:43.694187 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2007518464}
I20260709 13:00:43.696120 22587 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:43.698259 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:43.699080 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.670431 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:43.682483 20854 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:43.707459 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 122528us
I20260709 13:00:43.712754 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:43.671108 22584 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:00:43.717586 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:43.723781 20161 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:43.725756 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:43.726549 20163 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:43.731808 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.732870 20854 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:43.688987 22582 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.744978 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:43.745693 20857 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:43.746502 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.706940 22542 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.676146 22315 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:43.675097 22546 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:43.784257 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59979.6us
I20260709 13:00:43.783874 20163 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:43.675210 22463 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.798736 20852 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:43.803392 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 53629.7us
I20260709 13:00:43.803568 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:43.749320 22275 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:43.809062 20854 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:43.697113 22366 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:43.825163 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 239959us
I20260709 13:00:43.684247 22581 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:43.839681 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:43.806565 22589 raft_consensus.cc:2764] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:00:43.841498 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:43.782538 22582 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:43.847579 20163 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.852779 22587 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:43.783238 22542 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.861951 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:43.862783 20157 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.808323 22546 consensus_queue.cc:1059] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:00:43.862336 20744 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:43.870020 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 197080us
I20260709 13:00:43.875979 20163 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:43.837733 22359 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:43.882458 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:43.861073 22115 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:43.888397 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 56050.7us
I20260709 13:00:43.866508 22275 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.867848 22094 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:43.890054 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.904822 22590 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:00:43.910362 22601 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.871466 22366 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:43.912201 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:43.918475 22604 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.918221 22275 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:43.917161 22465 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:43.925664 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:43.929340 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 55636.2us
I20260709 13:00:43.931011 22582 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:43.932516 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:43.934029 22207 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:43.942615 20857 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:43.896072 20157 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:43.883929 22315 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:43.951238 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:43.959185 20743 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:43.947880 22584 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:00:43.970057 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:43.978497 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:43.976788 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:43.987520 20014 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:43.989763 20017 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:43.990969 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:43.952095 22452 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:43.958364 22465 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.934666 22587 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:44.003376 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:44.002368 22465 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:44.014371 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.020717 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.021404 20011 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.022591 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:44.023361 20855 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:44.015650 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 83247.3us
I20260709 13:00:43.965217 22207 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:43.974437 22589 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:44.027352 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.026721 20014 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:44.027743 20157 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:43.975183 22600 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:43.949039 22601 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.030138 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.032922 20158 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:44.034018 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:43.988747 22281 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.031759 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.034651 20016 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.031109 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:44.041963 20850 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.037416 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:44.042804 20857 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.038782 22275 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:44.002784 22452 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.040660 20858 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:44.040571 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:43.958639 22604 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.056756 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:44.058499 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:44.031459 22207 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.095037 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:43.986620 22315 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.054823 20159 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:44.108664 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.031808 22589 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:44.108731 20013 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:44.109649 20090 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:44.111084 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.106951 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.040452 22546 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:44.051177 22281 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.124648 20086 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:44.016782 22611 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 13:00:44.051963 22609 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.095098 22275 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:44.055017 22452 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.094612 22378 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:44.118531 20857 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:44.138370 20014 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:44.128285 20017 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:44.142957 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.041953 22463 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.153827 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.114261 22589 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.129412 22281 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.162005 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:44.165438 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 143091us
W20260709 13:00:44.165308 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106867us
I20260709 13:00:44.167358 20855 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:44.144038 22609 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.171356 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.180825 20010 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.174010 20013 raft_consensus.cc:2410] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:00:44.201253 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.202170 20014 raft_consensus.cc:2368] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
W20260709 13:00:44.173702 22602 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:00:44.165653 22581 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:44.180704 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.237497 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.175107 22611 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:44.181862 22609 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.239682 20009 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:44.179399 22546 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:44.160740 22275 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:44.239331 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46007.6us
I20260709 13:00:44.261824 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.262728 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:44.263164 20010 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.191372 22359 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:44.243570 20011 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:44.202189 22542 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.260465 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.228427 22585 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:44.255204 22499 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:44.266716 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.286080 20129 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:44.277904 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.304965 20009 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:00:44.310951 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 137621us
I20260709 13:00:44.315128 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:44.287376 22462 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.287740 22598 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:44.330639 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.329098 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.338542 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.339111 20011 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:00:44.308686 22603 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:00:44.257169 22611 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.342962 20014 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:44.351663 20012 raft_consensus.cc:2368] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:44.361603 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.362352 20011 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 2.
I20260709 13:00:44.363310 20090 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:44.308760 22585 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:44.369338 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.370134 20011 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
W20260709 13:00:44.355647 20744 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 1.602s (SENT) [suppressed 14 similar messages]
I20260709 13:00:44.366127 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:44.369408 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 105623us
I20260709 13:00:44.337872 22462 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.382030 22483 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:44.382887 22283 raft_consensus.cc:3055] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:44.397146 22283 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:44.398166 22283 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.384943 22315 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:44.385648 22482 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.400851 22616 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:44.399787 22283 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.401485 22616 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:44.406687 22613 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:00:44.340628 22542 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.400254 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.345952 22600 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:44.411471 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.309966 22499 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.364895 22326 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.415577 20016 raft_consensus.cc:2410] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:44.430153 20855 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:44.430877 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.422892 20014 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:44.430992 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:44.441501 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 126520us
W20260709 13:00:44.395275 22507 consensus_meta.cc:287] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.718s	user 0.016s	sys 0.000s
I20260709 13:00:44.443187 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:44.434108 22320 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.445138 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:44.369776 22601 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.396135 22462 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.453100 20856 raft_consensus.cc:2368] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7.
I20260709 13:00:44.457733 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.454856 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.460569 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:44.403775 22616 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.466181 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:44.467067 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.468921 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.475461 20159 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:44.473971 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:44.475394 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.476305 20130 leader_election.cc:400] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:44.476930 20130 leader_election.cc:403] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:44.477228 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.479326 20158 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:44.354780 22598 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:44.480652 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.458794 22507 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:44.483587 20009 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:44.456557 22604 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.495683 20016 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:44.494022 20164 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:00:44.489218 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.477896 20162 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:44.474364 22482 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.499504 20130 leader_election.cc:400] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:44.507032 20130 leader_election.cc:403] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:44.469823 22631 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:44.504768 20858 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:44.470582 22378 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:44.498498 22463 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.453372 22326 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:44.511679 22615 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:00:44.532682 20850 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:44.533672 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.534495 20009 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:44.510924 22275 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.523048 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.535274 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.541774 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.543293 20010 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:44.470566 22320 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.527495 22598 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:44.509204 22482 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:44.438838 22546 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.522346 22382 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:44.478289 22585 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.505259 22588 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4.
I20260709 13:00:44.509320 22208 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:00:44.498950 22611 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.456247 22600 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.523847 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.548316 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.500204 22499 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.549165 22281 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:44.535223 22609 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:44.496119 22629 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:44.553323 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 84444.8us
I20260709 13:00:44.600502 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.523759 22378 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.577565 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.571544 22631 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:44.606366 20852 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:44.607540 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.542574 20009 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:44.587935 20858 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
W20260709 13:00:44.528637 22465 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.002s
I20260709 13:00:44.593492 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:44.594828 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.594759 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 71117.4us
I20260709 13:00:44.597426 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:44.611560 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.593593 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.603415 22587 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.601356 20856 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:44.611954 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:44.594400 22572 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:00:44.605067 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.581204 22482 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.578894 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.573515 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.612298 20091 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:44.617202 20010 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:44.591904 22452 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.614900 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.616456 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:44.612468 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.616914 22616 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:00:44.620296 22320 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:44.612790 22465 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.623512 22465 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.624073 22608 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:44.626293 22378 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.628042 20090 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:44.619508 22621 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:44.609504 22631 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:44.618381 20083 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:44.632649 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.630982 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.638557 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.640275 20085 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:44.640566 20087 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:44.641821 22208 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.645556 20082 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.640148 20091 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:44.649058 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.649726 20010 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:00:44.654703 22465 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.656632 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:44.657856 20009 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:44.659214 20015 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:44.660157 20850 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:44.661270 22452 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.640012 22621 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.663794 22621 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.665272 22281 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:44.666787 22609 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.636304 22608 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:44.667659 22482 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.668192 22608 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.668812 20086 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:44.674873 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 58510.1us
I20260709 13:00:44.675182 22281 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:44.675772 22281 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.676133 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.676927 20016 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:44.669009 22208 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:44.655299 22465 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.693747 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:44.694509 20158 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:44.692448 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.696880 22482 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.697338 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.705679 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.706373 20160 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:44.706826 20162 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:44.705480 22621 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:44.709947 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:44.711740 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:44.697054 20159 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:44.712790 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.715555 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:44.716588 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.716928 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.716948 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.717916 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:44.718139 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:44.720108 20010 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:44.717212 20009 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:44.721903 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.722947 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.729000 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.713681 22639 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:44.717725 22452 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:00:44.725247 20056 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:44.715817 22482 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.741606 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:44.718978 22463 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.718935 22094 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.747208 22606 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:44.757282 20056 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:44.724380 22595 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:44.732771 22208 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:44.757095 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:44.740823 22589 consensus_meta.cc:287] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.569s	user 0.006s	sys 0.006s
I20260709 13:00:44.765738 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:44.744057 22382 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:44.770951 20089 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
W20260709 13:00:44.756204 22366 consensus_meta.cc:287] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.806s	user 0.005s	sys 0.007s
I20260709 13:00:44.741381 22378 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.746021 22616 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:44.723853 22614 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:00:44.778367 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.782781 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:44.792937 20083 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:44.784288 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:44.793445 20011 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:00:44.785682 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:44.746111 22587 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:00:44.797478 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:44.771922 22208 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:44.807080 20083 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:44.771555 22462 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.745745 22320 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:00:44.794868 22359 raft_consensus.cc:2764] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:44.792783 22595 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:44.801554 22465 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.802117 22600 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:44.802894 22207 consensus_meta.cc:287] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.000s
I20260709 13:00:44.803792 22631 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.774456 22581 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:44.773818 22588 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:44.835803 22207 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.831292 22595 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.837338 22595 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.838155 22207 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.838819 20163 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.781694 22589 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.840371 20159 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:44.840610 20157 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.829938 22366 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:44.841946 22366 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.842291 22366 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.842443 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.843595 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.844491 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.845862 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.847170 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.850278 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.809744 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:44.811182 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:44.812350 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:44.799530 22645 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:00:44.815554 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.815554 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:44.822430 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.861974 20855 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:44.771231 22598 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:00:44.817183 22275 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:44.821635 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:44.787124 22585 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:44.829660 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:44.829043 22462 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:44.833554 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:44.873108 20158 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.833846 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.814303 22208 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.877966 20857 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:44.857319 20083 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:44.864603 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.864919 19981 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:44.880287 20853 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:44.865723 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:44.870314 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 48778.6us
I20260709 13:00:44.876510 22589 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:44.883195 20015 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:44.887701 20157 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:44.889223 20858 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:44.882866 22463 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:44.862785 22588 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:44.895974 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.896878 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:44.898989 20857 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:44.907239 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:44.907756 20159 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:44.908128 20858 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:44.877404 22465 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:44.877526 22585 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:44.913268 22588 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.913967 22585 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:44.883091 22378 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:00:44.926162 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 61319us
I20260709 13:00:44.919523 22616 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:44.896873 22359 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:44.928217 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 112759us
W20260709 13:00:44.931490 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 100344us
I20260709 13:00:44.892441 22463 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:44.872846 22275 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:44.944187 20129 leader_election.cc:400] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7.
I20260709 13:00:44.954348 20129 leader_election.cc:403] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:00:44.956439 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:44.973277 20129 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:44.981611 22595 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:44.933399 22581 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:44.984098 22207 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7.
I20260709 13:00:44.919515 22208 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:44.933388 22465 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:44.953951 22463 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.992791 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:44.993602 20165 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.986375 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:45.010463 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:45.011363 20160 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:44.995544 20158 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:45.013756 20162 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:45.015753 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56715.5us
I20260709 13:00:45.015846 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:45.026360 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.026572 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:45.030282 19981 leader_election.cc:400] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9.
I20260709 13:00:45.030781 19981 leader_election.cc:403] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:45.040385 22465 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:45.043576 22208 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:45.044123 22616 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:45.046476 22275 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:45.010076 22378 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:45.057355 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:45.058727 22653 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:00:45.028057 22607 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:45.090937 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:45.091948 20161 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:45.062049 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:45.066990 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 49941.9us
I20260709 13:00:45.068744 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:45.017170 22382 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.108733 22326 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:45.075142 22281 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:45.060422 22482 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:45.086655 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:45.095558 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:45.095890 20083 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:45.096830 20157 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:44.985028 22581 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:45.032976 22595 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:45.033537 22629 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:45.055102 22369 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:45.069279 22598 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9.
I20260709 13:00:45.112587 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 975) took 1102 ms. Trace:
I20260709 13:00:45.152868 20163 rpcz_store.cc:276] 0709 13:00:44.010549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.010733 (+   184us) service_pool.cc:224] Handling call
0709 13:00:44.011155 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.839209 (+828054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.112223 (+273014us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b5400573354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:45.112489 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1903031552}
W20260709 13:00:45.142895 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 106648us
W20260709 13:00:45.097499 22637 log.cc:927] Time spent T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.541s	user 0.000s	sys 0.003s
I20260709 13:00:45.052542 20055 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:45.033174 22609 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:45.170615 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 118166us
I20260709 13:00:45.138775 22607 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.167672 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:45.170627 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:45.140424 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:45.180990 20013 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:45.150374 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.182358 20160 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:45.064502 22378 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.165445 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:45.168483 20156 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:00:45.123333 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:45.183866 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:45.179208 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:45.184754 20011 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:45.145030 22507 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.179291 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:45.184337 22302 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:45.220248 22622 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:45.220715 22482 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:45.222540 22482 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:45.222829 22621 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:45.223608 22621 consensus_queue.cc:1243] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:45.223515 22326 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:45.224934 22610 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:45.226102 22610 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:45.226292 22598 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:45.227072 22610 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:45.228062 22625 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:45.228467 22625 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:45.229344 22625 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.230612 22625 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.232230 22596 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:45.232640 22596 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:45.194967 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:45.195315 20165 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
W20260709 13:00:45.235045 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 111846us
I20260709 13:00:45.236577 20164 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:45.198539 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:45.191449 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:45.222977 22482 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:45.257138 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 62297.6us
W20260709 13:00:45.261797 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 70447.3us
I20260709 13:00:45.263727 22636 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:45.264298 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:45.264767 22636 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:45.267843 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:45.268672 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:45.259884 20859 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:45.269798 20852 raft_consensus.cc:2368] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
I20260709 13:00:45.269915 22600 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.274057 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:45.270937 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 72534.5us
I20260709 13:00:45.276115 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:45.259419 22482 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.271324 22326 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:45.166975 22589 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:45.297935 19981 leader_election.cc:400] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:45.299882 19981 leader_election.cc:403] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:45.185911 22607 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:45.302623 20013 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:45.312873 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:45.316612 20856 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.187124 22378 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.199326 22586 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:45.234233 22608 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:45.326294 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:45.327484 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 67005us
I20260709 13:00:45.284498 22369 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.284248 22462 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:45.258189 22302 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:45.285193 22598 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.200202 22595 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.124s
I20260709 13:00:45.186271 22359 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.136s
I20260709 13:00:45.301482 22281 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.200042 22585 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.125s
I20260709 13:00:45.185732 22609 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:45.152130 22629 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:45.378155 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 155752us
I20260709 13:00:45.172451 22591 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.153s
I20260709 13:00:45.320017 22621 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:45.390542 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 114531us
I20260709 13:00:45.324921 22507 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:45.345167 22611 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:45.376158 22369 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:45.394819 22656 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.398726 19981 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:45.328467 22589 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.349378 22462 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:45.330224 22608 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.325145 22283 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:00:45.375635 22598 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:45.381181 22581 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.380111 22606 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:45.405215 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 131234us
I20260709 13:00:45.432866 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1031) took 1266 ms. Trace:
I20260709 13:00:45.233515 22596 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:45.453150 20017 rpcz_store.cc:276] 0709 13:00:44.166162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.166298 (+   136us) service_pool.cc:224] Handling call
0709 13:00:44.174393 (+  8095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.174621 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:45.069336 (+894715us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b18004be4a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf
0709 13:00:45.069554 (+   218us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.432398 (+362844us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.432468 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:45.432526 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.432660 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.432800 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18226,"prepare.queue_time_us":88428,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":11473,"raft.queue_time_us":35417,"spinlock_wait_cycles":6700288,"thread_start_us":353183,"threads_started":2}
I20260709 13:00:45.431631 22656 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:45.437371 22608 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.398036 22281 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:45.433565 22462 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:45.547746 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 142394us
W20260709 13:00:45.466372 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 140121us
I20260709 13:00:45.426848 22507 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:45.459761 22611 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.428337 22621 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.471702 22606 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:45.681015 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:45.630697 22596 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:45.736594 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 349882us
I20260709 13:00:45.741109 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:45.609665 22302 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:45.784730 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 331890us
W20260709 13:00:45.785830 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44834.9us
I20260709 13:00:45.788264 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:45.790592 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:45.780323 22463 consensus_meta.cc:287] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.761s	user 0.021s	sys 0.004s
I20260709 13:00:45.751227 22594 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 13:00:45.802852 22679 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:00:45.799260 22675 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:45.804970 22369 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:45.805672 22675 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:45.806185 22675 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:45.808980 22676 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:00:45.810294 22281 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:45.794243 22681 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.791834 22359 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:45.793458 22611 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:45.791616 22606 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:45.824405 20856 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.507s	user 0.003s	sys 0.009s
I20260709 13:00:45.836131 20856 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:45.804934 22621 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:45.794860 22463 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:45.867054 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 49659.3us
I20260709 13:00:45.816816 22609 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:45.764760 22656 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.817674 22611 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:45.869987 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:45.820225 22669 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
W20260709 13:00:45.876415 22667 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.410s	user 0.001s	sys 0.000s
W20260709 13:00:45.877167 22674 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:00:45.881269 22601 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:45.834913 22359 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:45.820726 22594 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:00:45.886386 22611 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:45.893131 22601 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:45.817836 22681 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:45.895292 22507 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:45.898293 22586 raft_consensus.cc:2764] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:00:45.903523 22482 consensus_meta.cc:287] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.625s	user 0.021s	sys 0.000s
W20260709 13:00:45.912360 20129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.011s	sys 0.013s
I20260709 13:00:45.901630 22656 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.916927 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:45.919461 20129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.012s	sys 0.013s
I20260709 13:00:45.902681 22369 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.920778 20129 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:45.921767 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:45.920382 22326 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.000s
I20260709 13:00:45.928848 22684 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:45.930100 22677 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:00:45.935123 22607 consensus_meta.cc:287] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.001s
I20260709 13:00:45.936322 22586 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.937466 22463 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:45.943055 22378 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.000s
I20260709 13:00:45.924583 22542 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:45.927345 22369 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:45.944855 22542 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:45.945271 22542 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.946537 22369 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:45.948204 22601 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:45.949751 22601 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:45.949831 22629 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:45.950822 22359 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:45.928907 22326 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:45.951864 22546 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:45.960819 22546 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.961274 22546 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.935608 22607 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:45.952461 22585 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:45.943617 22378 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.964918 22585 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:45.965117 22607 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:45.966027 22585 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.966806 22378 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:45.964279 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 47472.9us
I20260709 13:00:45.967787 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:45.952967 22283 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.968009 22462 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:45.968669 22283 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:45.969508 22582 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:45.969746 22283 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:45.970412 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:45.971987 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:45.972117 22687 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:45.973186 22607 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:45.973569 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:45.953210 22598 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.974893 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:45.955742 22682 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:45.976300 22625 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.745s	user 0.010s	sys 0.017s
I20260709 13:00:45.976900 22625 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.977409 22462 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:45.977516 22625 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.975476 22542 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.980258 22542 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:45.921769 22482 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:45.959134 22624 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:45.985863 22624 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:45.986980 22682 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.986579 22482 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:45.986716 22624 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:45.987504 22682 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:45.954459 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 323844us
I20260709 13:00:45.988281 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:45.988871 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:45.989373 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:45.989946 20852 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:45.990041 22682 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:45.990660 22682 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:45.991681 20855 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:45.991875 22621 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:45.993587 22589 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:45.971575 22283 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:45.997323 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.974942 22598 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.000344 22624 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:46.002035 22672 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.545s	user 0.002s	sys 0.000s
I20260709 13:00:46.000247 22684 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:46.005226 20131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.012s	sys 0.004s
I20260709 13:00:45.967600 22585 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.008579 22675 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.010741 22369 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:46.017532 22675 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.017444 22369 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.011166 22462 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:46.011864 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:46.015408 20131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.013s	sys 0.004s
I20260709 13:00:46.018462 22462 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.020308 22681 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.020795 22462 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:46.005571 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 79407.4us
I20260709 13:00:46.024699 20129 leader_election.cc:400] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
I20260709 13:00:46.025136 20129 leader_election.cc:403] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:00:46.019618 22369 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:46.025897 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:46.026479 22369 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
I20260709 13:00:46.027209 22369 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:46.027712 22369 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:46.028692 22369 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:46.029311 22366 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.186s	user 0.010s	sys 0.006s
I20260709 13:00:46.029834 22366 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.030851 20010 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:45.956578 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 787) took 1126 ms. Trace:
I20260709 13:00:46.033321 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:46.033869 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 790) took 1074 ms. Trace:
I20260709 13:00:46.034368 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:46.033416 20008 rpcz_store.cc:276] 0709 13:00:44.830116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.848703 (+ 18587us) service_pool.cc:224] Handling call
0709 13:00:44.848981 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:44.849920 (+   939us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:44.869955 (+ 20035us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.161472 (+291517us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:45.185101 (+ 23629us) log.cc:844] Serialized 26 byte log entry
0709 13:00:45.185274 (+   173us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:45.185320 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.185409 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:45.956361 (+770952us) raft_consensus.cc:1685] finished
0709 13:00:45.956380 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.956521 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3604,"prepare.queue_time_us":22571,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":5346,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":12814,"thread_start_us":232218,"threads_started":1,"wal-append.queue_time_us":404865}
I20260709 13:00:45.991693 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.038836 22625 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.039436 22281 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.039296 22625 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:46.040557 20015 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:46.040707 22625 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:46.040884 22673 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.520s	user 0.002s	sys 0.000s
W20260709 13:00:46.041043 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49419.5us
I20260709 13:00:46.042696 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:46.044181 22622 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:46.044664 22622 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:46.044875 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:46.045462 20015 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.047191 20055 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.047374 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:46.047962 20855 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:46.048620 20857 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:46.049484 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:46.049897 22315 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:46.050310 22315 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.051673 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.052497 20013 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:46.053053 22595 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:46.053951 22281 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.054507 22281 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.055013 20163 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:46.050645 22315 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:46.056599 22281 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:46.056942 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.056625 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:46.058976 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.059340 22281 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.030740 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:46.059536 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 47773.9us
I20260709 13:00:46.061138 22499 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:46.062867 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:46.063311 20163 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.063621 22499 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:46.034219 20010 rpcz_store.cc:276] 0709 13:00:44.959426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.959579 (+   153us) service_pool.cc:224] Handling call
0709 13:00:44.959884 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.031310 (+1071426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.033644 (+  2334us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54009155d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:46.033815 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2462511488}
I20260709 13:00:46.066256 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:46.069020 22499 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:46.069172 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:46.069823 22281 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.070171 22687 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:46.070855 22687 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:46.071466 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.071805 22687 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.072861 20163 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:45.748838 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:46.070127 22275 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:46.074295 22275 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:46.074975 22281 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.075174 22275 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:46.075804 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:46.076398 22275 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:46.076581 20163 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:46.078372 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:46.079113 20163 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:46.080577 20744 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.080302 19839 catalog_manager.cc:5697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 3 to 6, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 6 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:46.082099 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:46.083448 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:46.084079 22281 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:46.084323 22639 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.086577 22680 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:46.087100 22639 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.089414 22639 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:46.084935 22604 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:46.090744 22687 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.090973 22366 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.093833 22621 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.094658 22621 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.095357 22621 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.099417 22625 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.100726 20017 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.102026 22625 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:46.107740 22608 consensus_meta.cc:287] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.002s
I20260709 13:00:46.108326 22608 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.059105 20851 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:46.110824 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:46.110900 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.112123 22621 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.113023 20857 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:46.113523 19839 catalog_manager.cc:5697] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 4, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 4 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:46.114270 22621 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.114758 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:46.115634 20857 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:46.118407 19838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 158) took 15099 ms (client timeout 15000 ms). Trace:
I20260709 13:00:46.115191 19839 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.120661 22682 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:46.120462 19839 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 2 to 4, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.123106 19839 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:46.111644 20851 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:46.125445 22499 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:46.126322 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44301us
I20260709 13:00:46.154939 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:46.128660 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.131760 20157 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:46.131050 20163 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:46.132685 20164 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:46.132968 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:46.134584 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.155441 20165 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:46.165772 20015 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:46.172991 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:46.173960 20851 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.171253 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.124791 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.178498 19839 catalog_manager.cc:5697] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
W20260709 13:00:46.214851 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 60271.7us
I20260709 13:00:46.196605 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:46.215879 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:46.177403 22681 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:46.219684 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:46.197041 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 72353.1us
I20260709 13:00:46.229810 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.223950 20851 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:46.172574 22614 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.231369 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:46.234200 20851 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:46.134476 22546 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:46.242347 20851 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:00:46.241142 20056 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.025s	sys 0.042s
W20260709 13:00:46.246053 20056 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.025s	sys 0.042s
I20260709 13:00:46.248903 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:46.250454 20857 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.241906 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:46.246914 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 792471us
I20260709 13:00:46.254161 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:46.256914 20744 leader_election.cc:336] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 1.537s (SENT)
I20260709 13:00:46.188938 22611 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.258435 22675 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:46.258183 19839 catalog_manager.cc:5697] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:46.259363 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:46.255175 20156 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:46.260079 22611 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.218320 22208 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:46.260623 19839 catalog_manager.cc:5697] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.261507 22208 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:46.261972 22208 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:46.248908 22465 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:00:46.262292 19839 catalog_manager.cc:5697] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:46.264045 22681 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:46.249049 22581 consensus_queue.cc:1059] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:00:46.264787 22320 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:46.265190 22681 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.266160 22581 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:46.266912 22581 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:46.263900 19839 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:46.267285 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.215293 22281 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:46.246091 22621 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:46.268755 22482 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:46.270543 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.271139 20008 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:46.271170 20089 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:46.272324 20083 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:46.268239 19839 catalog_manager.cc:5697] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 3 to 4, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.273979 20086 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:46.275015 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.271170 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.228312 22207 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.276773 20087 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:46.141585 19838 rpcz_store.cc:269] 0709 13:00:31.019321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.019489 (+   168us) service_pool.cc:224] Handling call
0709 13:00:32.712349 (+1692860us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54008b1fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71c090 00007fa02e7baf3a 00007fa0345711cc 00007fa03466f37e 00007fa032b302a4 00007fa032b30232 00007fa032b30161 00007fa032b300dc 00007fa032b2f392 00007fa02caed48c 00007fa02caec8c6
0709 13:00:46.118183 (+13405834us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:41.417751 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:41.418452 (+   701us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:41.418500 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:00:41.418521 (+    21us) tablet.cc:662] Decoding operations
0709 13:00:41.585832 (+167311us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:41.589093 (+  3261us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:41.602825 (+ 13732us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:00:42.068217 (+465392us) tablet.cc:701] Row locks acquired
0709 13:00:42.068266 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:00:42.089108 (+ 20842us) write_op.cc:270] Start()
0709 13:00:42.102655 (+ 13547us) write_op.cc:276] Timestamp: P: 1783602042085324 usec, L: 0
0709 13:00:42.102683 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:00:42.140386 (+ 37703us) log.cc:844] Serialized 6305 byte log entry
0709 13:00:42.458907 (+318521us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b18001301e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:42.808426 (+349519us) op_driver.cc:464] REPLICATION: finished
0709 13:00:43.377146 (+568720us) write_op.cc:301] APPLY: starting
0709 13:00:43.377319 (+   173us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:43.377854 (+   535us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:43.377903 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:43.438745 (+ 60842us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:43.504082 (+ 65337us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:00:43.513076 (+  8994us) write_op.cc:312] APPLY: finished
0709 13:00:43.592442 (+ 79366us) log.cc:844] Serialized 112 byte log entry
0709 13:00:43.592705 (+   263us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:43.593213 (+   508us) write_op.cc:454] Released schema lock
0709 13:00:43.593318 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:43.593883 (+   565us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:43.593919 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3454939,"reported_tablets":26,"spinlock_wait_cycles":327920896,"child_traces":[["op",{"apply.queue_time_us":556282,"apply.run_cpu_time_us":7782,"apply.run_wall_time_us":216900,"num_ops":16,"prepare.queue_time_us":558980,"prepare.run_cpu_time_us":15132,"prepare.run_wall_time_us":741330,"raft.queue_time_us":273204,"raft.run_cpu_time_us":41189,"raft.run_wall_time_us":466257,"replication_time_us":663005,"spinlock_wait_cycles":12603136,"thread_start_us":903869,"threads_started":4,"wal-append.queue_time_us":20047,"wal-append.run_cpu_time_us":41146,"wal-append.run_wall_time_us":2139926}]]}
I20260709 13:00:46.280937 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.254968 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.281863 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.285275 20163 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.285547 20010 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:46.275373 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:46.217207 22656 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:46.218544 22639 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:46.285636 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.286767 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.287899 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:46.302632 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:46.304066 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.240376 22595 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:46.309348 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.310232 20013 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:46.310890 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:46.292454 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.229116 22499 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:00:46.299578 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.305084 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:46.301088 20016 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:46.306320 20130 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:46.306350 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.335247 20743 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.306778 20859 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:46.283593 20089 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:46.244124 22598 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.313509 20011 raft_consensus.cc:2410] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:46.313033 20013 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:00:46.336858 20013 pending_rounds.cc:85] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 5
I20260709 13:00:46.325217 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.337535 20158 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.283841 22608 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:46.337148 20013 pending_rounds.cc:107] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:00:46.348357 20013 raft_consensus.cc:2889] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:46.325764 20163 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:46.350088 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.350867 20163 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:46.239780 22596 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:00:46.332803 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.356930 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:46.357681 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.364753 20016 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:46.357004 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.365793 20014 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:46.365885 20015 raft_consensus.cc:2410] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:00:46.196560 22695 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.141s
I20260709 13:00:46.367020 20856 raft_consensus.cc:2410] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:46.298297 22482 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:46.249684 22625 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:00:46.249754 22682 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:00:46.368274 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:46.368789 20014 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:46.369599 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:46.384131 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 127258us
I20260709 13:00:46.371906 20012 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:00:46.384917 20744 leader_election.cc:336] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 1.602s (SENT)
I20260709 13:00:46.377815 20165 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:46.288522 22366 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:00:46.339167 22598 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:46.338706 22684 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:46.385599 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:46.329107 22639 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:46.384207 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.390337 20016 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:46.391649 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:46.307891 22359 consensus_queue.cc:1059] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:00:46.397001 20161 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:46.363314 22621 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.245535 22614 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:46.298465 22675 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:00:46.324271 22378 consensus_queue.cc:1059] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:46.307298 22656 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.388363 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.413756 22694 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:00:46.414196 20161 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:46.333469 22595 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.415892 22507 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:46.467722 20087 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:46.418190 22382 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:46.415956 22463 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.419185 22462 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:46.420389 22359 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:46.420558 22688 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:46.420929 22684 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:46.421026 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 77687.1us
I20260709 13:00:46.397116 22611 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:00:46.415390 20086 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:46.388209 22622 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.435333 20016 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:00:46.380352 22465 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:46.341765 22315 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:46.380139 22693 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.122s
W20260709 13:00:46.445379 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53861.6us
I20260709 13:00:46.399837 19839 catalog_manager.cc:5697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 2, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.402396 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:46.404771 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 784) took 1524 ms. Trace:
I20260709 13:00:46.529033 20012 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:46.451442 20085 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:46.472734 20857 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:46.530794 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:46.473403 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.400516 22598 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.399878 22499 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:46.482580 22482 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:46.537447 19839 catalog_manager.cc:5697] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 1 to 4, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.484149 20090 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:46.489795 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:46.490134 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.490911 20014 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:46.491487 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:46.543404 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.496904 20158 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:46.501045 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.413745 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:46.544806 20160 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:46.347247 22581 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.528944 20854 rpcz_store.cc:276] 0709 13:00:44.861200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.861301 (+   101us) service_pool.cc:224] Handling call
0709 13:00:44.870770 (+  9469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.870965 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:45.151431 (+280466us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.328365 (+176934us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.951501 (+623136us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800279040. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:00:46.008057 (+ 56556us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b1800279040. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6932a6 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1
0709 13:00:46.038051 (+ 29994us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b1800279040. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:46.040130 (+  2079us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:46.385043 (+344913us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:46.385092 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:46.385143 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.385324 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:46.385446 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":647114,"prepare.queue_time_us":67306,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":78,"raft.queue_time_us":624016,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":25889,"spinlock_wait_cycles":40240768,"thread_start_us":849134,"threads_started":3,"wal-append.queue_time_us":274363}
I20260709 13:00:46.493497 22462 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:46.468775 22281 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:46.490558 22585 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.544138 20087 raft_consensus.cc:2368] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
I20260709 13:00:46.525637 20016 pending_rounds.cc:85] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 4
I20260709 13:00:46.548837 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:46.484687 22588 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.539332 19839 catalog_manager.cc:5697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:46.543635 20082 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:46.521646 22283 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.010s
I20260709 13:00:46.501389 22359 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:46.566985 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 182087us
I20260709 13:00:46.536773 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:46.569125 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.591593 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:46.577333 20016 pending_rounds.cc:107] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:00:46.593067 20016 raft_consensus.cc:2889] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:46.519074 22546 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:46.598489 20012 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:46.598768 20129 leader_election.cc:400] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
I20260709 13:00:46.602849 20853 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:46.500965 22463 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.600629 20089 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:46.606735 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:46.606694 19839 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:46.537078 22499 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:46.616464 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:46.560911 22378 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.565716 22606 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:46.623845 20090 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:46.533599 22675 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:00:46.621284 19839 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:00:46.593057 22581 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:46.605715 22624 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.605s	user 0.021s	sys 0.001s
W20260709 13:00:46.599754 22275 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.014s
I20260709 13:00:46.576866 22623 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:46.520969 22688 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:46.727463 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 136616us
I20260709 13:00:46.593662 22601 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:46.728145 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:46.734318 20850 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:46.632371 22695 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:46.740377 19839 catalog_manager.cc:5697] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 2 to 4, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 4 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:46.636736 22606 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:46.746487 20163 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:46.744726 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.600180 22588 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.567632 22639 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.747730 20087 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:46.729825 22507 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.634408 22587 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 13:00:46.768298 22281 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.770078 22692 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:46.776754 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.778277 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.770687 22692 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.792795 22692 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:46.793373 22692 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.793812 22625 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:46.794317 22625 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:46.720468 22378 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.795224 22625 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.752151 22275 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.824595 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:46.827947 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49729.6us
I20260709 13:00:46.831883 20855 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.833077 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:46.834940 20744 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:46.718303 22684 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.741457 22601 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:46.796244 22283 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:46.840749 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.843637 20744 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:46.735152 22623 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.836288 22709 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.805787 22546 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:46.805088 22711 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:46.746541 22695 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.746709 22482 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.771775 22507 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.805711 22624 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:00:46.793821 22692 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.843652 22606 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.875597 22719 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.876295 22719 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:46.850453 22595 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:46.862727 22688 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.862991 22696 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:46.864032 22710 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:00:46.849661 22682 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:46.850080 22546 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.920063 22546 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.920635 22382 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:46.842069 22275 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.921245 22382 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:46.922665 22275 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.923270 22382 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:46.856106 22656 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:46.886572 22706 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:00:46.796407 22499 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:46.925377 22499 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.862128 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:46.889128 22601 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:46.928722 22275 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.889539 22588 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:46.871449 22315 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:46.914680 22625 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:46.930171 22601 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.930701 22601 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:46.888379 22623 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.933511 22601 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:46.928788 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66761us
I20260709 13:00:46.914927 20855 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:46.934893 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:46.936028 22275 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:46.938340 22275 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:00:46.947790 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 429942us
I20260709 13:00:46.875613 22606 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.950594 20744 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:46.913875 22687 consensus_meta.cc:287] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.822s	user 0.009s	sys 0.013s
I20260709 13:00:46.962725 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:46.963768 20156 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:46.965777 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:46.955188 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.884745 22622 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:46.891481 22507 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:46.880959 22711 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:46.985167 20165 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.014s	sys 0.009s
I20260709 13:00:46.997843 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:46.955710 22326 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.016s	sys 0.005s
W20260709 13:00:46.999060 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.986532 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:46.987895 20159 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:46.995795 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45315.8us
W20260709 13:00:46.985539 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 351592us
I20260709 13:00:47.001216 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:46.948603 22719 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:47.005765 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:47.007754 20744 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:47.011612 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:47.012432 20744 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:46.993986 22687 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:46.999087 22326 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.016s	sys 0.006s
I20260709 13:00:46.993836 22320 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.016625 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.959879 22601 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:00:47.071852 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.085386 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:46.993880 22586 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:47.088706 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:47.008320 22622 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.081545 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 80391.8us
I20260709 13:00:47.060133 22624 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.113298 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:47.107262 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:47.025830 22682 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:47.010984 22675 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:00:47.095788 22207 consensus_meta.cc:287] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.796s	user 0.018s	sys 0.002s
I20260709 13:00:47.119182 20017 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
W20260709 13:00:47.118198 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 101749us
W20260709 13:00:47.131228 20744 leader_election.cc:336] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 1.666s (SENT)
I20260709 13:00:47.131954 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:47.120976 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:47.007130 22631 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 13:00:47.007557 22587 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:00:47.015431 22688 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:00:47.081297 22721 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:47.052294 22729 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:47.110513 22625 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:00:47.146777 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.115784 22624 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.141734 22207 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:47.159098 22601 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.137952 22281 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:47.135267 22709 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.105158 22728 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:47.146297 22680 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:47.148911 22675 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:47.173409 22208 consensus_meta.cc:287] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.911s	user 0.014s	sys 0.002s
I20260709 13:00:47.105297 22320 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:47.215766 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 108591us
I20260709 13:00:47.119343 22687 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.121414 22622 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:47.219743 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.171535 22624 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.171741 22721 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:47.180639 22601 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.172655 22696 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:47.183619 22625 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:47.232285 22320 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:47.217897 22713 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.233771 22622 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:47.235428 22687 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:47.236402 22703 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:00:47.178474 22656 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:47.175608 22281 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:47.250167 22684 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.250465 22700 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.633s	user 0.003s	sys 0.000s
I20260709 13:00:47.251271 20164 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:47.251956 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.287777 20159 raft_consensus.cc:2410] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:47.252830 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.184689 22680 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:47.223870 22728 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:47.240222 22705 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:00:47.289798 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 267442us
I20260709 13:00:47.223351 22208 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:47.302311 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.269975 22740 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:47.304029 20164 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:47.280114 22281 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.277967 22695 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:47.277451 22656 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:47.308096 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.302883 22595 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:00:47.308668 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:47.308647 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:47.307421 22740 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:47.304667 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 173482us
I20260709 13:00:47.293437 22680 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:47.315809 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.315246 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:47.315621 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:47.318754 20161 raft_consensus.cc:606] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:47.319159 20161 raft_consensus.cc:614] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:47.297863 22705 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:47.320884 22705 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:47.321501 22731 log.cc:927] Time spent T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 13:00:47.322476 22727 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:00:47.325275 22601 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:47.326421 22737 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:00:47.327939 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 839) took 1224 ms. Trace:
I20260709 13:00:47.328323 20855 rpcz_store.cc:276] 0709 13:00:46.103252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.935169 (+831917us) service_pool.cc:224] Handling call
0709 13:00:46.935627 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.936405 (+   778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.936449 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.937135 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.996354 (+ 59219us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:47.106322 (+109968us) log.cc:844] Serialized 14 byte log entry
0709 13:00:47.106836 (+   514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.107008 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.327721 (+220713us) raft_consensus.cc:1685] finished
0709 13:00:47.327741 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.327861 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2812,"prepare.queue_time_us":235574,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":8899,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":25193,"thread_start_us":154123,"threads_started":2,"wal-append.queue_time_us":209293}
I20260709 13:00:47.316498 22675 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.330415 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:47.333464 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:47.329130 22656 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.335127 22656 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.336545 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 799) took 1304 ms. Trace:
I20260709 13:00:47.336989 20858 rpcz_store.cc:276] 0709 13:00:46.032231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.032369 (+   138us) service_pool.cc:224] Handling call
0709 13:00:46.032651 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.034735 (+  2084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.034763 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.035310 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.371239 (+335929us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:46.372064 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.372158 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.249783 (+877625us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:47.336241 (+ 86458us) raft_consensus.cc:1685] finished
0709 13:00:47.336265 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.336479 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6234,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":12156288,"thread_start_us":335353,"threads_started":1,"wal-append.queue_time_us":475561}
I20260709 13:00:47.338755 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:47.339772 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:47.340288 20858 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:47.342305 22607 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:47.342823 22607 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:47.216123 22709 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.289855 22684 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.346647 22709 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:47.319806 22585 consensus_meta.cc:287] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.003s
I20260709 13:00:47.348261 22684 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:47.348336 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:47.264689 22713 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.343680 22607 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.350107 22607 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.350325 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 842) took 1244 ms. Trace:
I20260709 13:00:47.350580 22713 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:47.308854 22723 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:00:47.351948 22723 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:47.352305 22723 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:47.352661 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 792) took 1365 ms. Trace:
I20260709 13:00:47.353205 20856 rpcz_store.cc:276] 0709 13:00:45.986992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.987989 (+   997us) service_pool.cc:224] Handling call
0709 13:00:47.352369 (+1364380us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b5400008cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:47.352550 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868728704}
I20260709 13:00:47.353277 22723 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.348587 22585 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:47.355173 20856 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:47.355247 22723 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:47.355521 20857 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:47.357343 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 812) took 1253 ms. Trace:
I20260709 13:00:47.336624 22728 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.362567 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:47.350790 20854 rpcz_store.cc:276] 0709 13:00:46.105831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.349502 (+1243671us) service_pool.cc:224] Handling call
0709 13:00:47.349802 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.349989 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.350031 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.350083 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:47.350121 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.350174 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.350271 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.364112 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 849) took 1093 ms. Trace:
I20260709 13:00:47.309810 22589 raft_consensus.cc:3055] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:47.370977 20856 rpcz_store.cc:276] 0709 13:00:46.262378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.354560 (+1092182us) service_pool.cc:224] Handling call
0709 13:00:47.354946 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.355656 (+   710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.355860 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.373399 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 52858us
I20260709 13:00:47.357764 20859 rpcz_store.cc:276] 0709 13:00:46.106283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.990144 (+883861us) service_pool.cc:224] Handling call
0709 13:00:46.990470 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.990816 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.990845 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.006066 (+ 15221us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.211025 (+204959us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:47.211090 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.211198 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.357152 (+145954us) raft_consensus.cc:1685] finished
0709 13:00:47.357171 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.357290 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1602,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8152,"thread_start_us":88919,"threads_started":1,"wal-append.queue_time_us":197102}
I20260709 13:00:47.181764 22711 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:47.298151 22499 consensus_queue.cc:1059] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:00:47.375873 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.285744 22721 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:47.317139 22740 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:47.399623 20160 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:47.407730 20854 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:47.405920 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:47.424674 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.425757 20158 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
W20260709 13:00:47.427619 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.408710 20008 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:47.429241 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 850) took 1151 ms. Trace:
I20260709 13:00:47.429778 20854 rpcz_store.cc:276] 0709 13:00:46.277847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.366687 (+1088840us) service_pool.cc:224] Handling call
0709 13:00:47.379482 (+ 12795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.428977 (+ 49495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.429188 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.370172 22713 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.441557 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:47.443006 19982 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:47.376610 22589 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:47.447393 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:47.451423 20160 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:47.448483 20857 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:47.448668 20854 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:47.453872 20856 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:47.454735 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 853) took 1176 ms. Trace:
I20260709 13:00:47.455117 20856 rpcz_store.cc:276] 0709 13:00:46.278687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.430621 (+1151934us) service_pool.cc:224] Handling call
0709 13:00:47.453640 (+ 23019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.454381 (+   741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.454667 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.432070 22094 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.456802 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 854) took 1177 ms. Trace:
I20260709 13:00:47.395874 22682 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:00:47.457734 20858 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:47.458385 22756 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:47.460700 20854 rpcz_store.cc:276] 0709 13:00:46.278971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.430733 (+1151762us) service_pool.cc:224] Handling call
0709 13:00:47.448422 (+ 17689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.456533 (+  8111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.456746 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.436793 22709 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.376159 22546 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:47.470391 22704 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.862s	user 0.002s	sys 0.000s
I20260709 13:00:47.374848 22624 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.469370 22709 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.368124 22275 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:00:47.474090 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:47.465764 20055 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:47.565053 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 100676us
I20260709 13:00:47.452299 22601 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:00:47.520716 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.378973 22625 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.544725 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 816) took 1017 ms. Trace:
I20260709 13:00:47.554751 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 814) took 1038 ms. Trace:
I20260709 13:00:47.451470 22611 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:47.445487 22713 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.558368 22462 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.477598 22546 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:47.573555 20858 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:00:47.575277 20856 rpcz_store.cc:276] 0709 13:00:46.516088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.466351 (+950263us) service_pool.cc:224] Handling call
0709 13:00:47.554041 (+ 87690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.554289 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.554328 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.554386 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:47.554431 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.554515 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.554671 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.604070 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 205971us
W20260709 13:00:47.582858 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 108865us
I20260709 13:00:47.574924 20854 rpcz_store.cc:276] 0709 13:00:46.527488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.465495 (+938007us) service_pool.cc:224] Handling call
0709 13:00:47.544027 (+ 78532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.544282 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.544319 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.544376 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:47.544420 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.544499 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.544642 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.555449 22499 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.467345 22094 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:47.573565 22581 consensus_meta.cc:287] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.843s	user 0.016s	sys 0.007s
I20260709 13:00:47.559401 22589 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:47.574659 22681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.020s	sys 0.031s
W20260709 13:00:47.584491 22598 consensus_meta.cc:287] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.015s	user 0.006s	sys 0.010s
I20260709 13:00:47.570297 22601 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.622936 20088 raft_consensus.cc:2410] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:47.575083 22208 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:47.573971 22631 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:47.597374 22611 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.574234 22733 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:47.626614 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:47.656611 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:47.607765 22604 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.582024 22741 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:47.473683 22756 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:47.652084 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 817) took 1088 ms. Trace:
I20260709 13:00:47.658708 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:47.657382 22586 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.024s	sys 0.030s
I20260709 13:00:47.677579 20160 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:47.668125 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:47.613497 22747 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:47.668321 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:47.679792 20089 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:47.668174 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:47.670352 20856 rpcz_store.cc:276] 0709 13:00:46.563668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.609978 (+1046310us) service_pool.cc:224] Handling call
0709 13:00:47.651320 (+ 41342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.651597 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.651643 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.651706 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:47.651753 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.651834 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.652010 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.626390 22462 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:47.651574 22546 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.652097 22729 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.622020 22681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.020s	sys 0.031s
I20260709 13:00:47.629776 22601 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:47.670903 22611 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:47.676622 22586 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.025s	sys 0.031s
I20260709 13:00:47.661055 22589 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.691375 22735 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.617936 22499 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.640440 22598 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.685849 22631 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:47.692483 22693 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:47.669286 22604 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.706841 22586 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:47.737618 22631 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.738723 22747 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.740217 22743 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
W20260709 13:00:47.740877 20017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 421) took 1460 ms (client timeout 1666 ms). Trace:
W20260709 13:00:47.741273 20017 rpcz_store.cc:269] 0709 13:00:46.280874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.281417 (+   543us) service_pool.cc:224] Handling call
0709 13:00:47.740692 (+1459275us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54008d5254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:47.740798 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449058048}
I20260709 13:00:47.749187 20089 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:47.752568 20014 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:47.658690 22733 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.694391 22546 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:47.762617 19981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.000s	sys 0.022s
W20260709 13:00:47.763200 19981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.000s	sys 0.022s
I20260709 13:00:47.672777 22741 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.694062 22729 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:47.760619 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 101989us
I20260709 13:00:47.739264 22747 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.761445 22678 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:47.734774 22465 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:47.771416 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:47.764842 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.774391 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:47.780277 22691 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:47.780721 22499 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:47.780804 22691 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:47.777591 22745 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:00:47.774258 22465 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.772199 22624 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:47.731962 22735 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:47.794919 20743 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:47.791082 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:47.779078 22636 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:47.778623 22678 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:47.783540 22691 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:47.800221 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:47.803807 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:47.787789 22499 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:47.812464 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:47.813687 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:47.803297 22656 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.807276 22678 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:47.800159 22094 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:47.835249 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.827176 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:47.809551 22682 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:47.919235 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 105662us
I20260709 13:00:47.921494 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:47.918533 20011 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:47.924029 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:47.924979 20010 raft_consensus.cc:2410] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:00:47.928493 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:47.817770 22713 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:47.911274 22750 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
W20260709 13:00:47.930367 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 167581us
I20260709 13:00:47.816109 22759 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:00:47.936825 20857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 852) took 1658 ms (client timeout 1910 ms). Trace:
W20260709 13:00:47.920642 22757 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
W20260709 13:00:47.942562 20857 rpcz_store.cc:269] 0709 13:00:46.278413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.530516 (+252103us) service_pool.cc:224] Handling call
0709 13:00:47.936579 (+1406063us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54006eaad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:47.936744 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1895085824}
I20260709 13:00:47.917586 22499 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:47.962194 22598 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:47.965399 22754 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:00:47.968712 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:47.972149 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:47.973816 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:47.974943 22611 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:47.984849 20012 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:47.987071 22607 consensus_meta.cc:287] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.000s
I20260709 13:00:47.913383 22656 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:47.930394 22275 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:47.998555 22761 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
W20260709 13:00:47.999837 22699 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.586s	user 0.000s	sys 0.005s
I20260709 13:00:47.937934 22682 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:47.800508 22586 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:48.009518 22752 log.cc:927] Time spent T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:00:48.009346 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 890) took 1504 ms. Trace:
W20260709 13:00:48.033591 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.034986 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 823) took 1025 ms. Trace:
I20260709 13:00:48.051357 20857 rpcz_store.cc:276] 0709 13:00:47.009131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.946983 (+937852us) service_pool.cc:224] Handling call
0709 13:00:47.947343 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.034476 (+ 87133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.034527 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.034593 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:48.034641 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.034739 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.034904 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14930,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120}
I20260709 13:00:48.053462 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 824) took 1024 ms. Trace:
I20260709 13:00:48.053845 20857 rpcz_store.cc:276] 0709 13:00:47.029122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.052680 (+1023558us) service_pool.cc:224] Handling call
0709 13:00:48.052994 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.053167 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.053194 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.053232 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:48.053257 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.053309 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.053411 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:48.040247 22756 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:48.047606 22680 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.003s
I20260709 13:00:48.050904 20162 rpcz_store.cc:276] 0709 13:00:46.506268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.506502 (+   234us) service_pool.cc:224] Handling call
0709 13:00:46.507035 (+   533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.514181 (+  7146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.514222 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.336483 (+822261us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b2c001e7048. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:47.336884 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.572887 (+236003us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:47.572941 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.573047 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.008990 (+435943us) raft_consensus.cc:1685] finished
0709 13:00:48.009018 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.009268 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6633,"raft.queue_time_us":13521,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10428,"spinlock_wait_cycles":10494720,"thread_start_us":235582,"threads_started":1,"wal-append.queue_time_us":286311}
I20260709 13:00:48.063824 20857 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:48.071863 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 797) took 2063 ms. Trace:
I20260709 13:00:48.072329 20852 rpcz_store.cc:276] 0709 13:00:46.008803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.011331 (+  2528us) service_pool.cc:224] Handling call
0709 13:00:46.011726 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.012139 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.012164 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.012667 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.301900 (+289233us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:46.301928 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.301976 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.827435 (+1525459us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:48.071567 (+244132us) raft_consensus.cc:1685] finished
0709 13:00:48.071587 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.071804 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":416334720,"thread_start_us":288863,"threads_started":1,"wal-append.queue_time_us":400238}
I20260709 13:00:48.075251 22656 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.076936 22682 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:48.077903 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 529) took 1319 ms. Trace:
I20260709 13:00:48.078431 20163 rpcz_store.cc:276] 0709 13:00:46.759054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.760170 (+  1116us) service_pool.cc:224] Handling call
0709 13:00:46.760552 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.850018 (+ 89466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.850063 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.856025 (+  5962us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.084943 (+228918us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:47.084985 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.077430 (+992445us) spinlock_profiling.cc:243] Waited 78 ms on lock 0x7b5400586e54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:48.077503 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.077577 (+    74us) raft_consensus.cc:1685] finished
0709 13:00:48.077598 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.077827 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10981,"raft.queue_time_us":11115,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":16358,"spinlock_wait_cycles":179398912,"thread_start_us":208426,"threads_started":1,"wal-append.queue_time_us":353785}
W20260709 13:00:48.065958 22281 consensus_meta.cc:287] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.757s	user 0.010s	sys 0.006s
I20260709 13:00:48.084754 20855 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:48.065472 22499 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.085641 22499 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:48.086731 20855 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:48.066121 22680 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.066749 22733 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:48.110251 20009 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:48.141604 20009 pending_rounds.cc:85] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:48.141908 20009 pending_rounds.cc:107] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:48.142146 20009 raft_consensus.cc:2889] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:48.108677 20850 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 832) took 2078 ms (client timeout 1510 ms). Trace:
W20260709 13:00:48.107043 20853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 800) took 2074 ms (client timeout 1945 ms). Trace:
I20260709 13:00:48.117331 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:48.100952 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.115403 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55614 (request call id 16) took 1701 ms. Trace:
I20260709 13:00:48.146143 20012 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
W20260709 13:00:48.143442 20850 rpcz_store.cc:269] 0709 13:00:46.030388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.030515 (+   127us) service_pool.cc:224] Handling call
0709 13:00:48.108463 (+2077948us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b54000046d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:48.108589 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519597824}
I20260709 13:00:48.146530 20161 rpcz_store.cc:276] 0709 13:00:46.413879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.414038 (+   159us) service_pool.cc:224] Handling call
0709 13:00:48.115111 (+1701073us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b54001be4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ac853 00007fa03377a60a 00007fa02cd007d4 00007fa02cd00762 00007fa02cd00691 00007fa02cd0060c 00007fa02ccff8c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:48.115270 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2040296832}
I20260709 13:00:48.150810 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:48.143860 20853 rpcz_store.cc:269] 0709 13:00:46.032525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.034156 (+  1631us) service_pool.cc:224] Handling call
0709 13:00:48.106776 (+2072620us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b5400380254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:48.106888 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305878912}
I20260709 13:00:48.164191 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.165539 20850 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
W20260709 13:00:48.102072 22705 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.003s
I20260709 13:00:48.168339 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:48.168612 20009 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:00:48.089179 22698 log.cc:927] Time spent T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.480s	user 0.000s	sys 0.003s
I20260709 13:00:48.169988 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.170598 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:48.172540 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.101759 22369 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:48.109105 22482 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.154770 22589 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:48.167691 22542 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:48.096982 22601 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:48.087834 22499 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:48.198395 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.199493 20855 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:48.177436 22721 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:48.174095 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:48.095115 22281 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:48.098608 22604 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:48.177608 22208 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:48.097286 22729 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.083561 22741 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.172106 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:48.118768 22735 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:48.173470 20853 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:48.212733 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:48.201908 22740 consensus_meta.cc:287] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.762s	user 0.007s	sys 0.005s
I20260709 13:00:48.154392 22681 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:00:48.179469 22369 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:48.217581 20850 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:48.222719 20016 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:48.214076 22721 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.216318 22589 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:48.104504 22629 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.173097 22462 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:00:48.284235 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.173430 22705 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:48.166785 22359 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.194856 22482 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.307408 22208 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:48.307534 22721 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:48.359387 20856 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:48.334390 22729 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:48.363435 20013 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:48.334676 22735 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:48.368535 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.369634 20013 raft_consensus.cc:2410] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:48.324186 22692 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:48.349596 22320 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:48.459250 22463 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:48.475482 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.484841 22611 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:48.360474 22678 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:48.438841 22586 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.351176 22366 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.333600 22629 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.561575 22586 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:48.563148 22629 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:48.394459 22621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.926s	user 0.031s	sys 0.054s
W20260709 13:00:48.563853 22621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.134s	user 0.063s	sys 0.103s
I20260709 13:00:48.563944 22586 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.565353 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.566393 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.566584 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:48.567116 20011 raft_consensus.cc:606] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:48.567569 20011 raft_consensus.cc:614] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:48.226888 22740 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:48.568513 20017 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:48.569677 22740 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.571197 20017 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:48.411885 22596 raft_consensus.cc:3055] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:48.572103 22596 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:48.573011 22596 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.573530 20017 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:48.574092 20017 pending_rounds.cc:85] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:48.399601 22693 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:48.574540 20017 pending_rounds.cc:107] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:48.365672 22359 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:48.535116 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 263715us
W20260709 13:00:48.583814 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.545539 22320 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:48.429814 22774 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:00:48.510581 22778 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.097s	user 0.002s	sys 0.000s
I20260709 13:00:48.472127 22680 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:48.475881 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.457695 22773 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:00:48.376343 22735 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.367476 22729 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:48.508841 22777 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:00:48.546998 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.547526 22764 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:00:48.547847 22208 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:48.501410 22601 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.555099 22705 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:48.580410 22366 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:48.554962 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:48.519881 22465 consensus_meta.cc:287] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.715s	user 0.007s	sys 0.006s
I20260709 13:00:48.560577 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.639690 20013 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:48.641435 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:48.641813 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 829) took 1074 ms. Trace:
I20260709 13:00:48.567052 20012 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:48.574059 22596 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:48.393877 22723 consensus_meta.cc:287] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 1.038s	user 0.012s	sys 0.005s
W20260709 13:00:48.575141 22783 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:00:48.515551 22691 consensus_meta.cc:287] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.000s
W20260709 13:00:48.531189 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 121931us
I20260709 13:00:48.652627 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:48.576608 20017 raft_consensus.cc:2889] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:48.642239 20857 rpcz_store.cc:276] 0709 13:00:47.568208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.082967 (+514759us) service_pool.cc:224] Handling call
0709 13:00:48.155256 (+ 72289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.155506 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:48.372245 (+216739us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.641484 (+269239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.641541 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:48.641583 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.641657 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.641761 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1687,"prepare.queue_time_us":246189,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2090,"raft.queue_time_us":37952,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":10421,"thread_start_us":439028,"threads_started":2}
I20260709 13:00:48.485532 22482 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:48.547400 22207 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:48.588052 20083 raft_consensus.cc:1240] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:48.655838 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.589092 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 268671us
I20260709 13:00:48.596105 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.585186 22692 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.588591 22275 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:48.616153 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.646191 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.659256 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 903) took 1002 ms. Trace:
I20260709 13:00:48.665287 20013 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:48.647186 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.668437 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.668985 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.669632 20853 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:48.657773 20009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81fd99f42a47497f9d4372157320611b"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:00:48.617409 22740 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:48.670271 20009 raft_consensus.cc:606] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:48.670428 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.670686 20009 raft_consensus.cc:614] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:48.673903 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.673748 20130 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:48.664953 20164 rpcz_store.cc:276] 0709 13:00:47.657083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.977162 (+320079us) service_pool.cc:224] Handling call
0709 13:00:47.978822 (+  1660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.979282 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.979335 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.980649 (+  1314us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.143552 (+162903us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:48.143593 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.143671 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.659044 (+515373us) raft_consensus.cc:1685] finished
0709 13:00:48.659065 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.659201 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":162325,"threads_started":1,"wal-append.queue_time_us":246064}
I20260709 13:00:48.615618 22611 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.680757 20857 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:48.680923 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.682327 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:48.636694 22208 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.683151 20164 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.690836 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.693115 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:48.693480 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.667819 20012 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:48.695844 20161 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:48.697891 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:48.700750 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:48.706238 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:48.716588 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:48.676011 22601 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.718906 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:48.643604 22465 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:48.724110 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.720350 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.725267 20853 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:48.720608 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:48.717150 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.722936 20084 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:48.723984 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:48.658758 22691 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.744372 20162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb11384495c2485aaab243a655f75c86"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:48.745253 20162 raft_consensus.cc:606] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:48.745590 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:48.665812 22207 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.672796 22275 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:48.735548 22697 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 2.250s	user 0.003s	sys 0.011s
I20260709 13:00:48.737504 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:48.748008 20163 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:48.758690 20852 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:48.646968 22320 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:48.749346 20082 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:48.672353 22692 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.676838 22740 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.746415 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:48.749773 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76140.8us
I20260709 13:00:48.761899 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:48.765120 20161 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:48.686847 22611 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:48.772506 20091 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:48.771164 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 851) took 2493 ms. Trace:
I20260709 13:00:48.719576 22721 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:48.735550 22794 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:48.673295 22729 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.775174 20859 rpcz_store.cc:276] 0709 13:00:46.278111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.377972 (+1099861us) service_pool.cc:224] Handling call
0709 13:00:47.378401 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.396994 (+ 18593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.397041 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.094152 (+697111us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b2c0001afe8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:48.094595 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.202656 (+108061us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:48.328827 (+126171us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.359600 (+ 30773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.359700 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.759250 (+399550us) raft_consensus.cc:1685] finished
0709 13:00:48.770905 (+ 11655us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.771077 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46727,"prepare.queue_time_us":243036,"prepare.run_cpu_time_us":168,"prepare.run_wall_time_us":167,"raft.queue_time_us":76975,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":16207,"spinlock_wait_cycles":6080768,"thread_start_us":216886,"threads_started":2,"wal-append.queue_time_us":314761}
I20260709 13:00:48.735889 22601 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:48.771431 22678 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.796820 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.797861 20017 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:48.780748 22320 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.781128 22794 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.802434 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:48.808826 22542 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
W20260709 13:00:48.812520 22797 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:00:48.812879 22465 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:48.813956 22366 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:48.814538 22776 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.444s	user 0.002s	sys 0.002s
I20260709 13:00:48.819306 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 818) took 1970 ms. Trace:
I20260709 13:00:48.819797 20856 rpcz_store.cc:276] 0709 13:00:46.848627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.691075 (+842448us) service_pool.cc:224] Handling call
0709 13:00:47.691479 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.359897 (+668418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.819012 (+459115us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54006ea854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:48.819233 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535630080}
I20260709 13:00:48.821019 22740 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:48.821784 22749 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 1.140s	user 0.015s	sys 0.003s
I20260709 13:00:48.822513 22691 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:48.823381 22692 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:48.824292 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 896) took 1505 ms. Trace:
I20260709 13:00:48.825470 22611 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:48.824721 20159 rpcz_store.cc:276] 0709 13:00:47.319419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.319982 (+   563us) service_pool.cc:224] Handling call
0709 13:00:47.321272 (+  1290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.325806 (+  4534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.325839 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.326322 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.437591 (+111269us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:47.437632 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.437701 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.821463 (+1383762us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b24005ad148. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b70409d 00007fa02e33610e 00007fa02e34d88c 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221
0709 13:00:48.823994 (+  2531us) raft_consensus.cc:1685] finished
0709 13:00:48.824018 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.824230 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1141,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":19393280,"thread_start_us":110867,"threads_started":1,"wal-append.queue_time_us":287196}
W20260709 13:00:48.827193 22787 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:00:48.807078 22735 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:48.802225 22729 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.828568 22275 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:48.829710 22207 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:48.804606 22794 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:48.837462 22601 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.851758 20164 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:48.821611 22721 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:48.829725 22208 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:48.862675 22708 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.900s	user 0.011s	sys 0.002s
I20260709 13:00:48.866107 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:48.875941 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:48.867446 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 803) took 2273 ms. Trace:
I20260709 13:00:48.870729 20085 raft_consensus.cc:2410] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:00:48.881065 22781 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.452s	user 0.001s	sys 0.000s
I20260709 13:00:48.877069 20014 rpcz_store.cc:276] 0709 13:00:46.593522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.593773 (+   251us) service_pool.cc:224] Handling call
0709 13:00:46.594077 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.753677 (+1159600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.867037 (+1113360us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540073e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:48.867303 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2641378048}
W20260709 13:00:48.887522 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 114991us
I20260709 13:00:48.888926 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:48.923200 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 811) took 2820 ms. Trace:
I20260709 13:00:48.925803 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.932468 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.923833 20851 rpcz_store.cc:276] 0709 13:00:46.103249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.249040 (+145791us) service_pool.cc:224] Handling call
0709 13:00:46.249347 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.269991 (+ 20644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.270034 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:46.298412 (+ 28378us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.538486 (+240074us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:46.538526 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.538590 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.855367 (+2316777us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:48.916757 (+ 61390us) raft_consensus.cc:1685] finished
0709 13:00:48.916778 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.923094 (+  6316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":651,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":43763968,"thread_start_us":171692,"threads_started":1,"wal-append.queue_time_us":523000}
I20260709 13:00:48.939816 22546 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:48.940675 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.941527 20012 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:48.942348 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.943054 22812 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.943980 22812 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.944893 22794 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:48.946105 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 827) took 1581 ms. Trace:
I20260709 13:00:48.946489 20852 rpcz_store.cc:276] 0709 13:00:47.364979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.074718 (+709739us) service_pool.cc:224] Handling call
0709 13:00:48.074959 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.759188 (+684229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.945861 (+186673us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54000087d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:48.946040 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1572035456}
I20260709 13:00:48.947031 22794 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:48.943113 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.947157 22812 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:48.950110 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.915763 22207 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:48.952540 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 437) took 1165 ms. Trace:
I20260709 13:00:48.952906 22207 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.953374 22207 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:48.954470 22803 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:48.954960 22802 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.955044 22803 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:48.955590 22802 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:48.953012 20016 rpcz_store.cc:276] 0709 13:00:47.787045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.787234 (+   189us) service_pool.cc:224] Handling call
0709 13:00:48.952334 (+1165100us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b540073ebd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:48.952474 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":960146816}
I20260709 13:00:48.962891 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:48.965408 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:48.958098 22802 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:48.973476 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.961251 20129 connection.cc:256] RPC call timeout handler was delayed by -1.2791717860000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
I20260709 13:00:48.974596 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.977207 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:48.975692 20129 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:48.990228 20011 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:48.983881 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.986302 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.990583 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.991082 20851 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:48.993502 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:48.995447 20156 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:48.997452 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:48.964239 22808 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.999423 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:48.999603 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:49.000022 20854 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:00:49.000415 20854 pending_rounds.cc:85] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 4
I20260709 13:00:49.000633 20854 pending_rounds.cc:107] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:49.000713 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:48.998059 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.000865 20854 raft_consensus.cc:2889] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:49.001415 20856 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:49.002168 20857 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:49.001318 20156 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:48.998885 20851 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:49.003588 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:48.969882 22803 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:48.997501 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.004797 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:49.004834 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.006785 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:49.034809 20851 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:49.020030 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:48.982031 22729 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:49.036619 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:48.998397 22378 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:48.998386 22812 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:49.040330 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.041553 20129 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.006949 22208 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:49.046550 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.008248 22463 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:49.048503 22463 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:49.053591 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.057636 20160 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:49.039338 22729 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.038761 22803 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.056073 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:49.044652 22692 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:49.059026 22482 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:00:49.088979 22803 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.044725 22802 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:49.050135 22208 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:49.046944 22542 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:00:49.083756 22693 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:49.084570 22709 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.085166 22764 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:49.052165 22463 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:49.088356 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.089937 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.086098 20851 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:00:49.090783 20744 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:49.090512 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:49.094991 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.042217 22320 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:00:49.006973 22808 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.052477 22546 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.102267 20744 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:49.052816 22812 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:49.107486 20156 raft_consensus.cc:2410] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:00:49.094386 22208 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:49.093776 22275 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:49.116837 22094 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:49.097003 22721 consensus_queue.cc:1059] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:00:49.117885 22721 raft_consensus.cc:3055] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:49.112578 22465 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:49.118227 22721 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:49.102447 22764 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.096319 22693 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.121793 20015 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:49.122123 22764 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.122617 22326 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:49.123032 22693 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.127331 22589 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:00:49.116122 22359 consensus_meta.cc:287] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.531s	user 0.015s	sys 0.004s
I20260709 13:00:49.104216 22629 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.095072 22803 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:49.113229 20088 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 425) took 2638 ms (client timeout 2156 ms). Trace:
W20260709 13:00:49.153090 19981 connection.cc:256] RPC call timeout handler was delayed by -1.1731394019998334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
I20260709 13:00:49.155978 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:49.154762 20088 rpcz_store.cc:269] 0709 13:00:46.468323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.469456 (+  1133us) service_pool.cc:224] Handling call
0709 13:00:49.097794 (+2628338us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54006658d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:49.106214 (+  8420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479037312}
I20260709 13:00:49.110121 22709 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.129842 22808 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.130939 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:49.160054 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.094313 22682 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:00:49.130548 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.128401 22614 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.129518 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 436) took 1141 ms. Trace:
I20260709 13:00:49.144910 22369 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.149324 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.168820 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.171075 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:49.172103 20089 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:49.170579 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.111867 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:49.141364 20160 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:49.150144 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.150031 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:49.158032 20165 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:00:49.159384 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:49.119379 22721 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:49.127821 20015 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:49.162427 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.177793 20853 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:49.162930 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.161314 20014 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:49.155541 22629 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.109433 22546 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:49.164305 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.163830 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:49.161564 20854 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:49.123008 22326 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:49.166555 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.167057 20083 rpcz_store.cc:276] 0709 13:00:47.987823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.987959 (+   136us) service_pool.cc:224] Handling call
0709 13:00:47.988294 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.660016 (+671722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.129264 (+469248us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b54005a2154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:49.129429 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374404480}
I20260709 13:00:49.178907 20856 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:00:49.185854 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.186239 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.167913 22715 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.180022 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.180953 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:49.161875 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.189075 20850 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:49.189260 20851 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:49.182389 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:49.187716 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:49.182457 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 70649.2us
I20260709 13:00:49.157259 22728 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.163801 22802 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:49.195130 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.195761 20084 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:49.195822 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.185235 20857 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:49.164438 22610 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.160842 22709 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:49.194115 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:49.194314 20744 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:49.195597 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.197609 22728 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.198545 22755 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:00:49.169701 22693 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.198807 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:49.228634 20011 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:49.199054 22546 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.223106 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:49.166774 22691 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.201587 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:49.200259 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:49.202132 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.193360 22094 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.185141 22614 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:49.206894 22462 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:49.179664 22369 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:49.257685 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.258409 20083 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:49.206992 22326 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.269052 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:49.269281 22326 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:49.269769 20161 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.199883 22715 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.208071 22675 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:49.200364 22812 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.273692 22715 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:49.208433 22721 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.200302 22611 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:49.200044 22601 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.276901 22611 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.277483 22601 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.279242 22611 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:49.200189 22622 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:49.280205 22622 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.282645 22622 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:49.209388 22729 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.184342 22794 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:49.209537 22820 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:49.210709 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:49.283878 22820 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.240024 22693 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.284421 22820 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:49.285923 20011 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:49.209324 22709 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.216408 22747 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:49.216754 22802 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.217535 22281 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.217856 20091 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:49.220118 22789 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:49.220693 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:49.203790 22723 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:00:49.170908 22808 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:49.227303 20856 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:00:49.229511 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.231129 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:49.188989 22465 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:49.241436 20165 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.256383 20163 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:49.237149 22624 consensus_queue.cc:1059] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:00:49.255996 20016 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:49.224172 22696 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:49.255317 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.230300 22596 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:00:49.282990 20158 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:49.209697 22610 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.286080 22693 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.184966 22629 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.251118 22691 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.299187 20161 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:49.248380 22320 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:00:49.321094 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 100470us
I20260709 13:00:49.354559 19981 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:49.321470 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.328887 22755 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:49.279708 22601 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:49.248095 22094 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:49.333700 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.311216 22795 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.343555 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.318389 22462 consensus_queue.cc:1243] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:00:49.325598 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.353188 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:49.328423 22465 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.330107 22740 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:49.362051 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:49.367668 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.347128 22729 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:49.377739 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:49.322863 22802 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:49.385165 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 134406us
I20260709 13:00:49.374166 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.386607 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.360143 22320 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:49.366991 22094 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.375641 22596 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:49.389853 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.391873 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:49.385711 20165 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:49.394254 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 815) took 2873 ms. Trace:
I20260709 13:00:49.403009 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:49.354635 22281 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:00:49.366787 22715 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.400799 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.406230 20161 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:49.376799 22465 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:49.409426 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:49.395946 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.402833 20858 rpcz_store.cc:276] 0709 13:00:46.520872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.464714 (+943842us) service_pool.cc:224] Handling call
0709 13:00:47.465076 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.603872 (+138796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.393953 (+1790081us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400004bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:49.394190 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248801152}
I20260709 13:00:49.381870 22696 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:49.377111 22740 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.388415 22601 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.415613 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.414849 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.398128 22675 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:49.439698 20165 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:49.424505 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.414125 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:49.418756 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.446530 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:49.447413 20011 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:00:49.453651 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:49.459035 20085 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:49.410530 22715 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:49.484467 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:49.485034 20163 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:49.388917 22729 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:49.376224 22795 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.410192 22359 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.487366 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:49.396452 22803 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.505259 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:49.421526 22740 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:49.484339 22715 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.475239 22596 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:00:49.439846 22601 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:49.519222 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.425439 22589 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:49.411100 22207 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:49.506939 22812 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:49.509644 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:49.524461 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.558910 22823 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 13:00:49.559551 22822 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:00:49.506425 22359 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.452275 22675 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:49.561666 20164 raft_consensus.cc:2393] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:49.573796 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:49.561231 22320 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.562688 22208 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.585500 22207 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:49.586216 20163 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:49.560216 22601 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.748334 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:49.748963 22629 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.750589 20088 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:49.760793 22629 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.596387 22691 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.599381 22802 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.599604 22803 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.601593 22812 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:49.768473 22812 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:49.573246 22708 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:00:49.508127 22463 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.771631 22463 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.714922 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.772562 20851 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:49.773617 20159 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:49.716581 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.774329 20853 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:49.729480 20090 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:00:49.776763 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:49.731092 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.731249 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.517853 22693 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.192s
I20260709 13:00:49.712999 22589 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:49.731685 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.739658 22610 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:49.739547 22546 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:49.777968 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.782094 22589 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:49.778784 20091 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:00:49.712649 22675 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:49.742367 22094 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.784466 20086 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:49.786413 22094 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.789764 22465 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.792806 20852 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:49.780858 20085 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:49.771700 22740 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.812013 22740 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:49.812801 22740 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:49.812958 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 440) took 2024 ms. Trace:
I20260709 13:00:49.814373 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.813397 20008 rpcz_store.cc:276] 0709 13:00:47.788575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.969120 (+180545us) service_pool.cc:224] Handling call
0709 13:00:47.969438 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.047560 (+ 78122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.047591 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.105277 (+ 57686us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:48.105671 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.202415 (+ 96744us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:48.202462 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.812634 (+1610172us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b54009fa6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:49.812685 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.812734 (+    49us) raft_consensus.cc:1685] finished
0709 13:00:49.812748 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.812897 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4124,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":5727,"spinlock_wait_cycles":183989376,"thread_start_us":96379,"threads_started":1,"wal-append.queue_time_us":250610,"wal-append.run_cpu_time_us":1881,"wal-append.run_wall_time_us":1087681}
I20260709 13:00:49.816119 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.819499 20090 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:49.821164 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:49.822352 20087 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:49.824667 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:49.711747 22359 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:49.760272 22825 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
W20260709 13:00:49.779215 22326 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.003s
I20260709 13:00:49.779904 22629 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:49.839254 20859 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:49.761895 22795 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:49.839054 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 393673us
I20260709 13:00:49.805313 22207 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.843518 20083 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:49.839146 22629 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:49.751801 22820 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:49.855243 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 369222us
I20260709 13:00:49.856571 19981 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:49.781186 22610 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:00:49.861308 19981 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:49.781446 22546 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.813400 22740 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.835805 22696 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:49.891561 20160 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:49.891454 19981 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:49.896230 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.852524 22207 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:49.917464 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.902882 22715 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.913609 22696 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:49.923384 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.925145 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.929347 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:49.914332 22740 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:49.932690 22740 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.933157 22740 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:49.933610 22740 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.933766 22675 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.917089 22729 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:49.937887 22207 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.940291 20011 raft_consensus.cc:2368] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
W20260709 13:00:49.940743 22709 consensus_meta.cc:287] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.620s	user 0.006s	sys 0.004s
I20260709 13:00:49.941294 22709 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:49.943194 22208 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.907343 22629 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.944046 22610 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:49.944510 22629 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:49.944759 22610 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:49.945185 22610 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:49.945544 22610 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.948218 22812 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:49.949153 22622 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:49.949270 22803 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:49.949625 22622 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:49.950570 22622 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.928105 22692 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:00:49.952039 22465 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.952847 22709 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.916100 22820 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:49.966971 22601 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:49.985954 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.987207 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.988585 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.951766 22622 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:49.990605 20852 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:49.940436 22715 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:49.962759 22692 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:49.977224 22624 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.968233 22546 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:49.966475 22803 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:50.000981 20850 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:49.973469 22359 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:49.959901 22740 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.021647 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.022951 22740 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.023761 22803 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.026156 22803 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.026218 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:49.985322 22465 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.027696 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.027593 22465 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.029659 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.029827 22465 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:49.984762 22833 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:50.030618 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.031054 22833 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.018988 22836 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.032378 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.032500 22624 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.033639 22833 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.033792 20855 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.035023 22624 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:50.039955 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.020429 22675 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.035288 22601 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.042040 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.046133 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.043685 20850 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:49.997977 22596 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:50.048656 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.018632 22715 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.079763 22848 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:50.082854 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 186737us
W20260709 13:00:50.085115 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.086174 20855 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:50.088181 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:50.136471 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52792.1us
W20260709 13:00:50.137830 22798 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.358s	user 0.000s	sys 0.013s
I20260709 13:00:50.049530 22834 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:50.139580 22834 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:50.140558 22834 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.141438 22839 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:50.143168 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.144430 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.145052 20851 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.070228 22803 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:00:50.147078 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.148419 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:50.148217 22596 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.032440 22836 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.153406 22836 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.155418 22359 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.155725 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:50.076730 22813 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:50.156509 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.157122 22813 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.157605 22813 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.158231 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 911) took 1154 ms. Trace:
I20260709 13:00:50.158776 20158 rpcz_store.cc:276] 0709 13:00:49.003433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.003597 (+   164us) service_pool.cc:224] Handling call
0709 13:00:50.157893 (+1154296us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54008d22d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:50.158100 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562109184}
W20260709 13:00:50.159974 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.160478 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.161335 20850 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:50.162449 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.162469 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.163146 20850 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.163681 22359 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:50.108592 22820 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.164086 22359 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.164523 22359 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.164580 22820 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:50.165562 22820 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.167949 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:50.168733 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 347038us
I20260709 13:00:50.170040 20851 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:50.170421 20852 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:50.171497 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.169660 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.174042 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.172493 20017 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:50.174724 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.177201 20162 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:50.177351 22841 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:50.177646 22847 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:50.082775 22848 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.179193 22847 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.179601 22847 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:50.180999 22848 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.182921 22359 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.175700 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:50.184978 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.185523 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.187443 20158 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:00:50.188527 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.189468 20851 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.191376 20091 pending_rounds.cc:85] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 2
I20260709 13:00:50.191850 20091 pending_rounds.cc:107] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:00:50.192130 20091 raft_consensus.cc:2889] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:50.193836 22740 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.194561 22740 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.197064 22740 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.197734 22629 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:50.184366 22735 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:50.197459 22846 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:00:50.202198 22846 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:50.202580 22850 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.202692 22846 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:50.203080 22728 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.203198 22850 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:50.205184 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:50.190508 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:50.202059 22359 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.207114 20857 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:50.205430 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.208127 20085 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:50.208781 20744 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.096236 22715 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.212001 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:50.207065 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.212769 22624 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:50.214601 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.214705 22610 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.215684 20008 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:50.217514 22610 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.217741 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.216519 20012 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:50.188534 22208 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:50.218416 22320 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:50.218962 22208 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:50.218438 20008 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:50.219553 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.220089 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.219987 22208 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.220791 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.204384 22841 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:50.221388 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.221560 22841 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.222074 20008 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:50.222126 22841 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.224064 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.224752 20850 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:50.226302 22624 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.226629 22320 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:50.227259 22715 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.210605 22711 raft_consensus.cc:2764] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:00:50.229106 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:50.232826 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.233764 20008 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:50.237322 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 1182) took 1008 ms. Trace:
I20260709 13:00:50.237825 22711 consensus_queue.cc:1059] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:50.238818 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1018) took 1144 ms. Trace:
I20260709 13:00:50.239259 22711 raft_consensus.cc:3055] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:50.239727 22711 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:00:50.240625 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 871592us
I20260709 13:00:50.241834 20851 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:50.242432 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.239296 20156 rpcz_store.cc:276] 0709 13:00:49.094562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.094712 (+   150us) service_pool.cc:224] Handling call
0709 13:00:50.238556 (+1143844us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b5400586e54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:50.238746 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18061184}
I20260709 13:00:50.237918 20084 rpcz_store.cc:276] 0709 13:00:49.229115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.229246 (+   131us) service_pool.cc:224] Handling call
0709 13:00:50.237082 (+1007836us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b540060c2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:50.237251 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1791472000}
I20260709 13:00:50.203729 22846 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.248721 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:50.249348 19982 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:50.261329 22610 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.249246 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.272332 22610 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.273372 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.274231 22465 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:50.274565 22546 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:50.274706 22465 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.274981 22610 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.275314 22465 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.277063 22546 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.277577 22546 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.283428 20857 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:50.286060 22805 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:50.302338 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:50.305459 19981 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.665s (SENT) [suppressed 11 similar messages]
I20260709 13:00:50.263558 22359 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:50.304184 22735 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:50.300761 22740 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:50.324551 20129 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.248032 22728 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:00:50.304946 22709 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:50.319494 22741 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.271685 22833 consensus_queue.cc:1059] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:00:50.318441 22846 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.204169 22850 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.319269 22601 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:50.342582 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.321396 22622 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.240958 22711 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.356590 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.359368 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.398943 20011 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:50.336391 22692 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.400065 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.394788 22208 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:00:50.401032 22207 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:50.405597 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.405866 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.407132 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.412104 20011 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:50.353047 22740 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:50.364192 22803 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:50.418004 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.417552 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 112113us
I20260709 13:00:50.420187 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.422220 20744 leader_election.cc:400] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
I20260709 13:00:50.341941 22709 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.422627 20744 leader_election.cc:403] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:00:50.422508 20129 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.424139 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.425009 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.425146 22805 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:50.427275 22828 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.941s	user 0.002s	sys 0.000s
I20260709 13:00:50.423023 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.431257 20011 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:50.348928 22741 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.435062 20008 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:50.435711 20091 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:50.362650 22601 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.440402 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.412693 22207 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:50.441761 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:50.395222 22622 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.445191 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.445421 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.436406 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:50.450466 22859 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:00:50.450820 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.452176 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:50.455343 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 546) took 1136 ms. Trace:
I20260709 13:00:50.457984 20088 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:00:50.458657 20011 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:50.458696 20088 pending_rounds.cc:85] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 3
I20260709 13:00:50.459519 20088 pending_rounds.cc:107] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:50.459775 20088 raft_consensus.cc:2889] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:50.437837 22094 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.466850 22094 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.434315 22692 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.462427 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.450539 22709 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:50.469038 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.473452 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:50.468250 22542 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.455844 20165 rpcz_store.cc:276] 0709 13:00:49.319396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.151681 (+832285us) service_pool.cc:224] Handling call
0709 13:00:50.153643 (+  1962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.153856 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:50.333590 (+179734us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.454897 (+121307us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.454967 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:50.455043 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.455131 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.455273 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243789,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"thread_start_us":292832,"threads_started":2}
I20260709 13:00:50.457176 22805 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.428956 22803 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.508651 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.457839 22870 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
I20260709 13:00:50.485433 20084 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:50.512754 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 90505.2us
I20260709 13:00:50.520310 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.443435 22465 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.513999 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:50.472328 22715 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:50.472944 22711 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.508118 22542 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.527951 22542 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.529210 22601 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.523542 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.524284 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.531430 20165 raft_consensus.cc:2393] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:50.525554 22715 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.508291 22614 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:50.478503 22320 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:50.467730 22813 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.531275 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.533654 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:50.522902 22208 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:50.540575 22875 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.541877 22715 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.532449 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:50.544584 19981 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.549978 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.535022 22614 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:50.547519 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.544606 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.550210 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.553175 20165 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:50.552997 20850 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:50.550433 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:50.548256 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.554232 20857 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.555088 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:50.556344 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.551965 22875 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.557076 22688 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.557386 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.557540 22688 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:50.558007 22688 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.558930 22875 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:50.559554 22728 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:50.560156 22728 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:50.551805 22623 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:50.562233 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:50.562476 22623 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:50.563249 20850 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:50.563127 20165 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:50.477504 22692 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.562692 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.565254 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.554932 22614 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.567291 22359 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:50.567832 22359 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.568269 22359 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.569207 22794 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:50.569639 22794 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:50.570444 22794 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.571221 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.572044 22875 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:50.572468 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:50.573568 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.577872 20857 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:50.580943 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.581425 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:50.580039 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.586016 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:50.571647 22794 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.552934 22601 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:50.486363 22740 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.605449 20156 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:50.612272 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1015) took 1564 ms. Trace:
I20260709 13:00:50.612736 20162 rpcz_store.cc:276] 0709 13:00:49.048048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.048189 (+   141us) service_pool.cc:224] Handling call
0709 13:00:50.168802 (+1120613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.177669 (+  8867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.611993 (+434324us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54008d22d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:50.612192 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576405376}
I20260709 13:00:50.613881 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.600172 22715 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:50.615650 22735 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:50.616195 22735 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.614497 20162 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:50.619081 20744 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:50.620280 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:50.582926 22874 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:50.601295 22756 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:50.561333 22728 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:50.656646 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 125449us
I20260709 13:00:50.601596 22846 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.550575 22208 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.615656 22715 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:50.602249 22877 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:50.601778 22359 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.659629 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:50.656812 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.672895 20014 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:50.673408 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.674425 20162 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:50.675357 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.660902 20089 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:50.654376 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.661859 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.662052 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:50.577905 22614 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.657373 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 1176) took 1487 ms. Trace:
I20260709 13:00:50.663787 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:50.718559 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54873.8us
I20260709 13:00:50.720650 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:50.631371 22094 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.587116 22834 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:50.653308 22622 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.652360 22482 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:00:50.663548 22872 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.131s	user 0.001s	sys 0.001s
I20260709 13:00:50.679643 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:50.680132 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.656975 22813 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:50.697610 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.742723 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 847) took 1508 ms. Trace:
I20260709 13:00:50.652915 22623 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:50.698680 22847 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.002s
I20260709 13:00:50.698585 22208 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:50.699536 22688 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.700109 22756 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:50.658654 22874 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.700692 22542 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.704276 22881 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.705585 22359 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.654393 22621 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:50.716065 20857 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:50.666060 22870 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:00:50.744827 22622 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.717451 20086 rpcz_store.cc:276] 0709 13:00:49.170087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.170248 (+   161us) service_pool.cc:224] Handling call
0709 13:00:50.657211 (+1486963us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b540020e9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:50.657299 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331901312}
I20260709 13:00:50.734133 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.736310 20162 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:50.661680 22846 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:50.739784 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:50.803615 20011 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.743129 20859 rpcz_store.cc:276] 0709 13:00:49.233920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.234685 (+   765us) service_pool.cc:224] Handling call
0709 13:00:49.235015 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:49.843046 (+608031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.742456 (+899410us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b5400008cd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:50.742664 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382655232}
I20260709 13:00:50.746465 20014 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:00:50.737360 22792 raft_consensus.cc:993] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
W20260709 13:00:50.804747 20014 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 837) took 1606 ms (client timeout 1865 ms). Trace:
W20260709 13:00:50.805125 20014 rpcz_store.cc:269] 0709 13:00:49.198035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.200833 (+  2798us) service_pool.cc:224] Handling call
0709 13:00:50.656640 (+1455807us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b5800867998. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:50.804692 (+148052us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3256064}
W20260709 13:00:50.732625 22841 consensus_meta.cc:287] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.000s
I20260709 13:00:50.735690 22482 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.729254 22789 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.795650 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.744330 22709 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.632097 22601 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:50.781322 22833 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.806468 20014 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.807492 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.740760 22834 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:50.783387 22611 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:50.818621 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.827301 20014 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.829269 20859 raft_consensus.cc:2368] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8.
I20260709 13:00:50.830391 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.782253 22715 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.794684 22621 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.835281 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.829008 22833 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.853140 20162 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:50.854399 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.816612 22789 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.856564 22614 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.856714 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.817919 22622 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.858033 22789 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.859974 22622 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.815364 22841 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.861191 20012 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:50.862687 22621 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.862880 22601 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.840593 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.811650 22846 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.867642 22692 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.797145 22874 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.838194 22747 raft_consensus.cc:3055] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:50.872570 22747 raft_consensus.cc:740] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:50.787148 22847 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:50.873765 22747 consensus_queue.cc:260] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.875216 22747 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:00:50.839890 20164 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1029) took 1527 ms (client timeout 1926 ms). Trace:
I20260709 13:00:50.876200 22847 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.797783 22836 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:50.876690 20164 rpcz_store.cc:269] 0709 13:00:49.312461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.325246 (+ 12785us) service_pool.cc:224] Handling call
0709 13:00:50.839604 (+1514358us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400667454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:50.839806 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436511104}
I20260709 13:00:50.878266 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:50.878566 22546 consensus_meta.cc:287] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.600s	user 0.014s	sys 0.000s
I20260709 13:00:50.879149 20164 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:50.837414 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 914) took 1500 ms. Trace:
I20260709 13:00:50.880877 20163 rpcz_store.cc:276] 0709 13:00:49.337453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.504953 (+167500us) service_pool.cc:224] Handling call
0709 13:00:50.837123 (+1332170us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b540049be54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:50.837312 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172364544}
I20260709 13:00:50.881659 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.882725 20859 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:50.882969 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 915) took 1545 ms. Trace:
I20260709 13:00:50.835474 22275 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.796715 22803 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.884598 22275 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:50.835860 22709 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:50.850418 22542 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.850000 22688 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.812166 22465 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.821125 22881 raft_consensus.cc:1081] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:00:50.883709 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:50.854818 22833 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.813206 22482 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:50.815510 22756 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.848394 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1094) took 1607 ms. Trace:
I20260709 13:00:50.900292 20852 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:00:50.911212 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.922575 20164 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 13:00:50.916486 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.922996 20164 pending_rounds.cc:85] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 4
I20260709 13:00:50.923271 20164 pending_rounds.cc:107] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:00:50.908932 20859 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:50.923477 20164 raft_consensus.cc:2889] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:50.883488 20163 rpcz_store.cc:276] 0709 13:00:49.337795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.881959 (+1544164us) service_pool.cc:224] Handling call
0709 13:00:50.882325 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.882559 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.882597 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.882653 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:50.882697 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.882766 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.882904 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.858101 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:50.904336 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 840) took 1705 ms. Trace:
I20260709 13:00:50.903424 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 843) took 1658 ms. Trace:
I20260709 13:00:50.924599 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 916) took 1586 ms. Trace:
I20260709 13:00:50.924640 20013 rpcz_store.cc:276] 0709 13:00:49.199128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.200338 (+  1210us) service_pool.cc:224] Handling call
0709 13:00:49.202232 (+  1894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.293181 (+ 90949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.293233 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.293296 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:49.293346 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.904032 (+1610686us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b54000faad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:50.904117 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.904265 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208543744}
I20260709 13:00:50.906219 22744 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:00:50.927609 22744 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:00:50.914103 22876 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:00:50.939023 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:50.879283 22546 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:50.918774 20015 rpcz_store.cc:276] 0709 13:00:49.240947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.242039 (+  1092us) service_pool.cc:224] Handling call
0709 13:00:49.242349 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:49.242700 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:49.242727 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:49.251752 (+  9025us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.711078 (+1459326us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b18001f88e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:00:50.776578 (+ 65500us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:50.776620 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.776688 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.848080 (+ 71392us) raft_consensus.cc:1685] finished
0709 13:00:50.848102 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.848327 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5659,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":9489,"spinlock_wait_cycles":33909632,"thread_start_us":65297,"threads_started":1,"wal-append.queue_time_us":1561151}
I20260709 13:00:50.931950 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:50.945833 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.950376 20859 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.924804 20009 rpcz_store.cc:276] 0709 13:00:49.245472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.245615 (+   143us) service_pool.cc:224] Handling call
0709 13:00:49.245895 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.291091 (+ 45196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.291131 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.291180 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:49.291212 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.903115 (+1611903us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b54002c9554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:50.903173 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.903344 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103410048}
I20260709 13:00:50.934180 20162 raft_consensus.cc:2393] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:50.934876 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.925304 20164 rpcz_store.cc:276] 0709 13:00:49.338108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.921897 (+1583789us) service_pool.cc:224] Handling call
0709 13:00:50.922245 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.924123 (+  1878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.924531 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.951784 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:50.952137 20164 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:50.955130 20015 raft_consensus.cc:1240] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:50.907262 22841 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:50.962126 20009 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:50.965798 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 173796us
I20260709 13:00:50.887351 22275 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.889248 22834 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:50.970479 20082 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:50.965492 20013 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:50.929425 22614 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:00:50.981900 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 247873us
I20260709 13:00:50.988801 20055 leader_election.cc:400] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8.
I20260709 13:00:50.989812 20055 leader_election.cc:403] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:00:50.991674 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:50.995353 20854 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:50.929137 22756 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.005723 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:51.005172 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.007122 22833 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:00:50.996483 22631 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:00:51.004570 22275 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:51.013271 22611 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:51.013931 22611 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.998660 22614 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.021607 22275 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:50.981705 22094 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:00:51.022285 22275 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.031314 22691 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.033331 20010 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:51.037369 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:51.037912 20857 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.040588 22275 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.041148 22874 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.042236 20165 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:51.047107 22614 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.048027 22874 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.053567 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:51.023213 22887 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.107842 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.096458 20857 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:51.111706 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:51.113586 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:51.129622 20087 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:51.140204 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.145711 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.146209 20087 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:51.146699 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.147794 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.151160 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.151803 20086 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:51.152660 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.148669 20082 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
W20260709 13:00:51.152170 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 142815us
I20260709 13:00:51.153431 20084 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:51.157259 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:51.104099 22601 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:51.165751 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.166626 20013 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:51.160645 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.159669 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.162062 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.138937 22888 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:51.173358 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:51.151495 22735 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.001s
I20260709 13:00:51.173846 20086 raft_consensus.cc:1240] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:51.175864 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.129793 22275 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.004453 22744 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:51.204037 22275 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:51.204823 22744 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:51.206650 22275 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.208176 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:51.211331 20856 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:00:51.214988 20856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 876) took 1501 ms (client timeout 1966 ms). Trace:
W20260709 13:00:51.187290 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.165076 22601 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.190876 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.193497 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.196918 22756 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.173266 22696 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.200376 22546 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:51.180450 22833 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:00:51.180908 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:51.213263 20159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1022) took 2054 ms (client timeout 1889 ms). Trace:
I20260709 13:00:51.173883 22888 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:51.213850 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.232861 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.233970 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:51.216190 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.215613 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:51.158262 22623 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:51.110368 22846 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:51.215483 20856 rpcz_store.cc:269] 0709 13:00:49.713507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.713958 (+   451us) service_pool.cc:224] Handling call
0709 13:00:51.191780 (+1477822us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:51.214913 (+ 23133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6646528}
I20260709 13:00:51.219861 20016 raft_consensus.cc:2410] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:51.221292 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.223611 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.238613 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.224184 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.240418 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.226162 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.240453 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.203636 22621 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:51.228484 20159 rpcz_store.cc:269] 0709 13:00:49.159221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.159349 (+   128us) service_pool.cc:224] Handling call
0709 13:00:51.208092 (+2048743us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54006edf54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:51.213186 (+  5094us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383899904}
I20260709 13:00:51.174358 22885 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.219149 22601 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.228516 22881 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:51.244769 22888 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:51.228205 22696 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.250034 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:51.248263 22887 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8.
I20260709 13:00:51.251992 22623 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:51.252853 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.226467 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 556) took 1024 ms. Trace:
I20260709 13:00:51.266785 20856 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:51.258565 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.213042 22735 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.260998 20087 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:51.240662 22542 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:51.266989 20164 rpcz_store.cc:276] 0709 13:00:50.202228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.952974 (+750746us) service_pool.cc:224] Handling call
0709 13:00:50.953290 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.953521 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:51.066274 (+112753us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.225998 (+159724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.226081 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:51.226143 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.226236 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.226389 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":227616,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2037,"raft.queue_time_us":2819,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":7655,"thread_start_us":271116,"threads_started":2}
I20260709 13:00:51.276346 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:51.269080 22482 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.271085 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.272914 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:51.268394 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:51.275203 20850 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:51.279073 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.280146 20164 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:51.283736 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:51.286728 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:51.255398 22850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.020s	sys 0.002s
I20260709 13:00:51.252642 22874 consensus_queue.cc:1059] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:00:51.290158 22846 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.294948 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:51.293473 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 77920.7us
I20260709 13:00:51.277801 22888 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.294919 20090 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:51.317824 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:51.318789 20162 raft_consensus.cc:2393] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:51.320361 20159 raft_consensus.cc:2393] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:51.321097 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:51.322669 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:51.320399 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:51.293820 22850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.021s	sys 0.002s
I20260709 13:00:51.345944 20164 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:51.267127 22885 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:51.362203 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:51.363101 20159 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:51.361438 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.376883 20162 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:51.385936 22841 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:51.388152 22896 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:00:51.388829 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.393116 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 549) took 1208 ms. Trace:
I20260709 13:00:51.393641 20163 rpcz_store.cc:276] 0709 13:00:50.184886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.936283 (+751397us) service_pool.cc:224] Handling call
0709 13:00:50.936647 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.937117 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.937168 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.937732 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:00:51.176360 (+238628us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:51.176463 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.176598 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:51.392875 (+216277us) raft_consensus.cc:1685] finished
0709 13:00:51.392898 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.393053 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":22443,"thread_start_us":176556,"threads_started":1,"wal-append.queue_time_us":279860}
I20260709 13:00:51.397662 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 458) took 1183 ms. Trace:
I20260709 13:00:51.397876 22482 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.397809 20014 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:51.398252 20012 rpcz_store.cc:276] 0709 13:00:50.214105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.220533 (+  6428us) service_pool.cc:224] Handling call
0709 13:00:51.397437 (+1176904us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54002463d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:51.397601 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470883712}
I20260709 13:00:51.352957 22826 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:51.404569 22826 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.407119 22826 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.408595 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.408957 22696 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.411094 22885 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.411840 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.413156 22601 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.414407 22611 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:51.415349 22601 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.416522 22888 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:51.416644 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.416891 22877 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.419390 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:51.357702 22833 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.413295 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:51.420778 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 104343us
I20260709 13:00:51.418356 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:51.358498 22735 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.428587 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.428056 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 920) took 1182 ms. Trace:
I20260709 13:00:51.435883 20164 rpcz_store.cc:276] 0709 13:00:50.245938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.425575 (+1179637us) service_pool.cc:224] Handling call
0709 13:00:51.427544 (+  1969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.427715 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.427738 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.427777 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:51.427808 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.427859 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.427997 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:51.432370 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:51.437630 20012 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:51.417073 22888 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:51.416989 22621 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.445317 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:51.436749 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:51.475303 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 62081.3us
I20260709 13:00:51.477025 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.417487 22903 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:51.459784 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.474512 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.482311 20857 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:51.438123 20016 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:51.424578 22844 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:51.480904 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1101) took 1163 ms. Trace:
I20260709 13:00:51.486615 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.492379 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.493252 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:51.488930 20011 rpcz_store.cc:276] 0709 13:00:50.317771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.805318 (+487547us) service_pool.cc:224] Handling call
0709 13:00:50.805577 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.904232 (+ 98655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.904277 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.904337 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:50.904384 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.480621 (+576237us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b54004a74d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:51.480677 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.480841 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226195712}
I20260709 13:00:51.442973 22826 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.479616 22888 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:51.480788 22378 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:51.424377 22877 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:51.484609 22631 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:51.486023 22589 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:51.491906 22844 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.485123 22906 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.491724 22898 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:51.437811 22359 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:00:51.379619 22623 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.505632 22693 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.435186 22833 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.498040 22903 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:51.499504 20857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb11384495c2485aaab243a655f75c86"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 13:00:51.533965 20857 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:51.534385 20857 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.514767 22631 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:51.508622 22819 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:51.488447 22887 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:00:51.518523 22589 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.552183 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 851) took 1127 ms. Trace:
I20260709 13:00:51.552801 20014 rpcz_store.cc:276] 0709 13:00:50.424482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.830016 (+405534us) service_pool.cc:224] Handling call
0709 13:00:51.397601 (+567585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.398998 (+  1397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.551937 (+152939us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54002463d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:51.552108 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304680320}
I20260709 13:00:51.553442 22877 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:51.523177 22844 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.560755 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1095) took 1245 ms. Trace:
I20260709 13:00:51.561219 20010 rpcz_store.cc:276] 0709 13:00:50.315163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.800391 (+485228us) service_pool.cc:224] Handling call
0709 13:00:50.832063 (+ 31672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.038612 (+206549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.560495 (+521883us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400527bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:51.560697 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459760000}
I20260709 13:00:51.542678 22903 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:51.545771 20055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.008s	sys 0.011s
W20260709 13:00:51.566474 20055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.009s	sys 0.012s
W20260709 13:00:51.567137 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 562018us
I20260709 13:00:51.570603 20012 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:51.585245 20012 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:51.586158 20086 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:51.538255 22906 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:51.593585 20086 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:51.574800 22904 log.cc:927] Time spent T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 13:00:51.598577 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.541075 22819 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:51.541877 22631 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.552189 22589 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.588788 22207 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.614863 22819 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.632174 22207 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.633327 22844 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.635076 22207 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.636497 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:51.638897 22844 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.574460 22378 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:00:51.640385 22207 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:51.641073 22207 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:51.642256 22207 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.614287 22611 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:51.644572 22833 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.647606 22320 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:51.648221 22320 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:51.581691 20857 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:51.660694 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 926) took 1236 ms. Trace:
W20260709 13:00:51.622329 22907 log.cc:927] Time spent T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:00:51.664354 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.667966 20857 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:51.665733 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 175236us
I20260709 13:00:51.621217 22906 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.673374 22906 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.674614 22874 consensus_queue.cc:1059] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:51.672605 22898 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:51.678452 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:51.679004 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.666743 20158 rpcz_store.cc:276] 0709 13:00:50.424696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.422316 (+997620us) service_pool.cc:224] Handling call
0709 13:00:51.422752 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.423035 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:51.438989 (+ 15954us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.660251 (+221262us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.660306 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:51.660360 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.660428 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.660600 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19418,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":2616,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":4130,"thread_start_us":217054,"threads_started":1}
I20260709 13:00:51.680713 22874 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:51.673374 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 924) took 1255 ms. Trace:
I20260709 13:00:51.683897 22877 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:51.684335 22877 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:51.682684 20162 rpcz_store.cc:276] 0709 13:00:50.418020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.387448 (+969428us) service_pool.cc:224] Handling call
0709 13:00:51.414045 (+ 26597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.414264 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:51.414564 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.672694 (+258130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.672766 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:51.672815 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.672903 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.673291 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":581,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":57236,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":10103,"thread_start_us":257649,"threads_started":2,"wal-append.queue_time_us":235660}
I20260709 13:00:51.685179 22877 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:51.676498 22906 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.691872 20857 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:51.692322 20086 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:51.714298 20857 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
W20260709 13:00:51.705773 20156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1039) took 1414 ms (client timeout 1830 ms). Trace:
I20260709 13:00:51.692186 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.699407 20158 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
W20260709 13:00:51.715164 20156 rpcz_store.cc:269] 0709 13:00:50.291038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.573354 (+282316us) service_pool.cc:224] Handling call
0709 13:00:51.705603 (+1132249us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b54004d7454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:51.705712 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71822080}
I20260709 13:00:51.715829 20857 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:00:51.718532 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 241607us
I20260709 13:00:51.719307 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.685671 20162 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:51.641341 22888 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:51.724517 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44624.7us
I20260709 13:00:51.721170 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.720750 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 559) took 1250 ms. Trace:
I20260709 13:00:51.687860 22877 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.725737 19981 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:51.693285 22600 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:51.728277 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.694741 22820 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:51.726361 20162 rpcz_store.cc:276] 0709 13:00:50.469971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.684986 (+1215015us) service_pool.cc:224] Handling call
0709 13:00:51.685328 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.720470 (+ 35142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.720678 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:51.732527 22906 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.732892 22819 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.733299 22906 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:51.735033 22819 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.696034 22898 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:00:51.735388 19982 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.730659 22820 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:51.736313 22906 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.736807 22820 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.737704 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.738257 22820 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:51.738512 22906 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.739034 22631 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:51.740155 22826 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.741114 22906 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:51.741587 22906 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.729930 22499 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.742069 22906 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.742440 22499 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.747534 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.748277 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.748445 22600 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:51.749152 22922 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:51.749624 22922 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.750316 22922 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.750540 22207 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.751361 22898 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:51.751801 22898 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:51.752681 22898 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.749612 22600 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.748492 20090 raft_consensus.cc:2410] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:51.755983 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:51.756762 20090 raft_consensus.cc:2410] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:51.756567 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:51.758040 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.722764 22844 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:51.725021 22589 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:51.741624 22866 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.278s	user 0.003s	sys 0.002s
I20260709 13:00:51.759429 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.759088 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.762315 20744 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:51.763898 20091 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:51.768615 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.785346 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.786571 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.787858 20744 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:51.798265 20744 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:51.803382 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.748271 22499 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.805531 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.809738 22715 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:51.811998 22852 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:51.815541 22836 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:51.817168 20160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1024) took 2589 ms (client timeout 1753 ms). Trace:
I20260709 13:00:51.817664 22589 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:51.817596 20160 rpcz_store.cc:269] 0709 13:00:49.228218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.228339 (+   121us) service_pool.cc:224] Handling call
0709 13:00:51.816912 (+2588573us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400573354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:51.817070 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251744000}
I20260709 13:00:51.728922 22887 consensus_queue.cc:1059] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:00:51.820608 22887 raft_consensus.cc:3055] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:51.820192 22906 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.821139 22887 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:51.822100 22887 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.823686 22906 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:51.825142 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 852) took 1109 ms. Trace:
I20260709 13:00:51.825567 20850 rpcz_store.cc:276] 0709 13:00:50.715543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.716010 (+   467us) service_pool.cc:224] Handling call
0709 13:00:50.716301 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.278539 (+562238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.824904 (+546365us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54001900d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:51.825086 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284792704}
I20260709 13:00:51.764097 22833 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.823796 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 887) took 1619 ms. Trace:
I20260709 13:00:51.827934 20855 rpcz_store.cc:276] 0709 13:00:50.193014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.201098 (+  8084us) service_pool.cc:224] Handling call
0709 13:00:50.201404 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.205338 (+  3934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.205367 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.205808 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.361261 (+155453us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:50.361300 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.361368 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:51.695701 (+1334333us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:51.811606 (+115905us) raft_consensus.cc:1685] finished
0709 13:00:51.811630 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.811881 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3542,"raft.queue_time_us":5143,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":4974,"spinlock_wait_cycles":113104128,"thread_start_us":115852,"threads_started":1,"wal-append.queue_time_us":251062}
W20260709 13:00:51.830951 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45710.5us
I20260709 13:00:51.787631 22847 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:00:51.731217 22601 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:00:51.787904 22844 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:51.836778 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.798662 22747 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:51.857282 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:51.845474 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 125979us
I20260709 13:00:51.799940 22890 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:51.800081 22834 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:51.861816 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.862279 19982 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.860570 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:51.862416 20162 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.836035 22906 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.864131 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:51.854033 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:51.853106 22606 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.866457 22606 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.867359 22844 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.867609 22610 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:51.837018 22918 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:51.871896 22631 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:51.867621 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:51.854532 22833 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.874192 22918 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:51.886821 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:51.873777 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:51.876900 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.875522 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:51.888936 20159 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.881982 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.855450 22499 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:51.883129 22839 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:00:51.860040 22696 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:51.898473 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:51.931636 22696 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:51.932243 22629 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.933137 22629 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.933646 22629 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.936650 22629 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:51.887115 22833 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.939546 22833 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:51.941643 20088 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:51.909605 22820 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.915863 22888 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:51.918749 22542 raft_consensus.cc:3055] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:51.919821 20853 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:51.921269 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 873) took 1201 ms. Trace:
I20260709 13:00:51.922693 20089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1177) took 2749 ms. Trace:
I20260709 13:00:51.972154 20159 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:51.865221 22847 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:51.876345 22622 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:51.860918 22890 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.902823 22741 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:51.845870 22922 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:51.990374 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.992101 20010 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:51.967284 20163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 925) took 1545 ms. Trace:
I20260709 13:00:51.972774 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:51.972923 20852 rpcz_store.cc:276] 0709 13:00:50.719359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.721264 (+  1905us) service_pool.cc:224] Handling call
0709 13:00:50.721588 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.920654 (+199066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.920919 (+1000265us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54000064d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:51.921192 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410177280}
W20260709 13:00:51.993883 20164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.039s	sys 0.046s
W20260709 13:00:51.994354 20164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.039s	sys 0.046s
I20260709 13:00:51.997313 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:51.980712 22925 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.094s	user 0.005s	sys 0.000s
W20260709 13:00:51.973639 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 99935.3us
W20260709 13:00:51.976042 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.970854 22918 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:51.977794 22833 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.003547 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:51.989702 22908 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.449s	user 0.009s	sys 0.007s
I20260709 13:00:51.985607 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.984051 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.989756 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.990083 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:51.992285 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:51.977483 22542 raft_consensus.cc:740] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:51.980496 22905 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.385s	user 0.000s	sys 0.006s
I20260709 13:00:52.002004 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:51.847610 22320 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:51.977205 22499 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:52.002205 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:51.972417 20089 rpcz_store.cc:276] 0709 13:00:49.173469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.173603 (+   134us) service_pool.cc:224] Handling call
0709 13:00:49.181651 (+  8048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.678740 (+1497089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.922375 (+1243635us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54005a1754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:51.922597 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3401416448}
I20260709 13:00:51.971501 22874 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:51.986944 22622 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.025305 20163 rpcz_store.cc:276] 0709 13:00:50.421755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.395195 (+973440us) service_pool.cc:224] Handling call
0709 13:00:51.681609 (+286414us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:00:51.681873 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.682174 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:51.682549 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.966766 (+284217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:51.966845 (+    79us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:51.966915 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.967011 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:51.967189 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2379,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":5808,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":11503,"spinlock_wait_cycles":55172864,"thread_start_us":283750,"threads_started":1}
I20260709 13:00:51.988081 22834 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:52.011834 20016 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:52.038335 22622 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.015117 20014 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:51.987373 22890 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:52.015759 20008 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.005800 20088 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.019832 20083 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:52.040357 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:51.977875 22820 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:52.039781 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:52.021854 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:52.036125 20852 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:52.013664 20012 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:52.052253 22606 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:52.052548 22542 consensus_queue.cc:260] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:52.054451 22542 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.055169 22820 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:52.013145 22833 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.058408 22833 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:52.040019 22874 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:52.062029 22836 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.062959 22931 consensus_queue.cc:1059] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.064013 22898 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.041217 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:52.045656 20085 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:00:52.070019 20090 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:52.072716 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:52.089222 20009 raft_consensus.cc:2368] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:52.108666 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:52.109619 20852 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:52.039714 22834 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:52.114190 20850 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:52.162667 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 364491us
I20260709 13:00:52.157780 22606 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:52.170881 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.176465 22621 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.171901 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 284658us
I20260709 13:00:52.116155 22833 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.172250 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:52.173631 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 130676us
I20260709 13:00:52.117415 22852 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:52.181128 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:52.180400 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:52.176895 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:52.200012 20008 pending_rounds.cc:85] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 5
I20260709 13:00:52.204838 20008 pending_rounds.cc:107] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:00:52.205260 20008 raft_consensus.cc:2889] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:52.203577 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:52.202222 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:52.208645 20012 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:00:52.208380 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 179568us
I20260709 13:00:52.173254 22820 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.210364 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:52.215991 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:52.175609 22606 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:52.220466 20014 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:00:52.181424 22923 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.232400 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.223155 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.230397 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.232499 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:52.176908 22885 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.219503 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.195312 22207 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:52.260160 19981 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:52.205957 22596 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.264061 20156 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:52.264494 20159 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:52.280681 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 69620.1us
I20260709 13:00:52.263851 20163 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:52.288887 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:52.294598 20012 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.289644 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:52.292812 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.262923 20162 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:52.295823 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:52.296022 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:52.296998 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:52.298370 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.300825 19982 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:52.303117 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:52.199113 22621 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.218514 22836 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:52.334286 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.306643 20088 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:52.336432 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:52.260432 22747 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:52.315367 20012 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:52.339902 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:52.206847 22833 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.355384 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:52.361807 20012 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:52.361675 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:52.358093 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62236.1us
I20260709 13:00:52.351886 20158 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:52.265645 22839 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 13:00:52.373560 20159 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:52.349958 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:52.375982 20016 raft_consensus.cc:2410] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:52.359889 20163 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:52.378463 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:52.379578 20156 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:52.235297 22820 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.407022 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:52.411348 20014 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:00:52.284155 22923 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:52.412569 22935 log.cc:927] Time spent T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:00:52.279739 22852 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:00:52.412858 22946 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:52.412909 20163 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:52.414108 22923 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:52.333129 22596 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:52.414856 22938 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:00:52.290979 22885 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:52.415853 20163 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:52.416538 22924 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:00:52.416600 22596 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:52.417217 22885 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:52.417462 22596 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:52.417917 22596 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.417972 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1042) took 1955 ms. Trace:
I20260709 13:00:52.418412 22596 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:52.418952 22729 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:52.419237 22930 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.315s	user 0.000s	sys 0.002s
W20260709 13:00:52.419412 22934 log.cc:927] Time spent T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:00:52.419792 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 868) took 2396 ms. Trace:
W20260709 13:00:52.421058 22927 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:00:52.413992 22941 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.418452 20164 rpcz_store.cc:276] 0709 13:00:50.462165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.436798 (+974633us) service_pool.cc:224] Handling call
0709 13:00:51.437155 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.437374 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.437397 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:51.480002 (+ 42605us) log.cc:844] Serialized 26 byte log entry
0709 13:00:51.995156 (+515154us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:51.995198 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.995262 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.417771 (+422509us) raft_consensus.cc:1685] finished
0709 13:00:52.417790 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.417910 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":503728,"threads_started":1,"wal-append.queue_time_us":689609}
I20260709 13:00:52.331779 22942 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.426899 20856 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:52.423908 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138790us
I20260709 13:00:52.265074 22499 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:00:52.349393 22914 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:52.210986 22918 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:52.442646 20854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 875) took 2713 ms (client timeout 1967 ms). Trace:
W20260709 13:00:52.443089 20854 rpcz_store.cc:269] 0709 13:00:49.713143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.713279 (+   136us) service_pool.cc:224] Handling call
0709 13:00:50.840354 (+1127075us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b58000bdf98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:52.426029 (+1585675us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400240254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:52.426251 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273261824}
I20260709 13:00:52.420197 20858 rpcz_store.cc:276] 0709 13:00:50.023370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.028669 (+  5299us) service_pool.cc:224] Handling call
0709 13:00:50.028990 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:50.038228 (+  9238us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:50.050101 (+ 11873us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.184280 (+1134179us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b18003f80e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:00:51.742611 (+558331us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b18003f80e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287
0709 13:00:51.743045 (+   434us) log.cc:844] Serialized 14 byte log entry
0709 13:00:51.836084 (+ 93039us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:51.857077 (+ 20993us) log.cc:844] Serialized 52 byte log entry
0709 13:00:51.857246 (+   169us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:51.857322 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.857437 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.419591 (+562154us) raft_consensus.cc:1685] finished
0709 13:00:52.419612 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.419743 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11950,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"raft.queue_time_us":91599,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":19670,"spinlock_wait_cycles":50921600,"thread_start_us":354872,"threads_started":3,"wal-append.queue_time_us":665108,"wal-append.run_cpu_time_us":1079,"wal-append.run_wall_time_us":1001109}
W20260709 13:00:52.445196 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.233731 22834 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:52.352152 22747 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:52.383263 22940 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:52.419440 22729 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.409510 22320 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:52.305513 22207 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:52.445917 22914 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:52.487815 22877 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.757s	user 0.008s	sys 0.004s
I20260709 13:00:52.458894 22747 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:52.483265 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 195546us
I20260709 13:00:52.453125 22885 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:52.508097 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 541) took 3244 ms. Trace:
I20260709 13:00:52.474476 22729 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.477588 22834 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.511484 20157 rpcz_store.cc:276] 0709 13:00:49.252167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.253562 (+  1395us) service_pool.cc:224] Handling call
0709 13:00:50.240181 (+986619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.240476 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:50.347451 (+106975us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.518868 (+171417us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.518925 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:50.519089 (+   164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.519162 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.496602 (+1977440us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b5400586e50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:52.496735 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11800,"prepare.queue_time_us":184996,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":54296,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4490,"spinlock_wait_cycles":2272871040,"thread_start_us":277138,"threads_started":3,"wal-append.queue_time_us":188508}
I20260709 13:00:52.496675 22951 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.445534 22606 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:52.529489 22947 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:00:52.530375 22932 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:00:52.447767 22942 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:52.499477 22914 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.520972 22874 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:52.527523 22926 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
W20260709 13:00:52.555897 22542 consensus_meta.cc:287] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.000s
I20260709 13:00:52.477100 22940 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.520161 22885 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.568930 22941 consensus_queue.cc:1243] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:52.572372 22937 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:00:52.545547 22951 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:52.609210 22942 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:52.615896 22951 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:52.592573 22833 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:52.617877 22877 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:52.618639 22877 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:52.619086 22940 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:52.637360 22885 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:52.641052 22890 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.000s
I20260709 13:00:52.619897 22877 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.651319 22877 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:52.642175 22890 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:52.675525 22922 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:52.723620 22890 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:52.737082 22946 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:52.740275 22914 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:52.748672 22914 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:52.795944 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 494965us
I20260709 13:00:52.804618 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:52.804874 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:52.807898 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.811792 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.814548 20084 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.817826 20085 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:52.527346 20162 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:49.980232 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:52.829582 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1214) took 1527 ms. Trace:
I20260709 13:00:52.858793 20086 rpcz_store.cc:276] 0709 13:00:51.295033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.603543 (+308510us) service_pool.cc:224] Handling call
0709 13:00:51.603972 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.714019 (+110047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.821707 (+1107688us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b54006653d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:52.822110 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202704640}
W20260709 13:00:52.834430 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.840651 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 335894us
I20260709 13:00:52.861935 20744 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:52.862756 22207 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:52.864921 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 868) took 1266 ms. Trace:
I20260709 13:00:52.841642 20851 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:52.866750 20744 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:52.865598 20857 rpcz_store.cc:276] 0709 13:00:51.598461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.717161 (+118700us) service_pool.cc:224] Handling call
0709 13:00:51.717516 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.718059 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.718093 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:51.722428 (+  4335us) log.cc:844] Serialized 26 byte log entry
0709 13:00:52.045183 (+322755us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:52.045232 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.045302 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.864169 (+818867us) raft_consensus.cc:1685] finished
0709 13:00:52.864229 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.864591 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":489,"raft.queue_time_us":92,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":5220,"thread_start_us":272581,"threads_started":1,"wal-append.queue_time_us":439624}
W20260709 13:00:52.868584 22871 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.201s	user 0.006s	sys 0.003s
I20260709 13:00:52.740190 22946 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:52.870057 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1058) took 1710 ms. Trace:
I20260709 13:00:52.870383 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:52.870687 20087 rpcz_store.cc:276] 0709 13:00:51.158992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.159430 (+   438us) service_pool.cc:224] Handling call
0709 13:00:52.869512 (+1710082us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b54009b8a54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:52.869740 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208736000}
I20260709 13:00:52.872542 22621 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:52.872988 22885 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:52.873847 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.874219 22885 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.875269 22693 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.875145 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:52.846958 22965 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.100s	user 0.000s	sys 0.008s
I20260709 13:00:52.850018 22852 consensus_queue.cc:1243] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:52.845400 20085 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:52.588737 22820 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:52.857656 22894 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.613s	user 0.042s	sys 0.086s
I20260709 13:00:52.938907 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:52.943583 20858 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:52.876765 22693 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:52.945031 22611 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:52.945667 22611 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:52.948805 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 462) took 1727 ms. Trace:
I20260709 13:00:52.942747 20085 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:52.949678 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:52.950732 20858 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:52.949301 20091 rpcz_store.cc:276] 0709 13:00:51.221405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.552075 (+330670us) service_pool.cc:224] Handling call
0709 13:00:51.552391 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:51.797464 (+245073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.948217 (+1150753us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400258354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:52.948602 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485920000}
I20260709 13:00:52.952525 22693 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:52.878502 22933 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.458s	user 0.004s	sys 0.008s
I20260709 13:00:52.879338 22890 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:52.956802 22884 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:52.956661 22890 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:52.957773 22884 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:52.880224 22881 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:52.960886 22881 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:52.961959 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 157713us
I20260709 13:00:52.882079 22914 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:52.885212 22939 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.466s	user 0.008s	sys 0.005s
W20260709 13:00:52.886052 22963 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.243s	user 0.000s	sys 0.010s
I20260709 13:00:52.888525 22922 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:52.889169 22959 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.893426 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:52.905663 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.906049 22378 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:52.915432 20165 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 922) took 2577 ms (client timeout 1689 ms). Trace:
I20260709 13:00:52.919262 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1060) took 1759 ms. Trace:
I20260709 13:00:52.925384 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 878) took 1905 ms. Trace:
I20260709 13:00:52.929541 20851 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:00:52.932855 22596 consensus_meta.cc:287] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.004s
I20260709 13:00:52.932976 22887 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:52.940985 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 938) took 1533 ms. Trace:
I20260709 13:00:52.961818 22600 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:52.842693 22542 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:52.963994 22884 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:52.964797 22890 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:52.965750 22961 log.cc:927] Time spent T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.377s	user 0.010s	sys 0.000s
I20260709 13:00:53.036533 20161 rpcz_store.cc:276] 0709 13:00:51.407833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.408532 (+   699us) service_pool.cc:224] Handling call
0709 13:00:52.874655 (+1466123us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b58003e0498. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:00:52.940873 (+ 66218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21429632}
I20260709 13:00:52.967901 22929 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:00:52.970839 22968 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 13:00:52.972074 22969 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.215s	user 0.003s	sys 0.000s
I20260709 13:00:52.976109 22696 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:52.984000 22914 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:53.022060 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.987838 20165 rpcz_store.cc:269] 0709 13:00:50.337706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.580719 (+243013us) service_pool.cc:224] Handling call
0709 13:00:52.914703 (+2333984us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54002ec2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:52.914978 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059481856}
W20260709 13:00:53.066064 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 81724.4us
I20260709 13:00:52.989065 20856 rpcz_store.cc:276] 0709 13:00:51.019781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.424958 (+405177us) service_pool.cc:224] Handling call
0709 13:00:52.426636 (+1001678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:52.427563 (+   927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.924696 (+497133us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400240250. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:52.925132 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2302729600}
I20260709 13:00:52.988152 20082 rpcz_store.cc:276] 0709 13:00:51.159588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.159699 (+   111us) service_pool.cc:224] Handling call
0709 13:00:52.917951 (+1758252us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400782154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:52.919022 (+  1071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156435200}
I20260709 13:00:53.068624 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:53.066211 22281 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.083421 22715 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.084635 22600 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:53.085444 22600 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:53.092584 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 939) took 1237 ms. Trace:
I20260709 13:00:53.096385 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.109760 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:53.112886 20859 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:53.114403 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.115728 20859 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:53.110136 22755 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:53.120512 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 242977us
W20260709 13:00:53.123610 20853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 863) took 3101 ms (client timeout 1833 ms). Trace:
I20260709 13:00:53.088110 22971 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:00:53.068838 22596 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.104933 22281 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.090780 22929 raft_consensus.cc:3055] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:53.094383 22820 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:53.107530 22881 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.098713 22914 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.142153 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:53.148137 22971 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:53.148906 22542 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.150334 22596 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.081085 22378 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.151242 20850 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:53.151165 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:53.152137 20857 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:53.092094 22962 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:53.156363 22820 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:53.158056 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 51313.3us
I20260709 13:00:53.158509 20856 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:00:53.157274 20744 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:53.131767 20853 rpcz_store.cc:269] 0709 13:00:50.021901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.029418 (+  7517us) service_pool.cc:224] Handling call
0709 13:00:53.122970 (+3093552us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54000037d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:53.123354 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4345183104}
I20260709 13:00:53.169898 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 451) took 2962 ms. Trace:
I20260709 13:00:53.140734 22929 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:53.171535 20084 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:53.105130 20160 rpcz_store.cc:276] 0709 13:00:51.855087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.855950 (+   863us) service_pool.cc:224] Handling call
0709 13:00:51.872380 (+ 16430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.872718 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.872750 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:51.873220 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:00:52.167958 (+294738us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:52.168007 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.168076 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.698245 (+530169us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:00:53.092016 (+393771us) raft_consensus.cc:1685] finished
0709 13:00:53.092070 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.092353 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36568,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":14140800,"thread_start_us":294411,"threads_started":1,"wal-append.queue_time_us":545492}
I20260709 13:00:53.173453 20089 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:53.172894 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.180356 22281 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:53.180545 20854 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:53.167089 22971 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:53.184602 22881 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.160691 20851 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:53.170758 22542 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:53.252053 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 44769.9us
I20260709 13:00:53.254096 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:53.087479 22600 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:53.257242 22834 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.736s	user 0.017s	sys 0.004s
I20260709 13:00:53.190115 22696 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.257566 22600 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.258112 22696 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:53.264634 22747 consensus_meta.cc:287] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.748s	user 0.017s	sys 0.011s
I20260709 13:00:53.265450 22747 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.268658 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:53.269225 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 112156us
I20260709 13:00:53.271041 20130 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:53.272441 22923 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:53.270941 20744 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:53.279111 20159 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.280900 22937 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.281893 22937 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.283262 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 900) took 1267 ms. Trace:
I20260709 13:00:53.283663 20855 rpcz_store.cc:276] 0709 13:00:52.016212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.016419 (+   207us) service_pool.cc:224] Handling call
0709 13:00:52.047615 (+ 31196us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:52.048128 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:52.048164 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:52.089931 (+ 41767us) log.cc:844] Serialized 52 byte log entry
0709 13:00:52.239396 (+149465us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:52.239455 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.239573 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.842220 (+602647us) raft_consensus.cc:1685] finished
0709 13:00:52.842269 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.259714 (+417445us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b54001a0250. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:53.283085 (+ 23371us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9084,"raft.queue_time_us":637,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":15308,"spinlock_wait_cycles":19426560,"thread_start_us":72954,"threads_started":1,"wal-append.queue_time_us":393174}
I20260709 13:00:53.286530 22923 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:53.287102 22923 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.287467 22923 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.289701 22207 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.290751 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:53.291003 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:53.291879 22207 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:53.299388 22207 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:53.300400 22922 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.292452 20161 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 5: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:53.302500 22962 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.293485 20158 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:53.303264 22207 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.289700 22589 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:00:53.296311 22622 consensus_meta.cc:287] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.257s	user 0.017s	sys 0.011s
I20260709 13:00:53.308708 22820 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.308600 22622 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.096618 22959 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:53.310279 20013 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:53.311896 22959 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:53.293773 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.313156 20162 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
W20260709 13:00:53.312238 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.315851 22971 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.298409 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.317143 22971 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.317211 20159 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:53.317827 22881 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.319218 22890 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:53.319718 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 1118) took 1328 ms. Trace:
I20260709 13:00:53.312228 22622 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.321324 20858 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:53.294209 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 100419us
I20260709 13:00:53.332270 22542 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.334241 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:53.334359 22542 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.335327 20162 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:53.336881 22611 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.339097 22611 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.323436 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.330070 20013 rpcz_store.cc:276] 0709 13:00:51.991633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.000204 (+  8571us) service_pool.cc:224] Handling call
0709 13:00:53.319295 (+1319091us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54005cec54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:53.319613 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2982331264}
I20260709 13:00:53.340947 20853 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:53.347891 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1215) took 1914 ms. Trace:
I20260709 13:00:53.348459 20090 rpcz_store.cc:276] 0709 13:00:51.433372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.757382 (+324010us) service_pool.cc:224] Handling call
0709 13:00:51.757677 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:52.113297 (+355620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.344706 (+1231409us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540078f854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:53.347662 (+  2956us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717888768}
I20260709 13:00:53.350004 22378 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.219310 22921 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:53.320741 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:53.192328 20017 rpcz_store.cc:276] 0709 13:00:50.207425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.207572 (+   147us) service_pool.cc:224] Handling call
0709 13:00:50.208983 (+  1411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.209420 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.209802 (+   382us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.210345 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:00:52.799615 (+2589270us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800183fa0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:00:53.082432 (+282817us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:53.082509 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.082936 (+   427us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:53.161814 (+ 78878us) raft_consensus.cc:1685] finished
0709 13:00:53.162526 (+   712us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.169620 (+  7094us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136,"raft.queue_time_us":302,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":15358,"spinlock_wait_cycles":6833664,"thread_start_us":178069,"threads_started":1,"wal-append.queue_time_us":2931947}
I20260709 13:00:53.344259 22207 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.361799 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:53.369522 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 49007us
I20260709 13:00:53.367797 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.381567 20130 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:53.381518 22937 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:53.385600 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.385850 22962 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:53.358227 22589 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:00:53.386782 22962 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:00:53.387298 22962 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.387655 22962 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:53.386626 20130 leader_election.cc:400] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:53.401136 20130 leader_election.cc:403] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:00:53.392136 22877 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.016s
I20260709 13:00:53.386698 20013 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
W20260709 13:00:53.392819 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.393178 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 99896.7us
W20260709 13:00:53.394361 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 140550us
I20260709 13:00:53.396775 22834 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.397025 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.404668 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.376384 22622 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:53.406157 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:53.406051 22819 consensus_queue.cc:1059] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:00:53.410629 20853 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
I20260709 13:00:53.412585 20165 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:53.408991 20858 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:53.363909 22921 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.403905 22937 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:53.404191 22747 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:53.422979 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.424796 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.438545 22614 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:53.439126 20856 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:53.439390 22614 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.441757 22614 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.442612 22937 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.443270 22499 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:53.443806 22499 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:53.438760 22622 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:53.444854 22499 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.445987 22499 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.446748 22622 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:53.441782 20015 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:53.438742 20009 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
W20260709 13:00:53.447999 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 183366us
I20260709 13:00:53.434501 20859 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:00:53.413766 22834 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.455519 22542 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:53.456207 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.457063 22693 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:53.429509 22378 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:53.462275 22834 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.454718 22755 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.430927 22631 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.464704 22542 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:53.467197 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:53.468027 22729 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.910s	user 0.011s	sys 0.012s
I20260709 13:00:53.470685 22967 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:53.471531 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 65618.7us
I20260709 13:00:53.473521 22959 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:53.474355 22606 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:53.478566 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.453660 20091 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:53.476302 22693 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
W20260709 13:00:53.488349 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.490784 20853 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:53.483618 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:53.508539 22542 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.512157 22755 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:53.514339 22542 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.514530 22755 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:53.521795 20162 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:53.482957 22921 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:53.525591 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.493861 22631 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:53.507215 22881 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:53.507301 22967 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.532382 22931 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:53.533169 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.533407 22841 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.533859 20055 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:53.534729 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55954.1us
I20260709 13:00:53.497152 22622 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:53.534747 22631 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.522920 22589 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:53.546893 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.480878 22929 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.502981 22729 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.553601 20165 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:53.561117 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:53.537731 22959 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.578466 22744 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.579037 22744 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:53.579425 22744 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.554860 22622 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:53.587133 22378 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:53.588721 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.589780 22622 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:53.592008 22967 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.592934 20087 raft_consensus.cc:2368] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
W20260709 13:00:53.595112 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 142952us
I20260709 13:00:53.597734 22696 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:53.599002 22977 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.132s	user 0.001s	sys 0.001s
I20260709 13:00:53.599972 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:53.601015 22974 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.501s	user 0.000s	sys 0.004s
I20260709 13:00:53.603324 22281 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:53.573711 22600 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.574909 22841 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:00:53.604969 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.605795 20855 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:53.557430 22589 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:53.608357 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 74579.4us
I20260709 13:00:53.609586 20055 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:53.612116 22600 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.612636 22600 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:53.612530 22723 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:53.613153 22600 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.613895 22723 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:53.614246 22723 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:53.616434 22094 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.617158 22094 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.617695 22841 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.618486 22841 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.621310 22589 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:53.621981 22729 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.622601 22094 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.635430 22744 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.626192 22929 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:53.627290 22981 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:00:53.603782 22281 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:53.682724 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 874) took 1483 ms. Trace:
I20260709 13:00:53.682300 22281 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.621419 22841 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.624723 22959 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.639084 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:53.684667 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 45645.1us
I20260709 13:00:53.615464 22723 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:53.640997 22978 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
I20260709 13:00:53.686358 22723 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:53.685487 22890 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.687171 22890 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.687515 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.688135 22959 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:53.688601 22959 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.689016 22959 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:53.690214 22890 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.691125 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:53.643664 22744 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.693913 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.694931 20163 raft_consensus.cc:2410] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:53.695765 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.696120 20013 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:53.696540 20017 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:53.607797 22923 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.647943 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.652133 22755 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:53.652488 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:53.652737 22696 consensus_queue.cc:1059] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:53.653409 20015 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
W20260709 13:00:53.654068 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.652235 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.710819 20164 raft_consensus.cc:2368] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
W20260709 13:00:53.655201 22715 consensus_meta.cc:287] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.007s
I20260709 13:00:53.658017 22970 raft_consensus.cc:3055] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:53.712765 22970 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:53.712452 22755 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.713703 22970 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.715238 22970 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.663578 20085 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.683702 22281 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.716372 22755 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.712477 22696 raft_consensus.cc:3055] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:53.717363 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.718035 22696 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:53.711184 20158 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:53.683264 20008 rpcz_store.cc:276] 0709 13:00:52.199147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.199266 (+   119us) service_pool.cc:224] Handling call
0709 13:00:52.199788 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:52.206069 (+  6281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:52.206122 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:53.102720 (+896598us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:53.160395 (+ 57675us) log.cc:844] Serialized 26 byte log entry
0709 13:00:53.416372 (+255977us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:53.416424 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.429451 (+ 13027us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:53.682099 (+252648us) raft_consensus.cc:1685] finished
0709 13:00:53.682147 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.682499 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3166464,"thread_start_us":226200,"threads_started":1,"wal-append.queue_time_us":322632}
I20260709 13:00:53.723300 22715 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:53.664472 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 104280us
I20260709 13:00:53.719491 22696 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.725605 22715 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.726171 22696 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.726197 22715 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.727109 20129 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:53.676721 22979 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:00:53.728133 22378 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:53.729118 22378 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:53.729486 22378 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.729803 22378 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:53.741264 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.737919 20017 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:00:53.594266 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:53.615023 22962 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:53.680766 22980 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:00:53.688882 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.646565 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:53.683532 19839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32774 (request call id 152) took 18094 ms (client timeout 15000 ms). Trace:
W20260709 13:00:53.749378 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 171502us
I20260709 13:00:53.747318 20161 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:53.733822 22589 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:53.753432 20160 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:53.753896 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:53.751178 19982 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:53.752652 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:53.756716 20164 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.759217 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:53.758757 20009 raft_consensus.cc:2410] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:00:53.763407 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.783360 20013 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:00:53.781417 20129 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:53.790067 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.783797 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.791177 20161 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:53.752049 22589 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:53.734081 22890 consensus_queue.cc:1059] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:00:53.804279 22958 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.756345 22884 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:00:53.804564 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.805636 20859 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:53.827427 22841 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:53.779431 22744 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:53.792873 22606 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.828187 22884 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.814116 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.812912 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.784163 22914 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:53.812932 20156 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:53.789430 22820 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:53.861181 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.861330 22715 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.862497 22629 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.839340 22841 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.863075 22629 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.839438 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.864596 22836 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.865373 22951 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:00:53.865789 22378 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:53.838088 22962 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.839957 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:53.866703 22841 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:53.838014 20017 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:53.868116 20157 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:53.867810 20013 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:00:53.869118 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.869966 20157 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:53.870091 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.871018 20010 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
W20260709 13:00:53.840507 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44014.6us
I20260709 13:00:53.840087 22846 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:53.873605 22846 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:53.874701 20130 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:53.875600 22881 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:53.843175 19840 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:53.876299 22881 consensus_queue.cc:1243] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:53.843416 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:53.845212 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:53.845803 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 95218.5us
I20260709 13:00:53.848428 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:53.881979 20161 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:53.849448 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.849963 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:53.852737 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.853798 20082 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:00:53.840548 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 244641us
I20260709 13:00:53.854049 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:53.889765 20087 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:53.850359 20008 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:53.847663 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:53.897500 20163 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:00:53.897382 20855 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:53.877439 19840 catalog_manager.cc:5697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 10, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 10 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:53.893567 20164 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:53.901808 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:53.930930 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:53.934062 20017 raft_consensus.cc:2410] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:53.935246 20015 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:53.939178 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:53.933508 20009 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.848996 22884 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:53.846011 22207 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.938386 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.944384 20091 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:53.933723 22967 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:00:53.860330 22914 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:53.898900 22281 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:53.945815 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 71156.1us
I20260709 13:00:53.948205 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:53.860585 22820 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:53.946722 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.947587 19840 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 8, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
W20260709 13:00:53.948812 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 167512us
W20260709 13:00:53.962779 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.903119 22962 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.965399 20008 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:53.848330 22606 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:53.849000 22600 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:53.971050 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:53.847280 22958 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:53.973029 19840 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 6, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 6 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:53.973985 20087 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:53.947865 22846 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:53.936618 22841 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:00:53.977823 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:53.970413 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:53.995428 20082 raft_consensus.cc:2410] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:00:53.984323 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.960940 22378 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:53.968111 22820 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.003304 20085 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:53.994308 20013 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:54.007241 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1225) took 1197 ms. Trace:
I20260709 13:00:53.970234 22499 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.970419 22835 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:54.002219 19840 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 8, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 8 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.017763 20129 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:53.969973 22962 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:54.022039 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 89758.7us
I20260709 13:00:53.961896 22971 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:00:54.020768 22819 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.024784 22693 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.025316 22729 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.967445 22715 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:00:53.982946 22937 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:54.027652 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:53.966427 22959 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:54.038301 20850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.030s	sys 0.024s
W20260709 13:00:53.749815 19839 rpcz_store.cc:269] 0709 13:00:35.602048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.602374 (+   326us) service_pool.cc:224] Handling call
0709 13:00:53.683234 (+18080860us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:47.395777 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:47.673084 (+277307us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:47.685297 (+ 12213us) write_op.cc:435] Acquired schema lock
0709 13:00:47.688617 (+  3320us) tablet.cc:662] Decoding operations
0709 13:00:48.092138 (+403521us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:48.092220 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:48.092247 (+    27us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:00:48.093727 (+  1480us) tablet.cc:701] Row locks acquired
0709 13:00:48.093773 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:00:48.093938 (+   165us) write_op.cc:270] Start()
0709 13:00:48.094102 (+   164us) write_op.cc:276] Timestamp: P: 1783602048093875 usec, L: 0
0709 13:00:48.094123 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:00:48.094791 (+   668us) log.cc:844] Serialized 6698 byte log entry
0709 13:00:48.879133 (+784342us) op_driver.cc:464] REPLICATION: finished
0709 13:00:49.064781 (+185648us) write_op.cc:301] APPLY: starting
0709 13:00:49.064871 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:49.065195 (+   324us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:49.065316 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:49.067289 (+  1973us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:49.067561 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:00:49.067606 (+    45us) write_op.cc:312] APPLY: finished
0709 13:00:49.067964 (+   358us) log.cc:844] Serialized 118 byte log entry
0709 13:00:49.068780 (+   816us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:49.069190 (+   410us) write_op.cc:454] Released schema lock
0709 13:00:49.069263 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:49.075004 (+  5741us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:49.075049 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1422412,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":174179,"apply.run_cpu_time_us":4624,"apply.run_wall_time_us":10147,"num_ops":17,"prepare.queue_time_us":213303,"prepare.run_cpu_time_us":31281,"prepare.run_wall_time_us":970856,"raft.queue_time_us":205330,"raft.run_cpu_time_us":53385,"raft.run_wall_time_us":119893,"replication_time_us":784836,"thread_start_us":607775,"threads_started":4,"wal-append.queue_time_us":255783,"wal-append.run_cpu_time_us":40427,"wal-append.run_wall_time_us":1709159}]]}
W20260709 13:00:54.040277 20850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.030s	sys 0.024s
I20260709 13:00:54.007627 22207 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:54.007815 20085 rpcz_store.cc:276] 0709 13:00:52.809960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.816703 (+  6743us) service_pool.cc:224] Handling call
0709 13:00:52.817356 (+   653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:53.537160 (+719804us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b180007c7a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ed646 00007fa02e72758e 00007fa02e7a8f9e 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:00:53.602277 (+ 65117us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b58001370d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:00:53.663369 (+ 61092us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b5800890fd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:00:54.006921 (+343552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.007170 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1045409664}
I20260709 13:00:54.045665 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:53.967687 22744 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:00:53.983178 22281 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.023622 22962 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.977362 22589 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:54.048523 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:54.049146 20013 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:54.052670 20017 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:54.002677 22723 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:00:54.057830 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.060416 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:54.053399 20015 pending_rounds.cc:85] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:54.038333 19840 catalog_manager.cc:5697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.041898 22881 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.069271 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:54.069885 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 52601.9us
I20260709 13:00:54.075376 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.026773 22499 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.041565 22923 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.049744 22921 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:54.049532 22693 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.038524 22729 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.090379 22606 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:54.031014 22819 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.093243 22949 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.131253 22949 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.133340 22949 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:54.071246 22937 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:54.094431 22589 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.133888 22819 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.134722 22937 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:54.096243 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:54.097486 22499 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:54.039960 22835 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.137800 20017 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.137523 22881 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.064704 20015 pending_rounds.cc:107] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:00:54.102437 20009 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:54.146445 20015 raft_consensus.cc:2889] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:54.099992 22764 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.139513 22835 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:54.150116 22991 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:00:54.105119 22846 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.145069 19840 catalog_manager.cc:5697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 7, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 7 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.107014 22877 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:54.120780 22606 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:54.110720 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.122404 22762 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:54.111776 22378 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:00:54.125276 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:54.174079 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.174360 20850 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:54.128778 22958 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.078015 19982 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.149564 22764 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.062400 22629 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:54.155586 22931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.023s	sys 0.068s
I20260709 13:00:54.100572 22207 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:54.177030 20014 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:54.133340 22959 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:54.162531 20129 connection.cc:256] RPC call timeout handler was delayed by -1.1418296960000589s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2409s for a response
I20260709 13:00:54.177867 19840 catalog_manager.cc:5697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:00:54.186002 22744 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:54.156522 22877 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:54.171497 20013 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
W20260709 13:00:54.178596 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 100662us
I20260709 13:00:54.144596 22923 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:54.175293 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:54.178475 22606 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:54.142606 22921 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:54.133952 22600 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.117894 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.139791 22881 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.189759 22755 raft_consensus.cc:3055] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:54.190541 22931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.024s	sys 0.068s
W20260709 13:00:54.190752 22933 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:00:54.182425 22835 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:00:54.191802 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.246186 20850 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:00:54.246274 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 212978us
I20260709 13:00:54.173468 22846 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.155179 22967 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:00:54.193965 22747 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:54.194357 22986 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:00:54.157557 22693 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:54.138762 22729 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:54.194540 22764 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.205307 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:54.205585 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:54.115314 22914 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.208431 22715 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:54.186278 22836 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:54.237673 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.152881 22949 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:00:54.202266 22744 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.204160 22877 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.185393 22762 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.254869 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.254194 22971 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:54.257483 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.265172 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.253135 19840 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 6, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 6 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.214632 22755 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:54.275461 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:54.207984 22606 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:54.251022 22881 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.278752 20857 raft_consensus.cc:3060] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:54.204555 22923 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:54.287286 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:54.288601 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 83364.7us
I20260709 13:00:54.289705 20856 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:54.213907 22629 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:54.297663 20859 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:54.268604 22841 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.267899 22622 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.312688 19840 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 4 to 5, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.288070 22744 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:54.318387 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 243147us
I20260709 13:00:54.291302 22611 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:54.322714 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45564.1us
I20260709 13:00:54.321221 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.268162 22747 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.304911 22600 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:54.331288 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.333447 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:54.335351 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 129870us
I20260709 13:00:54.317230 22923 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.341387 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.313817 22629 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.343693 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.343014 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.344086 20855 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:54.321422 22281 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.348104 19981 leader_election.cc:400] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:54.351061 19981 leader_election.cc:403] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:54.346426 20085 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:00:54.306196 22971 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.357336 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:54.350327 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.360934 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.333809 22600 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.373888 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.374622 20165 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:54.332285 22378 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:54.362864 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.367491 20017 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:54.332049 22715 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:54.331305 22589 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.315455 22881 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:54.396112 22378 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:54.397505 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.397342 19840 catalog_manager.cc:5697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 4 to 7, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 7 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.333093 22207 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:54.401399 22988 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:00:54.401178 22820 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:54.402607 22994 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:00:54.373046 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:54.384860 20856 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:54.390193 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.388417 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.387419 20855 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:54.405108 20859 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:00:54.390735 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61810.6us
I20260709 13:00:54.328336 22841 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:54.392200 22990 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:00:54.394199 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.395505 23001 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:54.401777 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.413589 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.349684 22629 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.407410 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:54.430011 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72931.3us
I20260709 13:00:54.399633 19840 catalog_manager.cc:5697] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 3, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 3 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:54.383558 22600 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.438266 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.438848 20158 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:54.414053 20090 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:54.412642 20850 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:00:54.387473 22715 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.414256 20160 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:54.414430 20163 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:00:54.436460 20009 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:54.437098 20013 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:54.429957 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.349920 22622 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.446095 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.447527 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.445006 20087 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:54.450167 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 112162us
I20260709 13:00:54.446743 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.442222 19840 catalog_manager.cc:5697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 6, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
W20260709 13:00:54.457502 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 204839us
I20260709 13:00:54.453481 20165 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:54.461287 20160 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:54.456350 19982 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.463315 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.461674 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:00:54.467850 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.463990 20158 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:54.469832 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.339361 22921 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:54.470635 19840 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 8, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 8 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:54.450654 22611 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:54.473028 22611 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.473376 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.474807 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.475553 20855 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:54.476168 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.476509 22762 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.476617 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.477213 22971 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.477370 20856 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:54.476773 20854 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.447654 22715 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:54.484738 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.447445 22589 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:54.442551 22596 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 5. Current term is 6.
I20260709 13:00:54.440403 22998 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:54.510226 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.509872 19840 catalog_manager.cc:5697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:54.507867 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55774.2us
I20260709 13:00:54.495059 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:54.488260 20854 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:54.499004 20744 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:54.499887 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.491417 20082 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:00:54.514609 20850 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 2.
I20260709 13:00:54.512742 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:54.515482 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 108132us
I20260709 13:00:54.441614 22884 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:54.516670 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:54.519059 20010 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:00:54.407655 22606 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.523530 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.524176 20009 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 3.
I20260709 13:00:54.523842 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:54.518128 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:54.518631 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 115514us
I20260709 13:00:54.403507 22755 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:54.502679 22589 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.511101 22820 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:54.530529 20161 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:54.532315 19840 catalog_manager.cc:5697] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 8, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:00:54.517880 22207 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:54.519775 22846 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.540977 20008 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:54.543434 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:54.543471 22997 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:00:54.528549 22729 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.543596 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.541507 22931 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:00:54.548513 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 77396.1us
I20260709 13:00:54.530452 22881 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.531129 22589 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:54.549903 22622 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.529340 22937 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:54.539736 22207 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:54.561450 20857 raft_consensus.cc:2468] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:00:54.562356 19840 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:54.564817 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:54.548230 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.571056 22841 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:54.571241 22596 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:54.571898 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.633945 20850 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:54.572983 20852 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:00:54.575147 22962 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.002s
I20260709 13:00:54.576495 22378 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:54.576565 22747 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.579077 22606 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:54.540264 22998 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:54.530205 23004 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:54.582201 20082 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:00:54.582804 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 93504.4us
I20260709 13:00:54.589450 23001 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:54.579993 22729 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:54.590441 20016 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:54.592016 22207 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.597855 19840 catalog_manager.cc:5697] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:54.598515 22884 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:54.598927 20859 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:54.599260 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:54.602190 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:54.602845 22931 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:54.603834 22600 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.587039 22881 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:00:54.612205 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.612569 22923 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.613159 20012 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:54.614941 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.616473 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:54.622398 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:54.631296 20010 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:54.587183 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:54.638713 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 120638us
I20260709 13:00:54.661295 23001 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.656227 23004 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.652869 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.660223 19840 catalog_manager.cc:5697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:00:54.702178 19840 catalog_manager.cc:5697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 4 to 5, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:54.669785 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:54.672999 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 70876.8us
I20260709 13:00:54.677139 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.667439 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.705682 20851 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:54.674255 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.656610 22998 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:54.676043 22747 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:00:54.688589 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.690652 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.679069 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.703321 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:54.697029 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 109924us
I20260709 13:00:54.618866 22589 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.704205 20087 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:54.679041 22951 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.707688 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:54.730872 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.710289 20055 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:54.712376 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 113254us
I20260709 13:00:54.734055 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.719803 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.649847 22962 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:54.707666 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 184566us
I20260709 13:00:54.719663 20015 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:54.709251 20090 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:54.725602 19840 catalog_manager.cc:5697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 3 to 5, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 5 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.749686 20087 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:54.750190 20087 pending_rounds.cc:85] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 3
I20260709 13:00:54.750451 20087 pending_rounds.cc:107] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:54.726257 20857 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:54.709852 20085 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:54.750730 20087 raft_consensus.cc:2889] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:54.716187 20129 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:54.746537 20851 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:54.718438 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.724592 20163 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:54.749028 20130 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.744942 20016 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 5.
I20260709 13:00:54.752853 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.711249 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.709370 20089 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:54.753799 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:54.751742 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:54.760983 19840 catalog_manager.cc:5697] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:54.764906 20858 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:54.710638 22378 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:00:54.715448 22589 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.765357 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.708523 22747 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:54.791934 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 81703.2us
I20260709 13:00:54.792517 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.717669 22729 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:54.708089 22499 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:54.710332 23001 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.816039 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:54.717948 22841 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.780537 20161 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:54.824602 20158 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:54.824887 20082 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:00:54.825292 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.826647 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.790176 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:54.793053 20055 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:54.708151 22998 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:54.828636 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.794163 19840 catalog_manager.cc:5697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 6, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 6 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:54.834844 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:54.831451 19982 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:54.748637 22959 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:54.707998 22931 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:54.758121 22881 raft_consensus.cc:2804] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:00:54.710266 23004 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:54.753044 22819 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:54.801868 22958 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:54.812289 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:54.812901 22923 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:54.813088 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.813457 22715 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:54.815186 22755 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.802095 22499 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:54.707942 23009 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:00:54.821620 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:54.822649 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.885383 20857 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 2.
I20260709 13:00:54.823807 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.719565 22877 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:54.828665 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.835794 19840 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 9, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 9 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:00:54.828890 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:54.842442 20162 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:54.843333 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:54.834818 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:54.889789 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55041.5us
I20260709 13:00:54.847095 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.848278 20160 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:54.850332 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:54.850858 20165 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:54.801105 22747 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:54.778759 22951 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:00:54.859028 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:54.857347 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 206441us
I20260709 13:00:54.863636 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.864887 20008 raft_consensus.cc:2368] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
I20260709 13:00:54.836488 22378 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:00:54.877559 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.790199 22949 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:00:54.858739 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:54.840936 22589 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:54.840236 22729 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:54.850143 22762 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.042917 22762 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.043458 22762 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.851910 22929 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:54.845862 22884 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:54.799329 22693 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:00:54.845798 22931 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.898754 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:54.890605 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:55.054181 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 163674us
I20260709 13:00:54.899722 20851 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:00:54.923635 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 102108us
I20260709 13:00:54.859619 22958 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.824642 22841 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:54.928215 20089 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:55.092707 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:54.979746 22596 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.932224 20011 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:54.929395 20087 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:54.866078 22881 raft_consensus.cc:697] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:54.933763 20012 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:54.959146 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 125726us
W20260709 13:00:54.959419 22744 consensus_meta.cc:287] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.005s
I20260709 13:00:54.903954 22959 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:00:54.972064 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 220368us
I20260709 13:00:55.014640 20083 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:55.012110 23031 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:55.121062 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.019627 22729 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:55.015301 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:55.006760 22962 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:55.005492 22499 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:55.006117 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:55.129926 20857 raft_consensus.cc:2393] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:55.007359 22600 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:55.004606 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:54.970999 22629 consensus_meta.cc:287] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.004s
I20260709 13:00:55.024252 20091 raft_consensus.cc:2410] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:00:54.926187 22967 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:55.040473 22836 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:55.093048 20016 raft_consensus.cc:2393] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:54.928702 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 175346us
I20260709 13:00:54.955106 22207 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:54.991739 22589 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:54.922848 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 88083.3us
I20260709 13:00:55.177474 20055 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:55.005697 23017 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:55.047055 22929 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.059057 22958 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.125263 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:55.125125 20744 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:55.130843 20086 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:55.061755 22931 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.051159 22884 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.165280 20088 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 3.
I20260709 13:00:55.183308 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:55.168016 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.167727 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.172523 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:55.187536 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 62522.8us
I20260709 13:00:55.168810 23005 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:55.121393 23031 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:55.185650 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 60435.3us
I20260709 13:00:55.103104 22819 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:55.129429 22729 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.190651 20744 leader_election.cc:400] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
I20260709 13:00:55.193243 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.167495 22629 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:55.212190 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.213356 20085 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:55.195245 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:55.198797 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.179960 22589 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:55.199944 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.121773 23020 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:00:55.124423 22744 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:55.121102 22959 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.184881 22696 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:55.225821 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.226797 20089 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:55.206913 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:55.180254 23017 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.182693 22929 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:55.206486 20744 leader_election.cc:403] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:00:55.193980 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 187585us
I20260709 13:00:55.211246 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.211472 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.167629 22967 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:55.206688 20086 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:55.185264 22499 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:55.242602 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:55.193403 22951 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:55.190467 22914 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:55.193356 22962 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:55.193634 22890 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:00:55.240892 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 68483.3us
W20260709 13:00:55.244765 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.187424 22884 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.244143 20083 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:00:55.178316 22836 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:55.256848 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:55.200182 23031 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.277400 20017 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:55.254818 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.300280 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:55.278034 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:55.180385 22378 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:55.232707 22959 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:55.232342 23005 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:55.255390 22715 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:55.204162 22971 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.071s
W20260709 13:00:55.324437 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 81919.5us
I20260709 13:00:55.255235 22764 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:55.276639 22914 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:00:55.355234 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 164725us
I20260709 13:00:55.340310 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.233019 23017 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.246032 23009 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.274765 22951 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.254657 22693 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:55.356855 22715 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:55.345155 23015 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
W20260709 13:00:55.360569 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 153728us
I20260709 13:00:55.333580 22877 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:55.345574 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:55.372218 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.341872 23005 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.296984 22622 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.338651 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:55.296742 22890 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:55.363445 20130 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:55.352159 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 168883us
I20260709 13:00:55.242436 22696 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:55.353609 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:55.351152 22764 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.394881 20086 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 5: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:55.358239 22914 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:55.362128 22951 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.397863 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:55.397200 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.537s (SENT) [suppressed 13 similar messages]
I20260709 13:00:55.370755 22715 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:55.370509 22693 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:55.493060 22600 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:55.491847 23027 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:55.498045 22693 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:55.504165 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 133397us
I20260709 13:00:55.408485 22622 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:55.507802 23025 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:55.508754 23025 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:55.510078 22622 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:55.519863 22715 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.524080 23044 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 13:00:55.524896 22747 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.001s
I20260709 13:00:55.395056 23005 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:55.359961 23040 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:55.446992 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 146812us
I20260709 13:00:55.393688 22890 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:55.410844 23013 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.459s	user 0.002s	sys 0.000s
I20260709 13:00:55.345170 23036 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:00:55.408358 22764 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:55.407248 22629 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:55.444797 23021 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:00:55.407254 22744 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:55.487283 23050 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.488098 23047 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:55.489365 22207 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:55.418917 22877 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:55.246973 22881 consensus_queue.cc:237] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:55.348881 22819 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:55.401571 23052 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
I20260709 13:00:55.450062 22841 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:55.525715 22747 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:55.511660 23025 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:55.615221 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:55.615139 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 204912us
I20260709 13:00:55.694199 20130 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:55.690490 20859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.002s	sys 0.009s
W20260709 13:00:55.698590 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:55.698863 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 83739.5us
W20260709 13:00:55.689841 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 184075us
W20260709 13:00:55.698815 20859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.002s	sys 0.009s
I20260709 13:00:55.669405 23050 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:55.704475 23050 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:55.705921 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:55.659261 23055 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:55.754518 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 359405us
W20260709 13:00:55.689451 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 315385us
I20260709 13:00:55.736418 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:55.686591 23034 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:55.748816 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 351606us
I20260709 13:00:55.713340 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:55.790616 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1077) took 1460 ms. Trace:
W20260709 13:00:55.815202 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 101957us
I20260709 13:00:55.816416 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:55.794754 23026 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.430s	user 0.002s	sys 0.000s
I20260709 13:00:55.679898 23047 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:55.814344 20164 rpcz_store.cc:276] 0709 13:00:54.330349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.331281 (+   932us) service_pool.cc:224] Handling call
0709 13:00:54.331583 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.430850 (+ 99267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.430883 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.488265 (+ 57382us) log.cc:844] Serialized 26 byte log entry
0709 13:00:54.633231 (+144966us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:54.633285 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.783221 (+149936us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b54001441d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:54.783283 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.759483 (+976200us) raft_consensus.cc:1685] finished
0709 13:00:55.759511 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.790536 (+ 31025us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26287,"raft.queue_time_us":27511,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":8592,"spinlock_wait_cycles":150628608,"thread_start_us":110557,"threads_started":1,"wal-append.queue_time_us":322637}
I20260709 13:00:55.676045 23041 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:55.681074 22929 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:55.846006 20017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 889) took 1440 ms (client timeout 1774 ms). Trace:
I20260709 13:00:55.710350 23025 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.738653 22622 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:55.711774 22971 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:00:55.808359 23004 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.942s	user 0.012s	sys 0.000s
W20260709 13:00:55.836480 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 482942us
I20260709 13:00:55.676831 22744 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:55.829339 20016 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:55.800429 23018 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.818396 23034 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:55.841024 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 1232) took 1275 ms. Trace:
I20260709 13:00:55.837277 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 893) took 1380 ms. Trace:
I20260709 13:00:55.972810 23052 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.980787 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:55.822213 23053 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:55.828370 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1243) took 1003 ms. Trace:
I20260709 13:00:55.671548 22877 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:56.079655 20011 rpcz_store.cc:276] 0709 13:00:54.456812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.525209 (+ 68397us) service_pool.cc:224] Handling call
0709 13:00:55.836984 (+1311775us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54009155d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:55.837191 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723529600}
I20260709 13:00:55.765426 22836 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:00:55.844694 23045 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 13:00:56.009390 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:55.934723 23037 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.526s	user 0.001s	sys 0.000s
W20260709 13:00:55.833421 22755 consensus_meta.cc:287] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.957s	user 0.009s	sys 0.002s
W20260709 13:00:55.834393 23001 consensus_meta.cc:287] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.005s	user 0.007s	sys 0.008s
W20260709 13:00:55.823490 22762 consensus_meta.cc:287] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.779s	user 0.012s	sys 0.000s
I20260709 13:00:56.018414 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:55.821286 22499 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:56.044691 20859 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:55.687543 22729 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:55.838119 22747 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:55.826965 22937 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:00:56.010766 22998 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 1.154s	user 0.015s	sys 0.000s
I20260709 13:00:56.055045 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:56.046288 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:56.076665 20091 rpcz_store.cc:276] 0709 13:00:54.565749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.566732 (+   983us) service_pool.cc:224] Handling call
0709 13:00:55.840477 (+1273745us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54009b9454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:55.840870 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748317952}
I20260709 13:00:56.048446 20082 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:00:55.944446 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 133439us
I20260709 13:00:55.932466 23062 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:55.996762 20017 rpcz_store.cc:269] 0709 13:00:54.405516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.405672 (+   156us) service_pool.cc:224] Handling call
0709 13:00:55.845715 (+1440043us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b540064e6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:55.845914 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927510272}
I20260709 13:00:55.744463 22841 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.264s
I20260709 13:00:56.002493 23025 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:56.070498 22589 consensus_meta.cc:287] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.815s	user 0.010s	sys 0.001s
I20260709 13:00:56.104475 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.104902 20083 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:00:56.081769 22622 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:56.055393 23067 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:00:56.096871 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1244) took 1271 ms. Trace:
I20260709 13:00:56.105883 20086 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:56.157092 20083 pending_rounds.cc:85] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 4
W20260709 13:00:56.062789 23043 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
W20260709 13:00:56.060758 23039 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 13:00:56.060860 23011 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.016s	user 0.008s	sys 0.000s
W20260709 13:00:56.094288 22715 consensus_meta.cc:287] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.002s
I20260709 13:00:56.046937 23004 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.113121 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:55.836496 23014 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.340s
W20260709 13:00:56.114315 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 377988us
W20260709 13:00:56.055861 22958 consensus_meta.cc:287] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.863s	user 0.008s	sys 0.003s
W20260709 13:00:56.088102 22959 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.734s	user 0.005s	sys 0.005s
I20260709 13:00:56.100526 20090 rpcz_store.cc:276] 0709 13:00:54.825131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.829797 (+  4666us) service_pool.cc:224] Handling call
0709 13:00:54.830088 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.830301 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.830332 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.856386 (+ 26054us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.392319 (+535933us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:55.490928 (+ 98609us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.491335 (+   407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.491768 (+   433us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.828007 (+336239us) raft_consensus.cc:1685] finished
0709 13:00:55.828040 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.828278 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":359911,"prepare.run_cpu_time_us":3552,"prepare.run_wall_time_us":11525,"raft.queue_time_us":154141,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":10364,"thread_start_us":512554,"threads_started":2,"wal-append.queue_time_us":919767}
I20260709 13:00:56.076081 23034 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.070048 23027 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.143102 20853 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
W20260709 13:00:56.109213 22914 consensus_meta.cc:287] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.000s
I20260709 13:00:56.093240 22929 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:55.817308 23055 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:56.137971 20745 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.013s	sys 0.020s
W20260709 13:00:56.243440 20745 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.014s	sys 0.020s
I20260709 13:00:56.243428 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.241968 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:56.073086 23060 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:56.162343 20083 pending_rounds.cc:107] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:00:55.636492 23040 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.148016 22762 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.091121 22890 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.160187 22998 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:56.155751 22931 consensus_meta.cc:287] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.959s	user 0.013s	sys 0.000s
I20260709 13:00:56.044760 23063 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:56.011504 23041 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.161108 20084 rpcz_store.cc:276] 0709 13:00:54.825432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.829007 (+  3575us) service_pool.cc:224] Handling call
0709 13:00:54.851581 (+ 22574us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:55.119240 (+267659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.119300 (+    60us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:55.164484 (+ 45184us) log.cc:844] Serialized 52 byte log entry
0709 13:00:55.362067 (+197583us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:55.362105 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.362173 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.096613 (+734440us) raft_consensus.cc:1685] finished
0709 13:00:56.096645 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.096808 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":159748,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":367735,"threads_started":2,"wal-append.queue_time_us":862704}
I20260709 13:00:56.267788 23040 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.158636 23025 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.163822 23062 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.115854 22744 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.081643 23018 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.093070 23052 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:56.173790 22962 consensus_meta.cc:287] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.864s	user 0.012s	sys 0.001s
I20260709 13:00:56.278605 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 926) took 1554 ms. Trace:
I20260709 13:00:56.211203 22715 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:56.228212 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.229729 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 902) took 1323 ms. Trace:
I20260709 13:00:56.171999 23058 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.223212 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:56.237320 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.238335 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.156985 20087 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.295082 20085 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:56.140620 23001 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.248611 20083 raft_consensus.cc:2889] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:56.211539 22629 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.160236 22589 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.218035 22937 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.220531 23044 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.312436 22937 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.314311 22729 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.282404 23062 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.223680 22281 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.316665 22281 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.170351 22622 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.317705 23052 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.282998 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 922) took 1560 ms. Trace:
I20260709 13:00:56.319725 23041 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.283784 22755 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:56.320441 20850 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.320899 20013 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:56.321663 22629 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.322242 22715 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.322875 22715 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:56.276081 23076 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:00:56.283594 22931 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.283408 22929 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:56.281553 22998 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.286223 23063 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.324788 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 896) took 2176 ms. Trace:
I20260709 13:00:56.277212 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.285133 20165 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:56.210887 23047 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.218283 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:56.326998 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.329144 20089 raft_consensus.cc:2368] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:56.294112 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 925) took 1570 ms. Trace:
W20260709 13:00:56.279661 22884 consensus_meta.cc:287] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.943s	user 0.012s	sys 0.002s
I20260709 13:00:56.230806 22958 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:56.267791 22762 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:56.281878 22764 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.622s	user 0.005s	sys 0.006s
I20260709 13:00:56.237825 22959 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:56.282711 23005 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.000s
I20260709 13:00:56.236989 22914 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.357020 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.294698 20090 raft_consensus.cc:2376] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10. Re-sending same reply.
W20260709 13:00:56.306470 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64602.4us
W20260709 13:00:56.324481 23009 consensus_meta.cc:287] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.951s	user 0.007s	sys 0.004s
I20260709 13:00:56.286981 20852 rpcz_store.cc:276] 0709 13:00:54.730270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.735950 (+  5680us) service_pool.cc:224] Handling call
0709 13:00:54.759761 (+ 23811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.793371 (+ 33610us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:54.904088 (+110717us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.192336 (+288248us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.231701 (+ 39365us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.246242 (+ 14541us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:55.246567 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.689603 (+443036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.689692 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.278371 (+588679us) raft_consensus.cc:1685] finished
0709 13:00:56.278393 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.278548 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161758,"prepare.queue_time_us":218840,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":106,"raft.queue_time_us":305946,"raft.run_cpu_time_us":602,"raft.run_wall_time_us":45121,"thread_start_us":694657,"threads_started":3,"wal-append.queue_time_us":767272}
I20260709 13:00:56.312651 23044 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.172927 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:56.359897 20859 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:56.361662 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.362407 20859 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.290410 20858 rpcz_store.cc:276] 0709 13:00:54.906281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.932608 (+ 26327us) service_pool.cc:224] Handling call
0709 13:00:54.981400 (+ 48792us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.042906 (+ 61506us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:55.042944 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.043592 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.210400 (+166808us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:55.210497 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.210677 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.229441 (+1018764us) raft_consensus.cc:1685] finished
0709 13:00:56.229471 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.229652 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57535,"raft.queue_time_us":58951,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":151343,"threads_started":1,"wal-append.queue_time_us":329834}
I20260709 13:00:56.364691 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:56.365476 20858 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:56.366778 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.333544 20853 rpcz_store.cc:276] 0709 13:00:54.148495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.319472 (+170977us) service_pool.cc:224] Handling call
0709 13:00:56.324584 (+2005112us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540000a0d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:56.324693 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4138362112}
I20260709 13:00:56.332234 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:56.342584 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 124393us
I20260709 13:00:56.237144 23027 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.378219 23027 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:56.335600 20855 rpcz_store.cc:276] 0709 13:00:54.723679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.733508 (+  9829us) service_pool.cc:224] Handling call
0709 13:00:54.764781 (+ 31273us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:54.837185 (+ 72404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.837230 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:54.851103 (+ 13873us) log.cc:844] Serialized 52 byte log entry
0709 13:00:55.183493 (+332390us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:55.183580 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.183650 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.293821 (+1110171us) raft_consensus.cc:1685] finished
0709 13:00:56.293853 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.294033 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30348,"raft.queue_time_us":70064,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":15012,"thread_start_us":250842,"threads_started":1,"wal-append.queue_time_us":466111}
I20260709 13:00:56.346807 20850 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:56.293489 22962 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:56.320081 20856 rpcz_store.cc:276] 0709 13:00:54.722675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.732010 (+  9335us) service_pool.cc:224] Handling call
0709 13:00:54.754888 (+ 22878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.802022 (+ 47134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.802071 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.863366 (+ 61295us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.181278 (+317912us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:55.181334 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.181428 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.282771 (+1101343us) raft_consensus.cc:1685] finished
0709 13:00:56.282794 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.282932 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46388,"raft.queue_time_us":86884,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8971,"thread_start_us":188568,"threads_started":1,"wal-append.queue_time_us":755830}
I20260709 13:00:56.381951 20856 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:56.353718 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.385147 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.386253 20083 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:56.315701 22971 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:00:56.326344 23022 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.224s	user 0.003s	sys 0.006s
I20260709 13:00:56.389962 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 903) took 1776 ms. Trace:
I20260709 13:00:56.367765 20858 raft_consensus.cc:2368] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:56.390456 20009 rpcz_store.cc:276] 0709 13:00:54.613361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.691126 (+ 77765us) service_pool.cc:224] Handling call
0709 13:00:54.691454 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.716524 (+ 25070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.716577 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.717142 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:00:54.884147 (+167005us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:54.884201 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:54.884276 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.325952 (+1441676us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b240043bf08. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b54b9c4 00007fa02b549acb 00007fa02b704dc9 00007fa02b704a55 00007fa02b70409d 00007fa02e33610e 00007fa02e34d88c 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221
0709 13:00:56.389575 (+ 63623us) raft_consensus.cc:1685] finished
0709 13:00:56.389604 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.389879 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2989,"raft.queue_time_us":6732,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":26211712,"thread_start_us":146147,"threads_started":1,"wal-append.queue_time_us":262814}
I20260709 13:00:56.393234 20164 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
W20260709 13:00:56.346479 23017 consensus_meta.cc:287] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.970s	user 0.007s	sys 0.004s
I20260709 13:00:56.324872 23076 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:56.323908 22629 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.409515 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.332309 22589 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.328864 22931 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.330448 22890 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.330652 23078 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.308738 23053 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.136s
I20260709 13:00:56.401850 20090 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:56.449846 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.451319 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.322253 23073 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.453470 23073 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.330865 23004 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.319748 22622 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.410771 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:56.457136 22890 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:00:56.457235 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46561.2us
I20260709 13:00:56.458165 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:56.405546 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:56.459184 20158 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.459563 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.412766 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.459764 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:56.460717 20161 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:56.461516 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:56.461983 20161 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.401364 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:56.416122 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.462512 20162 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.417047 23001 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.305104 23058 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.417490 22959 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:56.418087 20084 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:56.408582 23017 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.417912 22755 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 13:00:56.418761 23031 consensus_meta.cc:287] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 1.060s	user 0.006s	sys 0.006s
I20260709 13:00:56.356576 22764 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:56.424295 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:56.426497 20859 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:56.352566 22884 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.329583 22929 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.268472 22877 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.351464 23057 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.332716 23063 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.329144 23055 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.409324 23076 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:56.404634 23009 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:56.435063 22914 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.401199 23005 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:56.437038 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.438031 22937 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.439386 23088 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.441000 22378 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:56.329262 23018 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.325304 22744 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.465617 20744 leader_election.cc:400] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
W20260709 13:00:56.466910 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 165033us
I20260709 13:00:56.467432 20090 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:56.467586 22207 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:00:56.458985 22951 consensus_meta.cc:287] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 1.000s	user 0.011s	sys 0.000s
W20260709 13:00:56.469521 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54976.7us
I20260709 13:00:56.478585 20087 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
W20260709 13:00:56.579945 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 114529us
I20260709 13:00:56.580189 22207 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:56.580971 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 189689us
I20260709 13:00:56.581723 20159 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:56.466254 22890 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:00:56.583668 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.467955 22747 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:56.586943 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.597864 20083 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:00:56.585510 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:56.599090 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.599665 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:56.604069 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.599880 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.645344 20162 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:56.606842 20856 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
I20260709 13:00:56.646347 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.646544 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.647251 20090 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:00:56.648769 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49207.4us
I20260709 13:00:56.649406 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.648788 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.651746 20091 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
W20260709 13:00:56.585759 23101 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:00:56.577364 22693 raft_consensus.cc:2764] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:00:56.655673 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.575524 23031 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.656764 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.656810 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.649462 22959 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.658123 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.659543 20083 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:56.660791 20055 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:56.608140 22967 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.662323 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.663875 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.664901 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:56.665169 20009 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:00:56.651955 22755 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:56.650429 23048 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.665234 20087 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:56.660772 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:56.667812 20854 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:56.669039 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.669687 20017 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:56.671099 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.465056 23078 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.672999 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.673616 20011 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:56.670661 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:56.585773 22951 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.677639 20083 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:56.680287 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.679535 20744 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:56.679636 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.596596 23057 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.675412 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:56.683636 20088 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:00:56.668682 23093 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:00:56.585400 22378 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.682430 23055 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.736619 23073 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:56.737017 23030 raft_consensus.cc:3055] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:56.737461 23030 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:56.738508 23094 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:56.738420 23030 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.737141 23055 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.739039 23094 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:56.676193 22762 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:00:56.745085 22835 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.745790 22835 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:56.749570 22764 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:56.751534 23025 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.752125 20161 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:56.753206 23044 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.673866 23048 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.753331 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.754097 20161 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:56.755039 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.658855 23052 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:00:56.655193 23076 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.666783 22959 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.676275 22629 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:56.814195 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:56.849571 20859 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:56.599004 23088 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.787298 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:56.807859 22207 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.806748 20085 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:00:56.787632 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.682487 22729 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:56.817713 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:56.810403 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.803714 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:56.863121 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45493.7us
I20260709 13:00:56.863408 20009 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:56.831211 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:56.803246 20855 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:56.804838 20011 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:56.841233 20158 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:56.785185 20163 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:56.797317 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:56.842496 20852 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:56.806646 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.831513 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:56.850411 20008 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:56.866186 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.676637 22958 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:56.804132 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.867612 20016 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.868481 20160 raft_consensus.cc:2393] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:56.869819 20156 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:56.869871 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:56.851624 20090 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:56.755748 20161 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:56.793798 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1250) took 1966 ms. Trace:
I20260709 13:00:56.854797 20056 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:56.871152 20082 rpcz_store.cc:276] 0709 13:00:54.827621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.951123 (+123502us) service_pool.cc:224] Handling call
0709 13:00:55.016653 (+ 65530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.156965 (+1140312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.793532 (+636567us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400851454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:56.793721 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2372132224}
I20260709 13:00:56.855443 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:56.873126 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:56.873332 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:56.873842 20161 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.740614 23001 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:56.783321 22378 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:56.758031 23034 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.001s
I20260709 13:00:56.861105 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:56.769697 23087 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.325s	user 0.003s	sys 0.000s
I20260709 13:00:56.755609 23048 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.887570 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.790030 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:56.860603 20010 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:56.771781 22951 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.793123 23005 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.768261 22929 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.875284 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.902846 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.903600 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.904083 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.904381 20014 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:56.905548 20014 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:56.905611 23094 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:56.906615 20161 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:00:56.763962 23031 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.849084 22629 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.847890 23073 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.907717 22629 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:56.908190 23073 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.910369 23073 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.777745 23057 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.882863 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.780534 23055 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.910693 23017 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.912292 23055 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:56.912726 23055 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:56.913146 23055 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.913789 20016 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:56.914944 22884 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.759582 23078 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.917443 22959 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:56.918216 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.918865 20009 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:56.878990 22606 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.920027 23079 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.920590 23041 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:56.920534 23079 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.921020 23041 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:56.921833 23041 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.922819 23079 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.923058 23041 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.878890 22715 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.924232 22715 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.926199 22715 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.879614 22600 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:56.926919 22281 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:56.927047 22600 raft_consensus.cc:697] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:56.927553 22281 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:56.928020 22600 consensus_queue.cc:237] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:56.929469 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.930107 20159 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:56.930935 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.879837 22729 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:56.931988 22729 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:56.762637 23058 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.867867 23018 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:56.940702 23018 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:56.942894 23018 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.944365 22835 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:56.946107 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.877122 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.875108 22967 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.858232 23061 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:00:56.900215 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.901019 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.875142 22755 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:56.906064 20157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1083) took 2122 ms (client timeout 2518 ms). Trace:
I20260709 13:00:56.906690 22834 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:56.959348 20157 rpcz_store.cc:269] 0709 13:00:54.784121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.793179 (+  9058us) service_pool.cc:224] Handling call
0709 13:00:56.905881 (+2112702us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54000be854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:56.906003 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3265264896}
I20260709 13:00:56.882558 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.961942 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.962568 20009 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:00:56.947446 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.907689 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.877192 22890 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.968125 22890 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.968539 22890 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.888476 23034 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.878875 23001 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:56.972136 23001 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:56.972309 23034 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:56.973031 20851 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:56.886425 22914 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:00:56.945590 22378 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:56.975273 22378 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:56.975641 22378 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.980333 23088 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:56.984717 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.985546 20016 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:56.986354 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.987350 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.988346 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.988953 20015 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:56.991350 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.991441 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:56.991900 20008 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.997313 22606 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.997807 22606 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.002411 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.003145 20165 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:56.987847 20016 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.004042 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.005143 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.006280 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.007346 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:56.960487 22755 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.961102 22967 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:56.876461 23062 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:00:56.955199 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:56.960835 20011 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:56.961628 22834 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:56.994238 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:57.010262 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.060134 20161 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
W20260709 13:00:56.894026 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 104056us
I20260709 13:00:57.061339 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.061932 20161 raft_consensus.cc:3060] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.062125 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.011526 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.063201 20159 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:57.064209 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.011650 20014 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:57.016705 20010 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:57.023571 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.012022 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.068898 20852 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:00:57.007898 20165 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:00:57.023784 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:57.064791 20159 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 4.
I20260709 13:00:57.027160 20089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1083) took 1822 ms. Trace:
I20260709 13:00:57.022984 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.038177 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.072762 20857 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:00:57.041546 20853 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 13:00:57.014420 20091 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:57.022338 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:57.020176 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 65705.3us
I20260709 13:00:57.036751 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.040686 20009 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.044834 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.080914 20163 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:57.046597 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:57.082662 20854 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:57.059285 20008 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:57.038573 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.085129 20157 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.045222 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 934) took 1064 ms. Trace:
I20260709 13:00:57.069417 20015 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:57.071123 20089 rpcz_store.cc:276] 0709 13:00:55.205041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.232195 (+ 27154us) service_pool.cc:224] Handling call
0709 13:00:57.026940 (+1794745us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54005a2154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:57.027095 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2314262400}
I20260709 13:00:57.029225 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.077785 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.086079 20851 rpcz_store.cc:276] 0709 13:00:55.980375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.980495 (+   120us) service_pool.cc:224] Handling call
0709 13:00:57.045020 (+1064525us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54000096d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:57.045150 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052511104}
I20260709 13:00:57.028379 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.095297 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.114364 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.094190 20160 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:57.008503 22755 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.035166 22696 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:57.123379 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.161881 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.129683 20010 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:00:57.167816 20161 raft_consensus.cc:2468] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:57.039976 23030 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.163794 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.026021 23076 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.021430 23062 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.170993 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:57.235800 20089 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 4: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:57.171185 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:57.171378 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44144.3us
I20260709 13:00:57.021721 23061 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:57.011991 22967 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:57.176332 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.182607 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.183080 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.261327 20082 raft_consensus.cc:2393] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:57.185710 20156 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.119297 22835 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:57.186264 20088 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.120074 23088 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:57.131114 23035 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:57.162045 22959 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:57.194410 20164 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.272902 20164 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:00:57.194613 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.037293 22629 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.193912 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.026708 23001 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.198181 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.277849 20015 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:00:57.019510 23036 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.165s
I20260709 13:00:57.055050 22890 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.204649 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:57.205637 20016 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:57.275509 23035 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:57.209313 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45605.6us
I20260709 13:00:57.018260 22834 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.191507 23058 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.226769 20009 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
W20260709 13:00:57.265380 22762 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.032s	sys 0.046s
I20260709 13:00:57.232756 20161 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:57.130286 22949 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:00:57.251151 20854 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 4.
I20260709 13:00:57.255330 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.296725 20008 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:57.259658 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.018633 23025 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:57.258188 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:57.261209 20014 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
W20260709 13:00:57.268199 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 394908us
I20260709 13:00:57.233935 23042 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:57.235144 23041 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.300307 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:57.292338 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 173423us
I20260709 13:00:57.288240 23058 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.309556 23001 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:57.313162 22629 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:57.313798 22606 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.314172 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.303211 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.271278 22959 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.293867 20156 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:57.294307 22762 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.032s	sys 0.046s
I20260709 13:00:57.316326 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.317075 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:57.318156 20856 raft_consensus.cc:2368] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:57.287879 22834 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.319253 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.331176 20015 raft_consensus.cc:2393] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:00:57.332120 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.332003 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:57.333065 20016 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:57.315735 22959 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:57.294629 23073 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:57.327037 20014 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:57.329236 20157 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
W20260709 13:00:57.304826 20055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.043s	sys 0.058s
I20260709 13:00:57.269390 23061 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.345568 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.332978 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:57.349062 22207 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:57.346378 20011 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:57.325812 22967 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:00:57.356067 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.357089 20008 raft_consensus.cc:2393] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:57.358950 20010 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:57.359156 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.359301 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.359794 20157 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:57.360414 20156 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:00:57.327226 22884 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:00:57.321015 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.370601 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.371137 22890 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:57.371734 20165 raft_consensus.cc:2393] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:57.336294 22606 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:57.334362 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.317833 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:57.327337 20159 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:00:57.255944 23062 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:57.338475 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50595.2us
I20260709 13:00:57.287426 23035 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:57.299566 22835 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:57.339212 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:57.342579 20055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.043s	sys 0.058s
W20260709 13:00:57.390465 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 715263us
I20260709 13:00:57.336647 23025 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:57.324086 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:57.381399 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 63645.7us
I20260709 13:00:57.380061 20159 pending_rounds.cc:85] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 6
I20260709 13:00:57.400882 20159 pending_rounds.cc:107] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:00:57.401258 20159 raft_consensus.cc:2889] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:57.383016 20163 raft_consensus.cc:2393] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:57.328293 23058 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:57.385769 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.330430 23079 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:00:57.327025 23041 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.407321 20055 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.356401 22967 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:57.409286 20858 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.364723 23048 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:57.366756 22884 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.400043 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:57.403355 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64223.4us
I20260709 13:00:57.397750 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.395275 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:57.410645 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.362107 23057 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:00:57.385152 23018 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:57.413810 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:57.365316 23066 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:00:57.416977 20161 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:00:57.392881 23073 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:00:57.423221 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:57.406620 22696 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:00:57.420855 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.412145 22967 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:57.432963 22715 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:57.447124 22890 consensus_queue.cc:1059] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:57.475620 22890 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:57.476182 22890 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:57.477031 22890 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.478188 22890 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.420399 22762 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:57.454083 23061 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:57.456650 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107522us
I20260709 13:00:57.481890 20130 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.482726 23088 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:57.483258 23057 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:57.461117 23073 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:57.483892 22693 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:57.484352 22693 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:57.484742 23073 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:57.485457 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.459688 22835 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.441783 22949 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:57.429194 22877 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:57.486503 22877 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:57.487342 22877 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.488395 22877 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.488842 20744 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:57.489753 23061 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.490121 23061 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.490445 23061 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.464496 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 985) took 1020 ms. Trace:
I20260709 13:00:57.407897 22951 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:00:57.411530 23048 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:57.492838 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52566.6us
I20260709 13:00:57.415557 23055 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.468258 20163 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:57.473196 23041 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:57.473839 22747 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:57.424337 23066 consensus_queue.cc:1243] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:00:57.415884 22884 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:00:57.418496 23018 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.440665 22696 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.483551 23088 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:57.465270 23062 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:57.491364 20162 rpcz_store.cc:276] 0709 13:00:56.444572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.655649 (+211077us) service_pool.cc:224] Handling call
0709 13:00:56.655977 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.657887 (+  1910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:56.657917 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:56.667547 (+  9630us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.075653 (+408106us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:57.075716 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.075807 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.464130 (+388323us) raft_consensus.cc:1685] finished
0709 13:00:57.464164 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.464352 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5496,"thread_start_us":296196,"threads_started":1,"wal-append.queue_time_us":619004}
I20260709 13:00:57.493150 20130 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.503859 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.504995 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.530400 20855 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
W20260709 13:00:57.507042 20743 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.000s	sys 0.018s
I20260709 13:00:57.508037 20157 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:57.520067 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:57.531987 20858 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.524921 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:57.522508 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.535027 20856 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:57.532570 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:57.537603 20743 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.003s	sys 0.019s
I20260709 13:00:57.538589 20857 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 4: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:57.506939 23079 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:57.536801 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:57.495159 22715 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:57.523741 22747 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:57.524729 22931 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.016s	sys 0.035s
I20260709 13:00:57.544260 20853 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
W20260709 13:00:57.538587 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 668740us
W20260709 13:00:57.541332 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 118192us
I20260709 13:00:57.548717 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.527341 23018 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.524217 22696 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:00:57.444430 22958 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:00:57.548424 20743 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:57.506610 23047 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:00:57.555053 22931 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.017s	sys 0.035s
I20260709 13:00:57.555956 22600 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:57.583422 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:57.543881 23062 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:57.582335 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:57.606213 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:57.543527 22949 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:57.547016 23058 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:00:57.611943 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63343.2us
I20260709 13:00:57.482700 22755 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.557231 22747 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:57.623066 20855 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:57.635308 20858 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:57.629453 23047 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:57.636785 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:57.485373 22693 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.630414 23104 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:57.637089 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:57.611128 22378 consensus_meta.cc:287] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.000s
I20260709 13:00:57.607569 23041 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:00:57.627743 23103 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:00:57.544287 23108 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:00:57.623122 22600 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:57.604363 23079 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:57.611472 23001 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:57.637908 23104 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:57.647039 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:57.633625 23058 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.634995 23094 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:57.657162 20855 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:00:57.661269 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:57.647375 23053 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.610885 22959 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:57.672627 20014 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:57.648054 23103 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:57.557024 20163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:00:57.675720 20014 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:57.676685 20015 pending_rounds.cc:85] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:57.680307 20015 pending_rounds.cc:107] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:00:57.680747 20015 raft_consensus.cc:2889] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:57.654587 23066 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:57.658466 22600 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.683905 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:57.687489 20012 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.668103 23030 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:57.662611 23058 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.679554 20014 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:57.692904 20014 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.686259 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.644558 22378 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.693814 20091 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:57.695569 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:57.677037 22729 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:57.705678 23055 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:57.680152 22606 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:57.706773 22606 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.709223 22606 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:57.674822 23042 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:57.710668 23042 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.667351 23001 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.712033 23115 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:57.712750 23115 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:57.712869 23042 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:57.713866 23001 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:57.713763 23115 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.716218 23047 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:57.695216 23105 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:57.717679 23105 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.718443 22967 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.648968 23088 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.720544 22967 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:57.721166 22755 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:57.695266 22949 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:57.723379 23061 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:57.723809 20017 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:57.658782 22715 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.726313 20859 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:57.697481 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.733593 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:57.738345 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.725554 23061 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:57.685922 22744 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.683517 23066 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.749748 20158 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:57.751044 20163 raft_consensus.cc:606] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.751407 20163 raft_consensus.cc:614] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:00:57.752100 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 54709.5us
I20260709 13:00:57.752398 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:57.719780 23105 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:57.726778 23094 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:57.759055 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.738474 23030 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:57.765790 20161 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.678503 23053 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.741387 22729 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:00:57.770337 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 351303us
I20260709 13:00:57.755609 22744 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:57.756198 23066 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:00:57.773313 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 375720us
I20260709 13:00:57.785130 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.759955 23042 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.749356 22378 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:57.780488 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:57.787488 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.793476 20008 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:00:57.799598 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.784768 23055 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.765532 23094 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.803675 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:57.784020 22967 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:57.774466 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 453496us
I20260709 13:00:57.809836 20013 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:57.790093 23042 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.770721 22606 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:00:57.810859 20129 leader_election.cc:400] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:57.818857 20129 leader_election.cc:403] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:00:57.802380 23116 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:00:57.802343 23061 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.823894 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:57.813092 23094 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.848609 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:57.850208 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.768852 23079 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:57.817783 23042 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.816531 22967 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.865914 22967 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:57.866736 22967 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:57.867373 22967 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:57.868213 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.869169 22744 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:57.869772 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.870965 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.854588 20161 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:57.796818 23036 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:00:57.812644 22729 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.875723 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.877100 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:57.819849 22378 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:57.767563 23103 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:57.904364 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.905972 20082 raft_consensus.cc:2393] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:57.907737 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.861868 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.874074 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:57.909518 20161 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.790668 23030 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:57.853518 20013 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:00:57.912236 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.902611 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:00:57.859834 23111 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:00:57.814406 23055 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:57.954319 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:57.943997 20086 raft_consensus.cc:2368] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:57.944808 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:57.854745 23025 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:57.920240 20085 raft_consensus.cc:2368] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
W20260709 13:00:57.986467 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 199064us
I20260709 13:00:57.987972 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:57.993435 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.950245 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:57.952314 20161 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:58.073657 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:58.081606 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 90320.8us
I20260709 13:00:58.083038 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:57.876183 22755 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:57.830152 23035 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:58.087649 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:58.125988 19982 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:58.132997 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 1266) took 1015 ms. Trace:
I20260709 13:00:58.133615 20091 rpcz_store.cc:276] 0709 13:00:57.118094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.161472 (+ 43378us) service_pool.cc:224] Handling call
0709 13:00:58.132698 (+971226us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540078f854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:58.132907 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220133120}
I20260709 13:00:57.855226 23061 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:58.136991 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 187042us
I20260709 13:00:58.137959 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:00:58.212684 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 85363.3us
W20260709 13:00:58.213960 20160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.073s	sys 0.083s
W20260709 13:00:58.214395 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:58.224747 20160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.073s	sys 0.084s
I20260709 13:00:58.215739 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:58.083137 23057 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:58.074503 23108 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.222280 22207 raft_consensus.cc:2764] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:00:58.215842 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 609698us
I20260709 13:00:58.225919 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:58.226357 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.081818 23063 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:00:58.235520 20083 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:58.227821 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:58.242018 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 591) took 1582 ms. Trace:
I20260709 13:00:58.077937 23062 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:58.136255 23025 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:58.244442 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:58.251748 20013 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:58.223449 22834 raft_consensus.cc:2804] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:58.214428 23076 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:58.262553 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:58.196966 23041 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:58.225622 22606 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:58.233886 22378 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.242652 20160 rpcz_store.cc:276] 0709 13:00:56.659533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.196017 (+536484us) service_pool.cc:224] Handling call
0709 13:00:57.212292 (+ 16275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.212556 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:57.679133 (+466577us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.230538 (+551405us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.241278 (+ 10740us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:58.241429 (+   151us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:58.241607 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.241756 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.241934 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":691031,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":35865,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":964326,"threads_started":2}
I20260709 13:00:58.148604 22755 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:58.288203 20013 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:58.234439 22914 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:00:58.309648 20090 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
W20260709 13:00:58.247198 22931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.002s	sys 0.026s
W20260709 13:00:58.315954 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:58.285957 22747 consensus_meta.cc:287] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.641s	user 0.015s	sys 0.005s
W20260709 13:00:58.384392 23121 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:00:58.429065 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 949) took 1011 ms. Trace:
W20260709 13:00:58.381764 23126 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:00:58.414707 22890 consensus_meta.cc:287] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.936s	user 0.009s	sys 0.004s
W20260709 13:00:58.380628 23117 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:00:58.382493 23062 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:58.374670 22696 consensus_meta.cc:287] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.790s	user 0.011s	sys 0.001s
I20260709 13:00:58.244277 23108 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:58.413189 22755 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:58.286068 22378 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:58.377421 23061 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.279745 22611 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:00:58.283759 23076 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:58.292192 22606 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.294281 23053 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.285442 23041 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.346400 23133 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.368330 23055 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.367250 23035 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:58.448405 22931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.004s	sys 0.026s
W20260709 13:00:58.383420 23042 consensus_meta.cc:287] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.000s
W20260709 13:00:58.430271 23120 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.183s	user 0.007s	sys 0.000s
I20260709 13:00:58.429569 23036 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:58.425927 22884 consensus_meta.cc:287] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.891s	user 0.010s	sys 0.001s
W20260709 13:00:58.393708 20012 consensus_meta.cc:287] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.699s	user 0.021s	sys 0.000s
I20260709 13:00:58.549286 20012 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
W20260709 13:00:58.296854 22949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.007s	sys 0.044s
I20260709 13:00:58.473361 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:58.552114 20164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.004s	sys 0.034s
W20260709 13:00:58.485195 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 347293us
I20260709 13:00:58.552969 20129 leader_election.cc:400] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 5. Current term is 6.
I20260709 13:00:58.553861 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:58.470660 20854 rpcz_store.cc:276] 0709 13:00:57.418113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.418650 (+   537us) service_pool.cc:224] Handling call
0709 13:00:57.441186 (+ 22536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.441458 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:57.854330 (+412872us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.428394 (+574064us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.428631 (+   237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:58.428687 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:58.428748 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.428834 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.428991 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28997,"prepare.queue_time_us":139869,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":374,"raft.queue_time_us":170442,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":205,"thread_start_us":874853,"threads_started":3,"wal-append.queue_time_us":609693}
I20260709 13:00:58.556017 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:58.407641 23025 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:58.460583 22600 consensus_meta.cc:287] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.721s	user 0.013s	sys 0.000s
I20260709 13:00:58.503755 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.504649 22378 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:58.508018 23073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.019s	sys 0.017s
I20260709 13:00:58.471833 22747 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.502437 22744 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:58.324267 20014 consensus_meta.cc:287] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.630s	user 0.013s	sys 0.001s
W20260709 13:00:58.552026 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 78794.6us
W20260709 13:00:58.484486 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 250761us
W20260709 13:00:58.557011 20164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.004s	sys 0.034s
I20260709 13:00:58.557080 20012 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:58.555536 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:58.502794 22696 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.472941 23048 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:58.580585 20743 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:58.555522 22949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.010s	sys 0.058s
I20260709 13:00:58.572762 20014 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:00:58.584559 20014 pending_rounds.cc:85] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 3
I20260709 13:00:58.584888 20014 pending_rounds.cc:107] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:58.585171 20014 raft_consensus.cc:2889] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:58.573732 22207 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:58.547940 23137 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:00:58.571533 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 999) took 1449 ms. Trace:
I20260709 13:00:58.583822 20743 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:58.567669 23073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.020s	sys 0.017s
I20260709 13:00:58.591260 20743 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:58.559728 23055 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:58.595000 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:58.599112 20014 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:58.559132 23150 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:58.600195 20853 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:58.600760 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:58.526862 23133 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:58.604370 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:58.589262 23149 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:00:58.549806 23042 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.597684 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:00:58.601121 23058 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.863s	user 0.014s	sys 0.002s
W20260709 13:00:58.640663 23139 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 13:00:58.641645 23141 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:00:58.642261 23158 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:00:58.643898 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.660097 23091 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:58.660835 23091 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:58.662210 23042 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.660557 22747 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:58.661983 23091 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:58.664033 23123 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:00:58.653453 23018 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.075s	user 0.014s	sys 0.000s
I20260709 13:00:58.666440 23018 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:58.668125 20010 raft_consensus.cc:2368] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9.
I20260709 13:00:58.669253 23018 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.656308 23058 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:58.670281 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:58.670964 20160 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:58.672259 23058 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:58.672489 23154 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:00:58.651494 23048 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.674809 23041 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:58.678452 22696 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:58.663755 23091 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:58.681137 23155 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:00:58.602531 20744 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:58.640218 23153 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:58.684373 23153 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:58.686851 23153 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:58.555255 22884 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.552574 23036 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:58.691368 23036 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:58.689252 22884 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.593830 22755 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:58.653411 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 55820.2us
I20260709 13:00:58.697849 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.698510 20856 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:00:58.698865 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:58.699724 23076 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.700162 20743 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:58.700802 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.701108 22606 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.701534 23153 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:58.701742 23061 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:58.702050 23153 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.702296 23061 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.702661 23153 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.702770 23061 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:58.701126 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:58.705287 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:58.705737 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:58.706144 23132 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:00:58.706664 20856 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:58.707401 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 607) took 1381 ms. Trace:
I20260709 13:00:58.707754 20159 rpcz_store.cc:276] 0709 13:00:57.325802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.490927 (+165125us) service_pool.cc:224] Handling call
0709 13:00:57.491241 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.506564 (+ 15323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.506603 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.533093 (+ 26490us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.981051 (+447958us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:57.981124 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.981240 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.707215 (+725975us) raft_consensus.cc:1685] finished
0709 13:00:58.707236 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.707349 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14894,"raft.queue_time_us":31735,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":7920,"thread_start_us":376544,"threads_started":1,"wal-append.queue_time_us":783363}
I20260709 13:00:58.709069 20856 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
W20260709 13:00:58.710055 23148 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:00:58.568348 22600 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:58.593577 23151 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:58.712103 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 109619us
I20260709 13:00:58.712453 23151 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:58.713049 20088 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:58.713485 22600 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:58.643978 23055 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:58.714777 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:58.694499 20858 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:58.694700 23057 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:58.715444 23055 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:58.716832 23057 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:58.718321 20163 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:58.696784 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.716771 20088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:00:58.720407 20088 raft_consensus.cc:606] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:58.656252 23150 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.720834 20088 raft_consensus.cc:614] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:58.717259 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.696784 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 933) took 1025 ms. Trace:
I20260709 13:00:58.721639 20161 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:00:58.721999 20008 rpcz_store.cc:276] 0709 13:00:57.671011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.671116 (+   105us) service_pool.cc:224] Handling call
0709 13:00:57.684662 (+ 13546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.794014 (+109352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.696546 (+902532us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400567f54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:58.696717 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249237248}
I20260709 13:00:58.722543 23108 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.723109 23108 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:58.723731 23108 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:58.724275 23108 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:58.725103 23108 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.600574 22890 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:58.728698 23057 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:58.730330 23151 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:58.731071 23057 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.732419 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:58.736635 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.732149 23076 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:58.739450 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.738420 22949 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:58.741325 23133 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.741173 22600 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:58.743858 22600 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:58.745196 23094 consensus_meta.cc:287] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.014s
I20260709 13:00:58.745699 23094 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.720995 23150 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:58.750239 23150 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:00:58.753372 23066 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.958s	user 0.008s	sys 0.004s
I20260709 13:00:58.753997 23066 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:58.754658 20158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33788 (request call id 596) took 1990 ms (client timeout 1859 ms). Trace:
I20260709 13:00:58.755333 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:00:58.755208 20158 rpcz_store.cc:269] 0709 13:00:56.764161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.876867 (+112706us) service_pool.cc:224] Handling call
0709 13:00:58.754440 (+1877573us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b540009b354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:58.754572 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844936960}
I20260709 13:00:58.758252 22606 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.758772 22606 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:58.759225 22606 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.761050 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:58.765913 22207 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:58.769974 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:58.770982 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.788573 23165 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:58.788951 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.789183 23165 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.789592 20859 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:00:58.790603 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:58.773106 20858 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:00:58.791610 23165 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.784111 20856 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:58.785320 20160 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:00:58.786998 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.787618 20855 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:00:58.768751 23094 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.795635 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:58.798054 20856 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:58.798244 23165 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:58.791837 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:58.795482 20858 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:00:58.768751 22949 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:00:58.811046 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.811816 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.812160 20160 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:00:58.793222 22207 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.816926 20088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:00:58.817679 20088 raft_consensus.cc:606] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:58.818059 20088 raft_consensus.cc:614] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:58.815636 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.845741 20161 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:58.848973 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.849907 20158 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:00:58.810585 22951 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 13:00:58.857635 20852 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.009s	sys 0.021s
W20260709 13:00:58.859617 20852 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.009s	sys 0.021s
I20260709 13:00:58.813508 22762 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:00:58.846146 23165 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:58.860538 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 950) took 1442 ms. Trace:
I20260709 13:00:58.796952 22378 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.866452 20852 rpcz_store.cc:276] 0709 13:00:57.418422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.418573 (+   151us) service_pool.cc:224] Handling call
0709 13:00:57.441876 (+ 23303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.442102 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:57.858874 (+416772us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.860121 (+1001247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:58.860193 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:58.860253 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.860317 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.860459 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":114825,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":135016,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":1335527,"threads_started":3,"wal-append.queue_time_us":618148}
W20260709 13:00:58.862056 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 82145.3us
I20260709 13:00:58.877305 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:58.587637 20164 rpcz_store.cc:276] 0709 13:00:57.122303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.301565 (+179262us) service_pool.cc:224] Handling call
0709 13:00:57.301896 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.302146 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:57.800178 (+498032us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.570858 (+770680us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.571070 (+   212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:58.571128 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:58.571192 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.571286 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.571458 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":835206,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":220,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11304,"thread_start_us":1241323,"threads_started":2}
I20260709 13:00:58.882923 20084 raft_consensus.cc:2410] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:00:58.865687 23108 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.891052 20085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:00:58.892021 20085 raft_consensus.cc:606] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:00:58.892423 20085 raft_consensus.cc:614] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:58.888391 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:58.899538 20852 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:58.904827 23061 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.906384 23153 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.906814 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:58.907258 23061 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:58.907578 20858 raft_consensus.cc:2368] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7.
I20260709 13:00:58.908228 20859 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:58.910429 23150 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:00:58.910929 23150 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:58.911335 23150 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:58.911376 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 930) took 1408 ms. Trace:
I20260709 13:00:58.911908 20853 rpcz_store.cc:276] 0709 13:00:57.502964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.554608 (+ 51644us) service_pool.cc:224] Handling call
0709 13:00:58.911155 (+1356547us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54002404d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:58.911307 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2250310016}
I20260709 13:00:58.913445 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.914503 23165 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:58.916603 23165 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:58.917466 23057 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:58.917930 22378 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:58.918258 23166 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:00:58.918721 23166 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:58.919866 23166 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:58.921227 23166 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:00:58.922804 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:58.924420 23062 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.924853 23062 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.925208 23062 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.930310 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:58.914551 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:58.927606 23151 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.893862 23108 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:58.970315 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:58.971733 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:58.972038 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 935) took 1300 ms. Trace:
I20260709 13:00:58.972491 20852 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:58.972608 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 934) took 1301 ms. Trace:
I20260709 13:00:58.973415 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:58.975409 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 712947us
I20260709 13:00:58.982360 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:00:58.974845 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 700739us
I20260709 13:00:58.987404 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:58.990291 22207 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:58.990705 22207 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:00:58.990866 19982 connection.cc:256] RPC call timeout handler was delayed by -1.4953620259993841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:00:58.927739 23025 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:58.991637 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:58.997967 20163 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:00:58.920348 22606 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:58.995018 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:58.999504 23115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.002s	sys 0.018s
W20260709 13:00:59.000047 23115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.002s	sys 0.018s
I20260709 13:00:59.000162 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.000757 22606 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.972100 23063 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:00:59.000869 20852 raft_consensus.cc:2368] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:58.972462 20015 rpcz_store.cc:276] 0709 13:00:57.671676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.671783 (+   107us) service_pool.cc:224] Handling call
0709 13:00:57.672161 (+   378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:57.688588 (+ 16427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.688627 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:57.689302 (+   675us) log.cc:844] Serialized 52 byte log entry
0709 13:00:58.147433 (+458131us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:58.237914 (+ 90481us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.266261 (+ 28347us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.971418 (+705157us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b1800129640. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:00:58.971601 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.971667 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.971808 (+   141us) raft_consensus.cc:1685] finished
0709 13:00:58.971825 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.971982 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17664,"prepare.queue_time_us":115339,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":82,"raft.queue_time_us":68613,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":70,"spinlock_wait_cycles":40927744,"thread_start_us":430828,"threads_started":1,"wal-append.queue_time_us":667469}
I20260709 13:00:59.001940 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:59.005779 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:58.973347 20011 rpcz_store.cc:276] 0709 13:00:57.671338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.672116 (+   778us) service_pool.cc:224] Handling call
0709 13:00:57.687398 (+ 15282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:58.245639 (+558241us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:58.245687 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:58.284345 (+ 38658us) log.cc:844] Serialized 26 byte log entry
0709 13:00:58.504234 (+219889us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:58.504272 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.972195 (+467923us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540064ebd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:58.972240 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.972284 (+    44us) raft_consensus.cc:1685] finished
0709 13:00:58.972296 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.972513 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18478,"raft.queue_time_us":68718,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":32281,"spinlock_wait_cycles":260272512,"thread_start_us":618367,"threads_started":2,"wal-append.queue_time_us":331635}
I20260709 13:00:58.991130 22207 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:58.974558 22755 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:59.006999 20156 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:00:59.006150 20159 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:00:59.004021 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:58.928061 23167 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:00:59.014096 20129 leader_election.cc:400] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:59.016839 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.018047 20129 leader_election.cc:403] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:00:59.019431 23167 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:59.020090 23167 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:59.021206 23167 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.022991 23167 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.041729 23079 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:59.042397 23079 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:59.044945 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.045517 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.048118 22715 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.047855 22207 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.048460 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.048659 22715 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.050441 22207 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.051249 22715 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:00:59.052815 20008 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: []
I20260709 13:00:59.054930 23103 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:59.055266 20009 raft_consensus.cc:1275] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:00:59.060189 23153 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:59.073776 23108 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:59.075803 20015 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:00:59.005445 23063 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.076692 23063 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:59.078341 23079 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:58.985321 23151 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.081519 20016 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:59.082123 20016 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:59.082834 23151 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.083999 22755 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.086328 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.087232 20008 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:00:59.087404 22755 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:58.994282 22378 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:00:59.075479 23145 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:59.113886 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:59.089963 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.117982 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.149372 20010 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:59.088989 20009 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:00:59.082901 20016 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.152758 20016 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.093466 20012 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:59.095347 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.101581 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:59.154219 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 140325us
W20260709 13:00:59.155243 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53761.4us
I20260709 13:00:59.156857 20056 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.099283 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.160063 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.161123 20008 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:59.158428 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.156662 20129 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:00:59.164814 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67008.7us
I20260709 13:00:59.165954 22207 consensus_queue.cc:1059] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:59.166239 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:00:59.167156 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 180742us
I20260709 13:00:59.169576 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.168011 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:59.175904 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.176828 22207 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:00:59.177280 22207 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:59.119755 22715 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:59.178133 22207 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.179908 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.180395 23042 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:59.181032 22951 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:59.181612 22600 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.182413 22693 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:59.181757 23104 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:59.182785 23122 consensus_queue.cc:1048] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:59.183954 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:59.152495 22967 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.218202 23104 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.218073 22967 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.219379 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.220111 22951 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:59.220327 22967 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:59.220654 22606 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:59.221052 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.221184 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.168781 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.189247 20008 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:59.159910 23142 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.222275 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.222450 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.223142 20091 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:59.223398 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.196702 22600 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.192459 22744 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:59.226413 22744 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.226948 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.228602 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.229328 20091 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:59.229368 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.230304 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.231220 20091 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:59.189860 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:59.188782 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.187266 23146 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:00:59.243739 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54274.9us
I20260709 13:00:59.170821 23057 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:59.203871 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:59.210247 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.174211 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:59.187453 23031 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:59.207231 22834 raft_consensus.cc:697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:59.222616 23142 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:59.194911 22207 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:59.220592 22951 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.164829 22931 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.200927 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1104) took 1882 ms. Trace:
I20260709 13:00:59.202804 23164 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:59.212196 22914 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:59.190181 20016 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
W20260709 13:00:59.254118 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 80006.2us
I20260709 13:00:59.256101 20129 leader_election.cc:400] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:00:59.277575 20129 leader_election.cc:400] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9.
I20260709 13:00:59.233728 22755 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:59.314352 22755 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:00:59.259429 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 55664.7us
I20260709 13:00:59.317277 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:59.191223 23025 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:59.197629 23066 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.264075 20085 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 6: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:59.232648 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.233070 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.321579 20091 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:59.272387 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.322583 23025 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.323227 23025 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.328063 23025 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.337095 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:00:59.338440 22914 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:00:59.339336 22914 raft_consensus.cc:2764] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:00:59.323277 20089 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:59.341585 23146 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:59.342773 23146 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:59.265494 20165 rpcz_store.cc:276] 0709 13:00:57.318074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.460525 (+142451us) service_pool.cc:224] Handling call
0709 13:00:57.461081 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.461346 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:57.802691 (+341345us) log.cc:844] Serialized 14 byte log entry
0709 13:00:59.196646 (+1393955us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b1800045dc0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a
0709 13:00:59.199896 (+  3250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:59.200369 (+   473us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:59.200440 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.200547 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.200836 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2936,"prepare.queue_time_us":785496,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":5918,"raft.queue_time_us":3474,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":26775,"spinlock_wait_cycles":31565696,"thread_start_us":787854,"threads_started":2,"wal-append.queue_time_us":581360}
I20260709 13:00:59.350739 20165 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:59.197976 23036 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:00:59.351938 23036 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:59.353140 23036 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.354166 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.354940 23036 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:59.355113 20089 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 4.
I20260709 13:00:59.357179 23150 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.357930 23076 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:59.275704 20016 pending_rounds.cc:85] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 4
I20260709 13:00:59.360162 20016 pending_rounds.cc:107] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:00:59.360546 20016 raft_consensus.cc:2889] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:59.273825 22951 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.280367 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:00:59.368882 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 88578.9us
I20260709 13:00:59.369840 22755 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:59.370184 23025 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.265357 23005 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:59.269320 23031 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:59.373358 23031 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.250800 23122 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.288628 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 922) took 2306 ms. Trace:
I20260709 13:00:59.291541 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:00:59.376919 23020 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:00:59.377388 20082 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.298924 20088 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:59.299443 22834 consensus_queue.cc:237] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:00:59.383984 23176 log.cc:927] Time spent T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:00:59.262533 22967 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:59.393729 22967 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:59.393350 23062 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.300274 20012 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:00:59.376271 20017 rpcz_store.cc:276] 0709 13:00:56.981993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.984018 (+  2025us) service_pool.cc:224] Handling call
0709 13:00:56.984339 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.724191 (+739852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.288043 (+1563852us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b540073e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:59.288340 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1699889536}
I20260709 13:00:59.308938 20163 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:59.304456 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:59.396802 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 92434.9us
I20260709 13:00:59.199612 22715 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:59.398358 20130 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:59.305015 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:59.400218 20162 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.400665 22877 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:59.401284 23166 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.401487 22959 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:59.281241 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.306339 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:59.403571 20159 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:00:59.304430 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:59.404399 23124 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:59.399546 20130 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.404898 23124 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.405273 23124 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.406301 23166 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:59.407260 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.408582 23166 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.410276 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:59.413079 20163 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:59.413781 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:59.414419 20158 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:59.278060 23164 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.345925 20091 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: []
I20260709 13:00:59.394307 22967 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.299670 20164 raft_consensus.cc:2410] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:00:59.297477 22600 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.399646 22715 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:59.346714 22835 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:59.416759 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.430569 20162 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:59.431003 20013 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:00:59.422652 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:59.424041 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:59.369930 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:00:59.432410 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 62604.5us
I20260709 13:00:59.432977 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:59.433068 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:59.427574 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.427835 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:59.434227 22207 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:59.434587 22207 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.434868 22207 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:59.420403 20082 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:59.442452 20158 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:00:59.443991 20162 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:59.446134 20853 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:59.436645 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:59.346761 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.447321 20163 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:59.435062 20744 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:59.448590 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.449280 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:59.450296 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.451911 22207 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:59.455111 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 108465us
I20260709 13:00:59.455811 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:59.423664 23005 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.460213 20129 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.420045 23146 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:59.426740 23025 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:59.436812 22877 raft_consensus.cc:2764] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:00:59.432077 23076 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.436064 23066 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.468254 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:59.427667 23150 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.462141 20161 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:59.466529 20130 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:00:59.461512 23151 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:59.443542 22890 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:59.430068 22971 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.456368 23166 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:00:59.470867 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:59.461133 23042 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:59.461835 22600 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:59.468595 22207 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.475014 20082 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:59.456373 22755 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:00:59.461704 23057 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:59.466395 23025 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.467581 22884 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:59.490715 20015 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:00:59.491888 20161 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:00:59.487082 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:59.487742 20160 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:59.482836 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:59.488595 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:00:59.463320 23005 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.500358 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:59.500622 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.523525 23042 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:00:59.611722 20012 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:59.453727 23165 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:00:59.503623 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.612975 20859 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:59.506100 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.457486 23018 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:00:59.476202 23088 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:59.527441 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:59.471987 23076 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:59.481732 23122 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:00:59.607299 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 147137us
I20260709 13:00:59.481105 22890 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:00:59.525063 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:59.516165 23151 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:59.633724 20129 leader_election.cc:400] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 6 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7.
I20260709 13:00:59.528049 22207 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:59.523478 23025 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.523108 23020 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:59.620177 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 90256.9us
W20260709 13:00:59.620328 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 120060us
I20260709 13:00:59.521202 22600 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:00:59.473347 23066 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:59.622466 20857 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:59.617812 20855 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:00:59.647994 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1099) took 1801 ms. Trace:
I20260709 13:00:59.635234 20012 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:59.649758 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:00:59.648402 20084 rpcz_store.cc:276] 0709 13:00:57.846813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.849731 (+  2918us) service_pool.cc:224] Handling call
0709 13:00:59.647797 (+1798066us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54004c6154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:59.647939 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2309997312}
I20260709 13:00:59.655239 20859 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:00:59.656409 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.657008 20859 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:00:59.622846 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.657431 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.641746 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:00:59.636709 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:00:59.666203 20859 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:00:59.500541 22971 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:59.646056 22600 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.667336 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.677727 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.648041 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.632972 22914 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:59.622931 23122 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:59.678026 20859 raft_consensus.cc:2393] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:59.675657 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.684204 20085 raft_consensus.cc:2376] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9. Re-sending same reply.
I20260709 13:00:59.671244 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:59.643610 22834 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:59.638280 22890 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.706502 23042 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:00:59.708366 23151 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:00:59.715106 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.669147 23057 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:00:59.755200 19981 connection.cc:256] RPC call timeout handler was delayed by -1.469992911999725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
I20260709 13:00:59.764550 19981 leader_election.cc:400] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8.
I20260709 13:00:59.765920 23135 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:59.749006 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:59.768509 20859 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:00:59.760255 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.638222 23079 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:00:59.791293 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.769354 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:00:59.782734 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 122926us
I20260709 13:00:59.783365 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.795040 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.796190 20013 raft_consensus.cc:2410] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:59.712224 23122 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.703395 23165 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:00:59.774109 20084 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
W20260709 13:00:59.805440 20129 leader_election.cc:336] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.667s (SENT)
I20260709 13:00:59.753546 23020 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.813807 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:00:59.818710 20014 raft_consensus.cc:2393] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:59.639458 23146 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.818596 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.814149 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:59.775449 22693 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.000s
I20260709 13:00:59.822409 20090 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:00:59.823315 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:00:59.763552 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:00:59.659596 22378 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.714715 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:59.833204 20086 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:00:59.833178 20087 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:00:59.838590 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.839411 20012 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
W20260709 13:00:59.844792 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 203115us
I20260709 13:00:59.845525 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:00:59.828711 22744 consensus_meta.cc:287] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.602s	user 0.014s	sys 0.000s
I20260709 13:00:59.849136 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:59.850324 22378 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:00:59.846724 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:00:59.817858 23057 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:00:59.825868 22834 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:00:59.871709 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.882755 20012 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:00:59.881608 20859 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:59.885334 20087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1100) took 2038 ms (client timeout 1667 ms). Trace:
W20260709 13:00:59.885886 20087 rpcz_store.cc:269] 0709 13:00:57.847098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.849834 (+  2736us) service_pool.cc:224] Handling call
0709 13:00:59.885120 (+2035286us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b5400664ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:00:59.885245 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4545215232}
I20260709 13:00:59.863930 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:00:59.842854 22606 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.890797 20086 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:00:59.849207 23108 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:00:59.875660 23184 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.404s	user 0.000s	sys 0.001s
W20260709 13:00:59.869997 23063 consensus_meta.cc:287] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.793s	user 0.014s	sys 0.000s
I20260709 13:00:59.888010 20090 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:59.870452 22755 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:59.848413 23062 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:00:59.898393 20855 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:00:59.821957 23165 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:00:59.884724 23146 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:59.924369 23199 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:00:59.933890 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.827950 23020 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:00:59.867113 22744 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:00:59.923276 23187 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.421s	user 0.000s	sys 0.001s
I20260709 13:00:59.911867 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.833393 23190 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:00:59.925518 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:00:59.920634 23188 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.077s	user 0.001s	sys 0.001s
I20260709 13:00:59.870402 22958 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 13:00:59.849946 23182 consensus_queue.cc:1175] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e5eb27db496b47129df3e689e546d94a" responder_term: 9 status { last_received { term: 9 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 9 index: 6 } }
I20260709 13:00:59.957319 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 936) took 1024 ms. Trace:
I20260709 13:00:59.925386 22755 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:00:59.936815 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:00:59.926456 20086 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
W20260709 13:00:59.980063 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 174300us
I20260709 13:01:00.000092 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:00:59.936815 23198 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:00:59.990355 23186 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 13:01:00.005515 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.006735 23196 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.007390 23196 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:00.008659 23193 log.cc:927] Time spent T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:01:00.001595 22834 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.009738 23196 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:00:59.997486 23076 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.014284 22744 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:00:59.957168 23183 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.319s	user 0.001s	sys 0.001s
I20260709 13:01:00.016160 23165 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:00.016707 23020 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:00.016172 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 934) took 1084 ms. Trace:
I20260709 13:01:00.017273 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:00:59.989705 20854 rpcz_store.cc:276] 0709 13:00:58.932695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.031272 (+ 98577us) service_pool.cc:224] Handling call
0709 13:00:59.031671 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.083303 (+ 51632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.083334 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.084010 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.265022 (+181012us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:59.265084 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.390442 (+125358us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b5400005ad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:00:59.390522 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.956988 (+566466us) raft_consensus.cc:1685] finished
0709 13:00:59.957014 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.957251 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2489,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"spinlock_wait_cycles":117816064,"thread_start_us":180474,"threads_started":1,"wal-append.queue_time_us":360804}
I20260709 13:00:59.970197 20859 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:59.944242 23005 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:00:59.944706 23066 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:00:59.969867 23185 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.441s	user 0.001s	sys 0.000s
I20260709 13:00:59.963274 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:00:59.947834 23166 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:00:59.903563 22606 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.010277 22834 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:00.010890 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:00.067107 20165 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:00.005270 22729 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:59.936296 22971 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.017683 20852 rpcz_store.cc:276] 0709 13:00:58.932211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.019642 (+ 87431us) service_pool.cc:224] Handling call
0709 13:00:59.020030 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.020666 (+   636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.020701 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.021198 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.251020 (+229822us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:59.251065 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.251138 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.015983 (+764845us) raft_consensus.cc:1685] finished
0709 13:01:00.016004 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.016117 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":229431,"threads_started":1,"wal-append.queue_time_us":452721}
I20260709 13:00:59.983815 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:00.097018 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:00.098075 20014 raft_consensus.cc:2368] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10.
I20260709 13:01:00.099511 20014 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:00.039623 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:00.101393 20013 raft_consensus.cc:2368] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9.
W20260709 13:00:59.957849 23192 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:01:00.018312 20158 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:00.041165 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:00.048679 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.059639 22951 consensus_meta.cc:287] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.003s
W20260709 13:01:00.061034 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 97884.4us
I20260709 13:00:59.937747 23199 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.062312 20162 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:00:59.929217 23063 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:00.054812 22967 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.000s
W20260709 13:01:00.054205 23036 consensus_meta.cc:287] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.698s	user 0.002s	sys 0.012s
I20260709 13:01:00.038167 23198 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:00.069419 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:00.118136 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:00.118677 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 49372us
I20260709 13:01:00.049258 23165 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.125597 20011 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:00:59.938710 23057 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:00.065152 23005 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:00.106882 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:00.062644 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 487078us
I20260709 13:01:00.105647 22729 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:00.128767 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:00.106276 22951 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:00.130213 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 262713us
I20260709 13:01:00.127583 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:00.061156 23122 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:00.136472 20087 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:01:00.137548 20162 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:00.161999 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 55206.4us
I20260709 13:01:00.138283 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:00.140776 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1111) took 1150 ms. Trace:
I20260709 13:01:00.165380 20157 rpcz_store.cc:276] 0709 13:00:58.990724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.992121 (+  1397us) service_pool.cc:224] Handling call
0709 13:00:59.003599 (+ 11478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.004005 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.004124 (+   119us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.096062 (+ 91938us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.251511 (+155449us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:59.251537 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.251580 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.140493 (+888913us) raft_consensus.cc:1685] finished
0709 13:01:00.140524 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.140698 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26644,"raft.queue_time_us":110,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":11208,"thread_start_us":154897,"threads_started":1,"wal-append.queue_time_us":373307}
I20260709 13:01:00.133152 23165 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.123800 22715 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:00.144429 20164 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1113) took 1153 ms. Trace:
I20260709 13:01:00.143759 22729 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.145682 23025 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:00.146932 23124 consensus_meta.cc:287] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.741s	user 0.013s	sys 0.000s
I20260709 13:01:00.124233 23198 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:00.149083 22890 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:00.150974 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:00.152074 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.153962 20859 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:00.158155 20743 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:00.170298 20164 rpcz_store.cc:276] 0709 13:00:58.991255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.992386 (+  1131us) service_pool.cc:224] Handling call
0709 13:01:00.144220 (+1151834us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54001be4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:00.144371 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":667071872}
I20260709 13:01:00.171939 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.173143 22715 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.181424 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.184449 20082 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:00.118211 22967 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:00.219252 20014 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:00.221146 20158 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:00.168987 23203 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:01:00.212827 22715 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.213804 23199 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.214430 20857 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:00.215862 22890 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:00.224264 20085 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:00.175822 23020 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:00.115072 22755 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:00.174186 23017 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:00.193552 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:00.221786 19982 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.236021 20164 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:00.241149 22755 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.224944 23094 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:00.245191 23017 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:00.244701 22600 consensus_meta.cc:287] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.004s
I20260709 13:01:00.246338 22600 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.244709 23020 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:00.249637 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:00.250960 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:00.252202 22600 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.252610 23017 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.258836 20857 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:00.265223 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:00.265503 23031 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:00.266183 22600 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:00.266676 23199 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.219954 23194 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:00.267843 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 74375.4us
I20260709 13:01:00.269165 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:00.218762 22378 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:00.276444 20016 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:00.285912 20743 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:00.280715 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:00.186385 22949 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:01:00.217558 23036 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:01:00.300438 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:00.311002 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.190544 23124 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:00.312397 20157 raft_consensus.cc:1275] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:00.229481 22971 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
W20260709 13:01:00.367220 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 66857us
I20260709 13:01:00.300872 23194 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:00.380203 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.302865 22890 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:00.411298 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.303263 23203 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.305339 23202 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:00.425632 20859 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:00.237437 23201 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.176s
I20260709 13:01:00.299795 23018 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:01:00.432699 20012 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:00.432855 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 939) took 3401 ms. Trace:
I20260709 13:01:00.449934 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:00.441593 23142 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:00.442857 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.444172 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:00.445489 23164 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:00.446040 23203 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.405153 23031 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:00.451545 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 928) took 1547 ms. Trace:
I20260709 13:01:00.451628 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.450767 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:00.453680 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:00.500113 20853 rpcz_store.cc:276] 0709 13:00:58.903877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.914305 (+ 10428us) service_pool.cc:224] Handling call
0709 13:01:00.451336 (+1537031us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b540000a5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:00.451484 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1175456640}
I20260709 13:01:00.500790 20009 raft_consensus.cc:2376] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8. Re-sending same reply.
I20260709 13:01:00.345804 22755 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:00.346706 23088 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:00.475157 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.455041 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:00.503759 20854 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:00.409044 23207 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.480087 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 501) took 1270 ms. Trace:
I20260709 13:01:00.450119 20850 rpcz_store.cc:276] 0709 13:00:57.022794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.022897 (+   103us) service_pool.cc:224] Handling call
0709 13:00:57.023183 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:57.374588 (+351405us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:57.374900 (+   312us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.780003 (+405103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:57.780092 (+    89us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:57.780157 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.888847 (+1108690us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b18003b1b20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6932a6 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419
0709 13:01:00.423555 (+1534708us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54000073d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:00.423621 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.423797 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2258,"prepare.queue_time_us":522,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":4471,"raft.queue_time_us":9520,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6854,"spinlock_wait_cycles":821103360,"thread_start_us":395148,"threads_started":1,"wal-append.queue_time_us":712555,"wal-append.run_cpu_time_us":10696,"wal-append.run_wall_time_us":1369141}
I20260709 13:01:00.422813 22890 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:00.509056 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.801s (SENT) [suppressed 25 similar messages]
W20260709 13:01:00.521790 20159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.015s	sys 0.026s
W20260709 13:01:00.530179 20159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.015s	sys 0.027s
I20260709 13:01:00.532742 20855 raft_consensus.cc:1217] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-8.5]   Dedup: 7.4->[8.5-8.5]
I20260709 13:01:00.532910 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.533391 20855 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 13:01:00.534104 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 951) took 1032 ms. Trace:
I20260709 13:01:00.529824 20088 rpcz_store.cc:276] 0709 13:00:59.209199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.209340 (+   141us) service_pool.cc:224] Handling call
0709 13:01:00.479772 (+1270432us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b5400a5e954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:00.479998 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177083648}
I20260709 13:01:00.534618 20858 rpcz_store.cc:276] 0709 13:00:59.501469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.501633 (+   164us) service_pool.cc:224] Handling call
0709 13:00:59.502051 (+   418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:59.502530 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:59.573475 (+ 70945us) log.cc:844] Serialized 14 byte log entry
0709 13:01:00.109430 (+535955us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:00.117197 (+  7767us) log.cc:844] Serialized 52 byte log entry
0709 13:01:00.117434 (+   237us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:00.531950 (+414516us) log.cc:844] Serialized 14 byte log entry
0709 13:01:00.532252 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.532342 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.532383 (+    41us) raft_consensus.cc:1685] finished
0709 13:01:00.532394 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.534036 (+  1642us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47973,"prepare.queue_time_us":110549,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":68,"raft.queue_time_us":141749,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":3720,"spinlock_wait_cycles":24832,"thread_start_us":864957,"threads_started":3,"wal-append.queue_time_us":297596}
W20260709 13:01:00.530870 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 390967us
I20260709 13:01:00.463135 23017 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.544730 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:00.463958 22931 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:00.465199 22378 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:00.490302 23066 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:00.351049 23199 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.172s
I20260709 13:01:00.549435 20163 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:00.549829 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 503) took 1340 ms. Trace:
I20260709 13:01:00.500113 23164 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:00.504398 23063 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.503432 23088 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:00.550350 20083 rpcz_store.cc:276] 0709 13:00:59.209767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.210817 (+  1050us) service_pool.cc:224] Handling call
0709 13:01:00.549538 (+1338721us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54009b8554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:00.549744 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385853184}
I20260709 13:01:00.503194 22884 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:00.557472 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.539398 22949 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:00.489413 22958 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:01:00.535364 22890 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:00.546221 22378 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.593480 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 952) took 1091 ms. Trace:
I20260709 13:01:00.595100 22378 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.568293 22884 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:00.595611 22958 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.565357 23001 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.594643 22715 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:00.596601 22958 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.597023 23001 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.594508 20851 rpcz_store.cc:276] 0709 13:00:59.501792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.502817 (+  1025us) service_pool.cc:224] Handling call
0709 13:00:59.503137 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.833852 (+330715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.833896 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.841846 (+  7950us) log.cc:844] Serialized 26 byte log entry
0709 13:01:00.296594 (+454748us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:00.296639 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.567535 (+270896us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54004204d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:00.567604 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.593148 (+ 25544us) raft_consensus.cc:1685] finished
0709 13:01:00.593168 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.593408 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":151827,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6984,"spinlock_wait_cycles":366952320,"thread_start_us":540579,"threads_started":2,"wal-append.queue_time_us":659021}
I20260709 13:01:00.599488 20015 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:00.572731 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.567124 23164 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.553714 23017 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.524426 22755 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.596750 22884 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.581354 23203 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:00.602335 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:00.610047 20161 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.603266 22715 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.557441 22951 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.566731 23057 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:00.629114 22958 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:00.636196 20011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 939) took 1589 ms (client timeout 2084 ms). Trace:
W20260709 13:01:00.636689 20011 rpcz_store.cc:269] 0709 13:00:59.046643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.048146 (+  1503us) service_pool.cc:224] Handling call
0709 13:01:00.635889 (+1587743us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002ebb54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:00.636129 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454028672}
I20260709 13:01:00.642446 23017 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.605229 23165 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.603865 23122 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:00.654162 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1176) took 1254 ms. Trace:
I20260709 13:01:00.654623 20017 rpcz_store.cc:276] 0709 13:00:59.399553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.422445 (+ 22892us) service_pool.cc:224] Handling call
0709 13:00:59.422843 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.432368 (+  9525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.432398 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.882882 (+450484us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:00:59.883233 (+   351us) log.cc:844] Serialized 26 byte log entry
0709 13:01:00.286832 (+403599us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:00.286921 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.287014 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.653817 (+366803us) raft_consensus.cc:1685] finished
0709 13:01:00.653843 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.654095 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12886,"raft.queue_time_us":21454,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7585,"spinlock_wait_cycles":7254784,"thread_start_us":403170,"threads_started":1,"wal-append.queue_time_us":678677}
I20260709 13:01:00.656064 23164 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.657596 23001 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:00.660297 22755 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:00.595674 23088 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:00.662716 23088 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.605849 22914 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:00.537263 23207 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.668274 23048 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.668902 23048 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.670070 23207 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.671005 23048 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:00.671343 23103 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.672079 23103 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:00.673331 23191 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:01:00.614610 23025 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:00.676971 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 943) took 1039 ms. Trace:
I20260709 13:01:00.677517 20857 rpcz_store.cc:276] 0709 13:00:59.637564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.647685 (+ 10121us) service_pool.cc:224] Handling call
0709 13:01:00.676735 (+1029050us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54000f4bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:00.676875 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1230775168}
I20260709 13:01:00.679293 23057 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:00.680307 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:00.680649 23057 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:00.684005 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:00.685041 20162 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:00.699208 20161 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
W20260709 13:01:00.701566 20013 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.009s	sys 0.009s
W20260709 13:01:00.702310 20013 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.009s	sys 0.009s
W20260709 13:01:00.708127 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 490820us
I20260709 13:01:00.675314 23103 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.711665 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:00.712133 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:00.711349 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:00.714385 20083 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:00.711278 20091 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:01:00.724810 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.728364 20857 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:00.732004 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:00.733094 20082 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:00.734730 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:00.737929 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.725692 20851 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:01:00.727552 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:00.707567 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:00.745103 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.703902 22949 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.749397 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:00.751235 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:00.753568 23020 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.754320 23020 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.756970 23145 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:00.757513 23145 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:00.758536 23145 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:00.742273 23146 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:01:00.753638 22949 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:00.746140 23203 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:01:00.762462 22949 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.764865 22606 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.767243 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.739567 23076 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.768452 23076 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:00.769308 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.770545 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.770737 23076 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:00.771492 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 63997.1us
I20260709 13:01:00.774542 20088 raft_consensus.cc:1275] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:00.784612 22967 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:00.785607 22967 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:00.786404 23124 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:00.786835 22967 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:00.787091 23124 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:00.787420 22967 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:00.789163 22967 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:00.792210 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:00.792208 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:00.793610 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.793913 20160 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:00.794781 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:00.802517 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.711905 23062 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:00.810226 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 338169us
I20260709 13:01:00.814733 20160 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:01:00.815357 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:00.759786 23145 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:00.837555 20854 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:00.850689 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.855657 20854 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:00.838284 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:00.787739 23124 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:00.874493 20857 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:00.876442 23124 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:00.887210 20854 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:00.894994 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 59243.6us
I20260709 13:01:00.836768 23146 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.898437 23142 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:01:00.852578 23182 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:01:00.916218 23020 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:00.833652 23062 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:00.848835 23203 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:01:00.909538 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.924198 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.001715 20851 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
W20260709 13:01:01.002230 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.926074 22207 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:00.933894 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.004181 22207 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.004522 20858 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:01.005999 20854 raft_consensus.cc:2410] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:00.942613 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:00.948297 22693 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:00.948284 23005 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:01.035444 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.952817 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 1019) took 1307 ms. Trace:
W20260709 13:01:00.968814 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 518120us
W20260709 13:01:01.043324 20744 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.000s	sys 0.048s
W20260709 13:01:01.043900 20744 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.000s	sys 0.048s
I20260709 13:01:00.974293 20055 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:00.976372 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 524810us
W20260709 13:01:00.980659 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.006704 20859 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:01.039098 20159 rpcz_store.cc:276] 0709 13:00:59.645659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.646049 (+   390us) service_pool.cc:224] Handling call
0709 13:00:59.646372 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.646601 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:59.899143 (+252542us) log.cc:844] Serialized 14 byte log entry
0709 13:01:00.952475 (+1053332us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:00.952536 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:00.952583 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.952647 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.952755 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":486234,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1970,"raft.queue_time_us":44200,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5796,"thread_start_us":1275182,"threads_started":3,"wal-append.queue_time_us":784814}
W20260709 13:01:01.061568 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.046342 20744 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.071899 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.070475 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:01.038641 23227 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
W20260709 13:01:01.072510 20084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.006s	sys 0.029s
I20260709 13:01:01.072767 22207 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:01.035423 23198 consensus_meta.cc:287] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.002s
W20260709 13:01:01.082306 20084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.006s	sys 0.029s
W20260709 13:01:01.061889 22744 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.005s	sys 0.030s
I20260709 13:01:01.091661 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.092563 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:01.092784 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.106289 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.106936 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:01.098542 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.009567 22693 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:01.093715 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.108443 20854 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:01.108887 20851 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:00.987607 22835 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:01:01.062266 23017 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:01.118310 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 144103us
I20260709 13:01:01.117985 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.063191 23228 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:01.104063 22744 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.030s
I20260709 13:01:01.120781 22207 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:01.086628 22378 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.071437 23079 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:01.108744 20859 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:01.126267 22207 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:01.132650 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.118248 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.138083 20851 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:01.124938 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:01.067152 23063 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:01:01.140213 20161 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:01.062474 23164 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:01:01.123392 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:01.151906 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 313693us
I20260709 13:01:01.088759 23166 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:01:01.135381 22378 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:01.161727 22378 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.118240 22755 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:01.120743 23197 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.009s	sys 0.016s
I20260709 13:01:01.135248 22931 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:01.158763 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44719.9us
W20260709 13:01:01.164881 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 88711.8us
I20260709 13:01:01.123777 22834 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:01.167183 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1184) took 1128 ms. Trace:
I20260709 13:01:01.153996 23020 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.174502 23150 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:01.181085 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 947) took 1070 ms. Trace:
I20260709 13:01:01.185379 20013 rpcz_store.cc:276] 0709 13:01:00.039258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.101987 (+ 62729us) service_pool.cc:224] Handling call
0709 13:01:00.102387 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:00.109532 (+  7145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:00.109586 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:00.124065 (+ 14479us) log.cc:844] Serialized 26 byte log entry
0709 13:01:00.702878 (+578813us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:00.702941 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.703043 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.166873 (+463830us) raft_consensus.cc:1685] finished
0709 13:01:01.166904 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.167102 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35661,"raft.queue_time_us":9219,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3358,"thread_start_us":546582,"threads_started":1,"wal-append.queue_time_us":803344}
W20260709 13:01:01.171017 23197 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.010s	sys 0.016s
I20260709 13:01:01.191481 20015 rpcz_store.cc:276] 0709 13:01:00.110894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.177362 (+ 66468us) service_pool.cc:224] Handling call
0709 13:01:00.177684 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:00.600056 (+422372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.180814 (+580758us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54004a6fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:01.181019 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969579776}
I20260709 13:01:01.188262 22207 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.125105 23017 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.141952 23201 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:01:01.155746 19981 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:01.175024 23202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.014s	sys 0.028s
I20260709 13:01:01.149036 23079 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.177438 22931 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:01.202749 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47086.1us
I20260709 13:01:01.108413 23198 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.183465 22744 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:01.221450 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.166365 23239 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:01.184777 23207 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:01.207331 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 82502.8us
I20260709 13:01:01.135597 23228 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.213963 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.214768 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 961) took 1109 ms. Trace:
W20260709 13:01:01.204372 23202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.014s	sys 0.029s
W20260709 13:01:01.227593 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 53294.1us
I20260709 13:01:01.249540 20086 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.168578 22693 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.254041 22378 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.215337 22931 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:01.257658 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 956) took 1596 ms. Trace:
W20260709 13:01:01.257846 23230 log.cc:927] Time spent T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:01:01.279654 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.280642 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:01.281191 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.281838 20085 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:01.258267 23079 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.282426 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.282724 23079 consensus_queue.cc:1048] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:01.259812 23202 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:01.212627 23042 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:01.284411 23042 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.284283 20013 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:01.284885 23042 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.260010 22971 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.285002 23202 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:01.288369 23229 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:01:01.288559 20165 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.289891 23248 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.290567 23248 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.263443 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:01.293874 23243 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:01:01.265130 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.294759 23062 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:01.294937 20082 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:01.295292 22729 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:01.295852 22729 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.296269 22729 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:01.296782 20160 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:01.297370 23228 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:01.265360 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.299070 20087 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:01:01.265898 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:01.265345 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 155655us
I20260709 13:01:01.198585 23020 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.301020 23215 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.301192 20165 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:01.301400 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.301863 23020 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:01.301592 23215 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.302235 23237 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:01.302795 23237 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.304316 23215 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.304873 23215 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:01.305330 23215 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.310623 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.201452 23150 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.314404 23164 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.315302 23164 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.214540 23076 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.338315 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:01.270125 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56396.5us
I20260709 13:01:01.247772 20852 rpcz_store.cc:276] 0709 13:01:00.105689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.105848 (+   159us) service_pool.cc:224] Handling call
0709 13:01:00.107829 (+  1981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:00.108092 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:00.108412 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.212277 (+1103865us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.214248 (+  1971us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:01.214313 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:01.214368 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.214561 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.214712 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57151,"prepare.queue_time_us":137094,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":94,"raft.queue_time_us":23255,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":37448,"thread_start_us":1047498,"threads_started":3,"wal-append.queue_time_us":658325}
I20260709 13:01:01.274269 20856 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:01.274727 20855 rpcz_store.cc:276] 0709 13:00:59.660768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.935309 (+274541us) service_pool.cc:224] Handling call
0709 13:01:00.532537 (+597228us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:00.537732 (+  5195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.257339 (+719607us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b5400004bd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:01.257577 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262424960}
I20260709 13:01:01.271446 22755 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.294857 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:01.345197 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 50435.7us
W20260709 13:01:01.275744 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.271790 22600 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:01:01.251894 23142 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.278427 20088 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:01.299512 23218 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:01:01.324662 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.269042 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.326151 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.327692 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.367882 20159 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:01.223421 22890 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.005s
I20260709 13:01:01.340010 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.346156 20851 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:01.346395 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.221475 22744 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:01.361078 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.377816 20017 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:01.363456 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:01.360040 20056 connection.cc:256] RPC call timeout handler was delayed by -1.5718778529999327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
I20260709 13:01:01.369007 20010 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:01.378577 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.371042 20008 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:01.373013 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:01.395987 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.415457 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:01.375468 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 49460.1us
I20260709 13:01:01.364559 23066 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.417083 23194 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.417359 23066 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.417599 23194 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:01.417786 23066 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.418013 23194 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.418349 23165 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:01.419281 23165 consensus_queue.cc:1243] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:01.419866 23165 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:01.416553 20161 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:01.369771 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:01.345585 23233 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:01:01.382692 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.383996 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.385543 20010 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:01.385993 23238 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.407956 23215 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:01.417877 20129 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:01.394110 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 950) took 1282 ms. Trace:
I20260709 13:01:01.349920 23239 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.412189 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1274) took 1744 ms. Trace:
I20260709 13:01:01.375185 22207 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.423050 20013 raft_consensus.cc:2410] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:01.427831 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.409540 23198 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:01.432396 20160 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:01.429724 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:01.426671 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.429133 20012 rpcz_store.cc:276] 0709 13:01:00.111567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.658733 (+547166us) service_pool.cc:224] Handling call
0709 13:01:00.659213 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:00.704697 (+ 45484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:00.704742 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:00.705325 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.135380 (+430055us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:01.135422 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.136886 (+  1464us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.393878 (+256992us) raft_consensus.cc:1685] finished
0709 13:01:01.393906 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.394046 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90355,"raft.queue_time_us":46288,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":29382,"thread_start_us":429714,"threads_started":1,"wal-append.queue_time_us":660896}
I20260709 13:01:01.438588 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.443688 20744 leader_election.cc:400] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10.
I20260709 13:01:01.443511 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:01.445148 20744 leader_election.cc:400] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9.
I20260709 13:01:01.449232 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:01.449930 20013 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.450732 22207 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:01.360621 23207 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:01:01.431939 20084 rpcz_store.cc:276] 0709 13:00:59.667626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.813867 (+146241us) service_pool.cc:224] Handling call
0709 13:01:00.480050 (+666183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:00.480301 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:00.534177 (+ 53876us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.123231 (+589054us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.231325 (+108094us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:01.231383 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:01.231437 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.231512 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.411859 (+180347us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400a5e950. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:01.412123 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173812,"prepare.queue_time_us":50446,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":6890,"raft.queue_time_us":157736,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":22053,"spinlock_wait_cycles":1481566976,"thread_start_us":564585,"threads_started":1,"wal-append.queue_time_us":757532}
I20260709 13:01:01.400138 22971 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.403832 23146 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.468258 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.426088 23165 raft_consensus.cc:3055] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:01.469810 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.470548 20858 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
W20260709 13:01:01.407125 22967 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.006s
I20260709 13:01:01.417086 22884 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.407769 23254 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.407256 23151 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:01.464766 20162 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.466828 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.453202 23231 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:01.509799 20017 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:01.440254 23153 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.400274 23017 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:01.395037 23248 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:01.426833 23238 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:01.474082 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.514436 20163 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:01.467970 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.471607 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.461197 23094 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:01:01.440460 23235 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:01.476703 22207 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.439368 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:01.497507 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.431453 23240 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:01.382627 22890 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.452638 23062 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.508149 20015 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:01.455790 22877 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:01.474771 23165 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:01.511349 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.515831 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.492072 22835 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.542172 20163 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 6.
I20260709 13:01:01.473502 23199 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:01.530812 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.527383 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 302146us
I20260709 13:01:01.545724 23202 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:01.522267 22207 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.510485 23103 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:01.544600 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 82051.6us
I20260709 13:01:01.522207 23238 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.594751 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.595324 20017 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:01.544974 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:01.407588 22744 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.509420 23151 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:01.529744 20013 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:01.473057 23201 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:01:01.473233 22762 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:01:01.492398 22949 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:01:01.530701 23235 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.589519 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 1008) took 2141 ms. Trace:
I20260709 13:01:01.508412 22967 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.608803 22207 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.471076 23239 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:01:01.591683 20157 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:01.594048 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.594285 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.543833 22835 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:01.599558 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 54650us
I20260709 13:01:01.540987 22877 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:01.640360 22877 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.602808 22971 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:01.643010 22971 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:01.643313 23250 log.cc:927] Time spent T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:01:01.643534 22971 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.603037 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.608942 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:01.513753 23254 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.648495 22951 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:01.650398 23254 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:01.634963 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.651292 20013 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:01.652243 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:01.652899 20013 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:01.637305 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.638309 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.638520 20091 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:01.655112 23122 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.635596 23165 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.655462 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.656306 23165 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.639119 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.639276 20085 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.648327 20009 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:01.660697 20017 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:01.661825 20009 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.647461 23066 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.663880 23066 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.665346 22967 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:01.666419 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.668087 22835 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.667639 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:01.617656 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.673333 20082 raft_consensus.cc:2393] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:01.607347 20156 rpcz_store.cc:276] 0709 13:00:59.410556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.410667 (+   111us) service_pool.cc:224] Handling call
0709 13:00:59.410999 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:59.411349 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.411375 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:59.421878 (+ 10503us) log.cc:844] Serialized 52 byte log entry
0709 13:01:00.502145 (+1080267us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b180029cb60. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:01:00.803949 (+301804us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:00.804028 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.804157 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.536160 (+732003us) raft_consensus.cc:1685] finished
0709 13:01:01.536182 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.551628 (+ 15446us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1425,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6427,"spinlock_wait_cycles":3537664,"thread_start_us":301467,"threads_started":1,"wal-append.queue_time_us":1649729}
I20260709 13:01:01.674181 20084 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:01.596416 23202 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:01.675699 23202 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:01.677229 23263 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:01.677333 23202 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.600780 23103 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.680044 22729 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.680930 20087 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:01.681351 20088 raft_consensus.cc:2368] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:01.683072 22729 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.684202 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:01.685483 20130 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:01.590309 20850 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:01.686420 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.693329 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:01.694110 20009 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:01.708788 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 261804us
I20260709 13:01:01.708235 22207 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:01.709780 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.710245 22207 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:01.707839 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.712314 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:01.713300 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:01.713692 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.713917 20163 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.718109 20090 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:01.610263 23235 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:01.727208 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 133025us
I20260709 13:01:01.729235 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:01.719682 22207 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.669885 23042 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.636686 23201 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:01.708081 22755 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.746299 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.746404 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:01.745123 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.721565 23198 consensus_queue.cc:1059] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:01:01.745149 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.764196 20083 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.610296 23062 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.716647 23215 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.728752 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:01.715257 23122 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:01.721427 23017 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:01.748242 20091 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:01.738785 23207 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:01.735570 23248 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:01.717468 23240 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:01:01.745311 22890 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:01.759001 23124 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.808564 23241 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.809208 23241 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.810487 22877 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.786721 23122 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:01.811326 23241 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:01.811612 23122 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.812285 23248 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:01.735812 22931 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.813412 23194 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.814700 20009 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.812772 22914 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.767519 23201 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:01.746912 22951 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:01:01.648350 23151 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.747135 23254 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:01.825250 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:01.790169 23017 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.747077 23231 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:01:01.805120 23146 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:01.815160 23124 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.762193 22755 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.796370 22884 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:01.843564 20857 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:01.792093 23062 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:01.872340 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.834841 23103 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:01.847470 23017 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:01.849491 23124 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.852015 23151 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:01.889205 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 144149us
I20260709 13:01:01.890079 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.836854 22914 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:01.903359 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:01.917214 20857 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:01.849741 23146 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.887884 22884 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:01.890658 23005 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.839s	user 0.012s	sys 0.007s
I20260709 13:01:01.904403 23215 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.894814 23165 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.927423 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:01.930015 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 184196us
I20260709 13:01:01.922709 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:01.936362 20013 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:01.920385 23231 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.934041 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:01.938301 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:01.939940 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:01.924224 22877 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.929515 23005 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:01.961691 22207 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:01.965749 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.966755 20853 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:01.968009 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.968824 20853 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:01.969941 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:01.973093 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 969) took 1241 ms. Trace:
I20260709 13:01:01.973747 20850 rpcz_store.cc:276] 0709 13:01:00.731402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.731842 (+   440us) service_pool.cc:224] Handling call
0709 13:01:00.748840 (+ 16998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.686831 (+937991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.972739 (+285908us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54000091d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:01.973007 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1853064448}
I20260709 13:01:01.975605 20850 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.941172 23231 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.982563 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:01.961988 23265 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:01:01.959659 23198 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:01.989368 20163 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:01.942024 22951 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:01.936817 23165 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:01.957756 23042 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:01.930419 23215 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:01.943771 23240 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:01:01.921944 23248 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:02.004083 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 88184.1us
W20260709 13:01:01.994925 23260 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:01:02.004451 20857 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
W20260709 13:01:02.007190 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 73349.8us
W20260709 13:01:02.005666 23271 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:01:02.032704 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88095.1us
I20260709 13:01:02.015990 20850 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:02.036798 23287 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.037756 23240 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:02.037577 23287 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.038280 23240 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:02.038929 23062 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:02.039256 23240 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.040170 22914 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.040689 20857 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:02.042780 23240 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.015791 22931 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.014154 23198 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:02.044504 23198 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:02.045910 23278 log.cc:927] Time spent T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:01:02.047823 23268 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 13:01:02.048672 23267 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:01:02.049968 23279 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:01:02.050140 22835 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.008636 23285 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.013445 22877 raft_consensus.cc:2764] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:02.006135 23231 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.055810 23238 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.075990 23238 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.078691 23124 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:02.004346 23194 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.080533 22835 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:02.083801 20013 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:02.014958 23254 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.086573 20858 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:02.119042 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.125056 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.126075 20016 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:02.127214 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.128334 20016 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
W20260709 13:01:02.032650 23262 log.cc:927] Time spent T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.192s	user 0.001s	sys 0.001s
I20260709 13:01:02.087409 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:02.130968 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 196329us
I20260709 13:01:02.033210 22755 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.136396 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:02.066198 23165 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.139873 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:01:02.139371 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 52078.5us
I20260709 13:01:02.123905 23088 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.068984 23215 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.144445 20743 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:02.144964 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.148114 20163 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:02.149057 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.070001 23146 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:02.152709 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:02.137827 23287 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.156606 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:02.140264 23254 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:02.158455 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.151285 23194 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:02.163767 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:02.136211 22971 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.065713 23235 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.166967 23293 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.171103 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:02.136701 23198 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.175573 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.176124 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:02.176990 20090 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:02.157090 23215 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:02.154637 23292 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:02.187196 23165 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:02.187569 20009 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:01:02.153399 23122 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.161266 23287 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:02.151752 23088 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.187960 23299 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:02.209978 22378 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.210310 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.226370 20013 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:02.211936 23252 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.951s	user 0.003s	sys 0.003s
I20260709 13:01:02.191071 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.229526 20015 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:02.194203 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:02.175563 23254 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:02.230724 20163 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:02.222414 22967 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:02.233148 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:02.217231 23248 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:02.228925 23293 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:01:02.236804 23293 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:02.237190 23088 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.223076 23165 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.238313 23165 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:02.237987 22967 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.238389 23248 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.222268 23292 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.240437 23292 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.240823 23182 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.237231 23293 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:02.241523 23182 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.242169 23293 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.244767 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:02.245244 23122 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.245997 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:02.246127 22744 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:02.246263 23254 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.247430 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:02.217443 23215 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:02.248924 23122 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:02.248975 23124 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.249231 23215 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.251751 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.252481 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.255385 20158 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:02.265179 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.249914 23122 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.256855 22971 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:02.272235 20164 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:02.257421 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.258170 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.258427 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.265354 20130 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:02.274101 20156 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:02.269435 20162 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:02.274209 20017 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.274610 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:02.270422 20083 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:02.275713 22378 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:02.276376 22378 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.276815 22378 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:02.272164 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 48970us
I20260709 13:01:02.273308 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:02.278635 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.283535 20090 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 5.
I20260709 13:01:02.283684 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.285755 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.290351 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.287321 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:02.288173 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:02.291996 20851 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:02.291181 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:02.290388 20162 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:02.291035 20156 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:02.296672 20164 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:02.297200 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:02.229684 23232 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:01:02.300110 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:02.304453 20743 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.319264 20744 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:02.301926 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:02.321866 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.324200 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.298272 20083 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:02.282819 23088 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:02.329424 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:02.330135 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:02.338241 20857 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.283804 20015 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:02.339589 20017 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:02.340099 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.340996 20015 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:02.349161 20013 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:02.350721 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.284698 23194 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.351536 20013 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.356408 20850 raft_consensus.cc:2410] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:02.353209 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.363425 22378 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.336949 23199 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:02.368227 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.368898 23240 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.354796 23088 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.359863 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.369911 22890 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:02.371973 23238 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:01:02.373800 23202 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.002s
I20260709 13:01:02.352308 23281 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:01:02.376299 20857 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:02.280472 23287 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:01:02.401182 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.402504 22378 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.404134 20013 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:02.406338 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.409487 20015 raft_consensus.cc:2368] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10.
I20260709 13:01:02.284917 22877 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:01:02.409755 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.422933 20013 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:02.425800 22378 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.294836 22755 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:01:02.427250 22378 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.424168 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.432554 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.433359 20013 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:02.434464 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.435343 20013 raft_consensus.cc:2368] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12.
I20260709 13:01:02.436699 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.330682 22971 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:01:02.445384 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.447818 20015 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:02.404963 23088 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.403596 22890 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.496774 22890 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.492056 23199 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.499038 23199 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.410629 23285 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.430299 23238 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:02.483528 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.490223 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:02.490871 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 59057.7us
I20260709 13:01:02.510823 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:02.494817 22378 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.495111 22207 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.407743 23062 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:02.429168 23202 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.502861 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.489006 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.510960 20017 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:02.525149 20013 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:02.511740 22207 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.513849 20164 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:02.528506 20015 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:02.513720 23062 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.538549 23151 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:02.554677 23062 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.555256 20015 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:02.555189 23151 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:02.556145 22207 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:02.536342 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.557583 23151 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.558550 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.559428 20013 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:02.513494 23281 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:02.560230 23281 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.528831 23305 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:01:02.562628 23281 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:02.565667 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54923.9us
I20260709 13:01:02.438917 23292 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.574389 23287 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:02.575297 23287 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:02.576385 23287 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.578009 23287 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:02.583009 23301 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:01:02.596930 23302 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:01:02.601099 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.530932 23202 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.612505 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:02.613655 20857 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.624838 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:02.605160 23017 consensus_meta.cc:287] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.002s
I20260709 13:01:02.642102 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:02.644054 20851 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:02.600310 23285 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.620033 23151 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:02.626670 23235 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:02.674432 23306 log.cc:927] Time spent T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:01:02.650420 23199 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.628371 23057 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:01:02.669018 23151 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.670634 23122 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:02.648712 23017 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:02.697597 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 146235us
I20260709 13:01:02.635251 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:02.707757 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 72610.4us
I20260709 13:01:02.714784 20857 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:02.693282 23151 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:02.721474 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.724375 20161 raft_consensus.cc:2368] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13.
I20260709 13:01:02.725040 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:02.730262 23247 log.cc:927] Time spent T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.472s	user 0.000s	sys 0.004s
I20260709 13:01:01.549175 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:02.717650 23057 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.761379 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 949) took 2650 ms. Trace:
W20260709 13:01:02.762951 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.761965 20011 rpcz_store.cc:276] 0709 13:01:00.111335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.637746 (+526411us) service_pool.cc:224] Handling call
0709 13:01:00.638203 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:00.694170 (+ 55967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:00.694221 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:00.694903 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.121397 (+426494us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:01.121444 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.121528 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.729464 (+1607936us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:02.736778 (+  7314us) raft_consensus.cc:1685] finished
0709 13:01:02.736810 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.761280 (+ 24470us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9982,"raft.queue_time_us":55524,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":36155648,"thread_start_us":292386,"threads_started":1,"wal-append.queue_time_us":513193}
I20260709 13:01:02.679270 23238 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:02.767513 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:02.769287 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:02.770777 22207 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:02.759039 23254 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.085s	sys 0.163s
I20260709 13:01:02.777238 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:02.772768 23254 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.086s	sys 0.164s
I20260709 13:01:02.785482 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.786036 20011 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:02.772943 23235 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:02.772699 23285 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:02.782498 23165 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.544s	user 0.005s	sys 0.007s
I20260709 13:01:02.792232 20850 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:01:02.790302 23285 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.804716 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 1028) took 1422 ms. Trace:
I20260709 13:01:02.805387 23062 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.807785 20160 rpcz_store.cc:276] 0709 13:01:01.382312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.382446 (+   134us) service_pool.cc:224] Handling call
0709 13:01:02.804541 (+1422095us) spinlock_profiling.cc:243] Waited 9.26 ms on lock 0x7b54004d7954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:02.804651 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21290752}
I20260709 13:01:02.808583 23190 raft_consensus.cc:3055] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:02.809013 23190 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:02.809556 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.793880 23202 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:02.817698 23297 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.588s	user 0.001s	sys 0.000s
I20260709 13:01:02.819809 23292 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:02.822160 23300 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.290s	user 0.000s	sys 0.002s
I20260709 13:01:02.795336 23165 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.824018 23017 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.826152 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.826783 20160 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.827023 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:02.827610 20850 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:01:02.828528 23304 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:01:02.830345 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 970) took 1119 ms. Trace:
I20260709 13:01:02.830364 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1113) took 1567 ms. Trace:
I20260709 13:01:02.830861 20016 rpcz_store.cc:276] 0709 13:01:01.711396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.132532 (+421136us) service_pool.cc:224] Handling call
0709 13:01:02.132995 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.133752 (+   757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:02.133816 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:02.136622 (+  2806us) log.cc:844] Serialized 26 byte log entry
0709 13:01:02.285009 (+148387us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:02.285049 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.285114 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.821620 (+536506us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:02.823808 (+  2188us) raft_consensus.cc:1685] finished
0709 13:01:02.823828 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.830276 (+  6448us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2606,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2637,"spinlock_wait_cycles":8983424,"thread_start_us":130335,"threads_started":1,"wal-append.queue_time_us":291154}
I20260709 13:01:02.830993 20086 rpcz_store.cc:276] 0709 13:01:01.263439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.264079 (+   640us) service_pool.cc:224] Handling call
0709 13:01:01.264431 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:01.323329 (+ 58898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.323375 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:01.346272 (+ 22897us) log.cc:844] Serialized 52 byte log entry
0709 13:01:02.239851 (+893579us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18000e09e0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:01:02.420009 (+180158us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:02.492941 (+ 72932us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.493606 (+   665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.493714 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.830002 (+336288us) raft_consensus.cc:1685] finished
0709 13:01:02.830115 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.830294 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80627,"prepare.queue_time_us":70993,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3655,"raft.queue_time_us":73172,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":41682,"spinlock_wait_cycles":54399744,"thread_start_us":179832,"threads_started":1,"wal-append.queue_time_us":1248670}
I20260709 13:01:02.833259 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:02.834445 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.835011 20016 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.836938 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 524) took 1221 ms. Trace:
I20260709 13:01:02.837182 23237 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.837366 20087 rpcz_store.cc:276] 0709 13:01:01.615593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.615696 (+   103us) service_pool.cc:224] Handling call
0709 13:01:02.836732 (+1221036us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b540060c2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:02.836856 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47422080}
I20260709 13:01:02.838965 20157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 1036) took 1342 ms. Trace:
W20260709 13:01:02.841845 23273 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.917s	user 0.000s	sys 0.002s
W20260709 13:01:02.844086 23313 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:01:02.799736 23235 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:01:02.845284 20088 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 522) took 1230 ms (client timeout 1626 ms). Trace:
I20260709 13:01:02.846093 23235 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:02.845719 20088 rpcz_store.cc:269] 0709 13:01:01.614752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.614890 (+   138us) service_pool.cc:224] Handling call
0709 13:01:02.845082 (+1230192us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b5400664ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:02.845226 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61155584}
I20260709 13:01:02.853193 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 965) took 2890 ms. Trace:
I20260709 13:01:02.853621 20856 rpcz_store.cc:276] 0709 13:00:59.962595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.009241 (+ 46646us) service_pool.cc:224] Handling call
0709 13:01:00.009531 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.345133 (+1335602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.852945 (+1507812us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54006eaad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:02.853135 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2907926400}
I20260709 13:01:02.854519 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 960) took 1400 ms. Trace:
I20260709 13:01:02.854872 20858 rpcz_store.cc:276] 0709 13:01:01.454153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.535854 (+ 81701us) service_pool.cc:224] Handling call
0709 13:01:01.536696 (+   842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:02.123004 (+586308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.843731 (+720727us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54000046d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:02.851895 (+  8164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255655808}
I20260709 13:01:02.839433 20157 rpcz_store.cc:276] 0709 13:01:01.496764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.497152 (+   388us) service_pool.cc:224] Handling call
0709 13:01:02.838812 (+1341660us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54004d6054. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:02.838903 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160198528}
I20260709 13:01:02.858385 23153 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:02.806370 23285 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:02.858984 23153 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.861159 23153 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:02.857852 23190 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:02.862407 23190 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.864192 20160 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:02.828408 23240 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:02.867266 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:02.865754 23231 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.742s	user 0.013s	sys 0.002s
I20260709 13:01:02.868099 23231 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:02.868620 23215 consensus_meta.cc:287] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.003s
I20260709 13:01:02.870262 23231 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:02.870445 20017 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:02.865602 20744 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.873344 20160 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:01:02.876398 20016 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:02.876771 20017 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:02.877209 20013 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 10. Ops: []
I20260709 13:01:02.828044 23062 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:02.878575 20016 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:02.881310 20016 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:02.879500 22834 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:01:02.878399 23269 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:02.883586 20851 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:02.883807 20017 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:02.884546 20850 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:02.920886 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:02.885629 23150 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:02.886771 23094 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:02.922770 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:02.889312 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 323704us
W20260709 13:01:02.890575 22744 consensus_meta.cc:287] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.003s
I20260709 13:01:02.895460 22729 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:02.897715 23124 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:02.923640 22834 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:02.923707 23094 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:02.928869 20010 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:02.932286 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:02.901919 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.937096 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.904675 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.940874 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:02.904911 23165 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:02.869074 23215 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:02.907291 23264 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 1.100s	user 0.001s	sys 0.000s
W20260709 13:01:02.908201 23249 log.cc:927] Time spent T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 1.651s	user 0.004s	sys 0.001s
I20260709 13:01:02.909093 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 967) took 1237 ms. Trace:
I20260709 13:01:02.911536 22967 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:02.923028 20011 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
W20260709 13:01:02.890718 23266 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.996s	user 0.000s	sys 0.002s
I20260709 13:01:02.960415 20159 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:02.923961 20856 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:02.925767 23269 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:02.903734 23164 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:02.919443 23062 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:02.915469 23151 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.929504 22834 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:02.952816 23228 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.952766 20009 rpcz_store.cc:276] 0709 13:01:01.672063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.830756 (+158693us) service_pool.cc:224] Handling call
0709 13:01:01.845705 (+ 14949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.222793 (+377088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.908818 (+686025us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400568454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:02.909032 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785604352}
I20260709 13:01:02.950309 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:02.946856 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.957104 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.978673 20163 raft_consensus.cc:2410] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:02.959537 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:02.979651 20850 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:02.980737 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.982292 20157 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:02.924739 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.925987 22729 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:02.963418 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:02.945288 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:02.983135 20743 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.955724 22744 consensus_queue.cc:1059] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:01:02.929452 22600 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:01:02.969295 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:02.946360 23146 raft_consensus.cc:2764] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:02.943369 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:02.973273 20858 raft_consensus.cc:2393] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:03.006718 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:03.007560 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 959) took 1673 ms. Trace:
I20260709 13:01:03.007958 20014 rpcz_store.cc:276] 0709 13:01:01.334209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.334694 (+   485us) service_pool.cc:224] Handling call
0709 13:01:01.335147 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.335556 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.335584 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.336057 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.526788 (+190731us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:01.845492 (+318704us) log.cc:844] Serialized 14 byte log entry
0709 13:01:01.846409 (+   917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.846480 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.906714 (+1060234us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:03.007293 (+100579us) raft_consensus.cc:1685] finished
0709 13:01:03.007375 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.007502 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2355,"prepare.queue_time_us":421992,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":36,"raft.queue_time_us":8729,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":29823,"spinlock_wait_cycles":6531456,"thread_start_us":508719,"threads_started":2,"wal-append.queue_time_us":422155}
I20260709 13:01:02.973428 20156 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:02.973549 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:03.010028 20857 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.987151 23199 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:02.927750 23124 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:02.989120 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 958) took 1667 ms. Trace:
I20260709 13:01:02.990819 23215 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:02.927214 22890 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:02.930897 23285 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.997350 22693 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.004077 20852 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:03.012017 20744 leader_election.cc:400] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13.
I20260709 13:01:03.028393 20744 leader_election.cc:403] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:03.012744 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 69483.3us
W20260709 13:01:03.013008 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43734.4us
I20260709 13:01:02.971351 23231 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:03.014117 23197 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.015763 22949 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.930742 23094 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:02.985217 23239 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:03.036739 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:03.020674 20850 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:03.016778 20859 rpcz_store.cc:276] 0709 13:01:01.321407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.321571 (+   164us) service_pool.cc:224] Handling call
0709 13:01:01.321964 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.322448 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.322489 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.344618 (+ 22129us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.726173 (+381555us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:01.726237 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.726334 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:02.890116 (+1163782us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:02.988836 (+ 98720us) raft_consensus.cc:1685] finished
0709 13:01:02.988860 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:02.989059 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14686,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6980,"spinlock_wait_cycles":8245120,"thread_start_us":325331,"threads_started":1,"wal-append.queue_time_us":469177}
I20260709 13:01:03.018422 22744 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:03.026824 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:02.971112 23062 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.017510 23199 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.039001 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:03.019174 22729 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.040329 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:03.113204 20850 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:03.113615 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:03.048864 20857 raft_consensus.cc:2410] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:03.056342 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:03.058154 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:03.027657 23088 consensus_meta.cc:287] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.003s
I20260709 13:01:03.149592 20157 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:03.150424 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 94195.3us
I20260709 13:01:03.151430 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:03.131152 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 93418.4us
I20260709 13:01:03.154836 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:03.026677 23248 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:03.022931 22755 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:03.157830 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:03.157897 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:01:03.152971 23264 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:01:03.178256 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:03.178800 20130 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:03.180289 20157 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:03.181008 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:03.181046 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:03.181764 20012 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:03.182664 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:03.183543 20017 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:03.159492 20850 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:03.111747 23240 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13.
I20260709 13:01:03.153095 23153 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:03.053400 23197 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.203575 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:03.169729 20852 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:03.149326 22949 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:01:03.164623 23076 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:03.173310 22951 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:03.173867 23232 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.120476 23199 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:03.033636 23269 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.036608 23203 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 13:01:03.148535 22729 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.053242 23239 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.190567 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:03.190915 20856 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:03.189818 23165 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:03.153614 23151 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:03.060057 22744 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.158740 20744 leader_election.cc:400] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10.
W20260709 13:01:03.149098 22606 consensus_queue.cc:1175] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "48da76e971b84219a6ba80d1ba83b7eb" responder_term: 9 status { last_received { term: 8 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:03.053763 23164 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.057924 23231 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.207345 20011 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:03.167582 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:03.212066 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.153189 23124 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:03.253809 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:03.218928 20017 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:03.184931 23228 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:03.257599 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 90095.4us
I20260709 13:01:03.258340 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:03.154075 23088 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:03.235147 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 44697.3us
I20260709 13:01:03.060065 22834 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.240724 22949 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:03.242782 20744 leader_election.cc:403] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:01:03.295462 20089 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 1285) took 2553 ms (client timeout 1841 ms). Trace:
W20260709 13:01:03.296190 20089 rpcz_store.cc:269] 0709 13:01:00.730873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.731555 (+   682us) service_pool.cc:224] Handling call
0709 13:01:03.284288 (+2552733us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b540078f854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:03.284441 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811122432}
I20260709 13:01:03.230605 23076 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:03.245551 22835 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:03.245978 22951 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:03.226823 23232 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.236327 23239 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:03.236435 23165 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.242730 23017 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:03.274144 23164 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.276676 23254 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:03.277114 23293 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.278465 20857 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:03.279011 23228 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:03.280119 23285 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.281359 23231 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:03.243321 23317 log.cc:927] Time spent T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
W20260709 13:01:03.240811 23324 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:01:03.184357 22755 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.248142 22744 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:03.324709 23232 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:03.237990 23325 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:03.328291 23325 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.329850 23182 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:03.330829 23325 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:03.294231 23323 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:03.305859 23165 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.310140 23017 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:03.298764 22835 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:03.332747 22835 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.333441 22967 raft_consensus.cc:3055] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:03.334048 22967 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:03.334216 22835 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.308952 22834 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.339900 23088 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.340930 22890 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:03.342553 23197 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.303716 23151 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10.
I20260709 13:01:03.343626 23076 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:03.344503 23207 raft_consensus.cc:3055] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:03.344892 23207 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:03.345634 23207 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.346747 23207 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.347841 22951 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.349169 23124 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.304534 23150 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:01:03.335160 22967 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.352491 22967 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:03.353093 23094 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.353359 23323 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:03.354382 23164 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.336138 23017 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:03.309835 23254 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.354298 23323 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.300652 22949 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.364542 23194 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:03.380034 23228 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:03.512408 23194 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:03.516501 23293 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:03.517993 23293 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:03.438160 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.522629 23293 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.524741 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:03.301268 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 142677us
W20260709 13:01:03.528564 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.531780 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 981) took 2176 ms. Trace:
I20260709 13:01:03.532574 20854 rpcz_store.cc:276] 0709 13:01:01.355249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.376361 (+ 21112us) service_pool.cc:224] Handling call
0709 13:01:01.376746 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:01.460146 (+ 83400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.460191 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:01.493032 (+ 32841us) log.cc:844] Serialized 26 byte log entry
0709 13:01:01.887876 (+394844us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:01.887948 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.530459 (+1642511us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b5400005fd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:03.530587 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.530728 (+   141us) raft_consensus.cc:1685] finished
0709 13:01:03.530745 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.531500 (+   755us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5842,"raft.queue_time_us":6994,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":91,"spinlock_wait_cycles":177335552,"thread_start_us":333625,"threads_started":1,"wal-append.queue_time_us":406770,"wal-append.run_cpu_time_us":2293,"wal-append.run_wall_time_us":1094456}
I20260709 13:01:03.437980 20158 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
W20260709 13:01:03.388203 23320 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.155s	user 0.003s	sys 0.000s
I20260709 13:01:03.360320 23254 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:03.514851 23194 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:03.542865 23194 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.296998 20130 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:03.500911 23062 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:03.612067 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.624083 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1138) took 1351 ms. Trace:
W20260709 13:01:03.545202 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 248136us
I20260709 13:01:03.630848 20158 rpcz_store.cc:276] 0709 13:01:02.264274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.264588 (+   314us) service_pool.cc:224] Handling call
0709 13:01:03.539815 (+1275227us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400586e54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:03.615553 (+ 75738us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2691343616}
W20260709 13:01:03.645977 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.647322 23164 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:03.648412 23164 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:03.650436 23332 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.650609 23164 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.657276 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 652) took 1159 ms. Trace:
I20260709 13:01:03.653592 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 965) took 2279 ms. Trace:
W20260709 13:01:03.652323 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 392858us
I20260709 13:01:03.688231 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:03.692524 20088 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:01:03.696601 23042 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.697992 23042 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.661895 23228 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:03.651764 23332 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.671478 20008 rpcz_store.cc:276] 0709 13:01:01.373983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.383567 (+  9584us) service_pool.cc:224] Handling call
0709 13:01:01.384035 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.664532 (+280497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:01.664562 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:01.664599 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:01.664631 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.652633 (+1988002us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54000fb4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:03.652783 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.653201 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10602,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":644390272}
I20260709 13:01:03.715619 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:03.716163 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:03.717581 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:03.718035 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:03.718501 20089 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:03.718869 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:03.720320 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:03.720579 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 191195us
I20260709 13:01:03.721267 20087 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:03.722016 20744 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:03.723517 20090 raft_consensus.cc:2410] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:03.723815 23151 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:03.724488 23151 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:03.725039 23151 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.722235 23062 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.728823 22890 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:03.729562 23062 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.729895 22890 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.738785 22890 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.760604 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:03.764472 22949 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:03.765962 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:03.766769 20129 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:03.768267 23293 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:03.770210 23293 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:03.772426 23088 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:03.772571 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:03.773531 23088 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:01:03.774150 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.720592 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:03.775538 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 123513us
I20260709 13:01:03.775903 23062 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:03.776144 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47093.6us
I20260709 13:01:03.787490 22744 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.792647 20858 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:03.792457 23202 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:03.793910 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 529) took 1535 ms. Trace:
W20260709 13:01:03.797437 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.796424 19835 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 8 to 9, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 9 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:03.799151 20088 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:01:03.669044 20162 rpcz_store.cc:276] 0709 13:01:02.497587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.500342 (+  2755us) service_pool.cc:224] Handling call
0709 13:01:02.500808 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.572620 (+ 71812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:02.572675 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:02.573585 (+   910us) log.cc:844] Serialized 26 byte log entry
0709 13:01:02.719382 (+145797us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:02.719454 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.647522 (+928068us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b540009b354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:03.647643 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.647788 (+   145us) raft_consensus.cc:1685] finished
0709 13:01:03.647815 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.656905 (+  9090us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39019,"raft.queue_time_us":40281,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":74333824,"thread_start_us":145313,"threads_started":1,"wal-append.queue_time_us":211077}
W20260709 13:01:03.809033 23199 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.574s	user 0.019s	sys 0.011s
I20260709 13:01:03.811985 23232 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:03.813139 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 976) took 1184 ms. Trace:
I20260709 13:01:03.817140 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 650) took 1521 ms. Trace:
I20260709 13:01:03.819018 23332 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:03.741674 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 465216us
I20260709 13:01:03.820791 23276 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:03.817250 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 95480.2us
W20260709 13:01:03.824856 23299 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.302s	user 0.023s	sys 0.028s
I20260709 13:01:03.809819 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:03.739830 23042 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:03.733176 22755 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.836191 20156 rpcz_store.cc:276] 0709 13:01:02.295212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.296306 (+  1094us) service_pool.cc:224] Handling call
0709 13:01:02.296748 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:03.010786 (+714038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.816543 (+805757us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54006e4e54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:03.816925 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550835072}
I20260709 13:01:03.729952 23198 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:03.833698 20013 rpcz_store.cc:276] 0709 13:01:02.257800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.863765 (+605965us) service_pool.cc:224] Handling call
0709 13:01:02.877031 (+ 13266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:02.877562 (+   531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.793408 (+915846us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b540064fad0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:03.793651 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29647104}
W20260709 13:01:03.849989 23299 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.485s	user 0.024s	sys 0.028s
W20260709 13:01:03.850587 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 84112us
I20260709 13:01:03.850236 23202 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.853821 23202 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:03.856159 23232 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.856841 23326 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:03.858408 23326 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:03.859114 19835 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 3 to 6, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:03.859804 23326 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.864348 20158 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:03.835525 20853 rpcz_store.cc:276] 0709 13:01:02.629092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.629270 (+   178us) service_pool.cc:224] Handling call
0709 13:01:02.629733 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.630283 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:02.820697 (+190414us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b18001b6820. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 13:01:02.927528 (+106831us) log.cc:844] Serialized 14 byte log entry
0709 13:01:03.016383 (+ 88855us) log.cc:844] Serialized 14 byte log entry
0709 13:01:03.036347 (+ 19964us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:03.040687 (+  4340us) log.cc:844] Serialized 26 byte log entry
0709 13:01:03.040844 (+   157us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:03.040879 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.040948 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.387533 (+346585us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:03.781756 (+394223us) raft_consensus.cc:1685] finished
0709 13:01:03.812597 (+ 30841us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.812849 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33047,"prepare.queue_time_us":175336,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":80,"raft.queue_time_us":27888,"raft.run_cpu_time_us":462,"raft.run_wall_time_us":76003,"spinlock_wait_cycles":37779712,"thread_start_us":191297,"threads_started":2,"wal-append.queue_time_us":262154}
I20260709 13:01:03.866951 20159 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:03.843327 23199 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.868191 20084 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:03.868712 23326 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.870079 23199 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.869918 23276 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:03.871483 20859 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:03.873575 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:03.861490 23088 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:03.877818 22744 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.741518 23228 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:03.859654 23232 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:03.879431 22744 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:01:03.913198 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 59326.6us
W20260709 13:01:03.927641 23124 consensus_meta.cc:287] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.578s	user 0.015s	sys 0.015s
I20260709 13:01:03.930327 23228 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.932765 23293 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:03.953914 23293 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:03.963233 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:03.972302 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64812.2us
W20260709 13:01:03.981626 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 106227us
I20260709 13:01:03.997942 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.005568 23293 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:04.011845 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.013680 23207 consensus_meta.cc:287] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.014s
W20260709 13:01:04.016700 22834 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.676s	user 0.004s	sys 0.021s
I20260709 13:01:04.016834 20156 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:01:04.017836 23017 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.018642 23017 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.020658 20009 raft_consensus.cc:2410] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:01:04.023001 23017 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.023828 23124 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:04.024586 23124 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:04.026012 23124 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.028535 23293 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.030164 23299 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.030831 23299 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.034569 23299 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:04.036530 23248 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:04.041467 23239 consensus_meta.cc:287] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.010s
I20260709 13:01:04.041646 23241 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.042768 23239 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.043159 23241 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.040920 20129 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:04.048280 23151 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.036901 20744 leader_election.cc:400] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12.
I20260709 13:01:04.052336 20744 leader_election.cc:403] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:01:04.053975 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 107070us
I20260709 13:01:04.054661 23150 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:04.055295 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:04.056831 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.051571 23151 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.046499 19835 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 6 to 8, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 8 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:04.063107 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 70093.8us
I20260709 13:01:03.997722 22744 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.080195 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:04.085963 22693 consensus_meta.cc:287] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 1.052s	user 0.018s	sys 0.010s
I20260709 13:01:04.002727 23276 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.079650 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:04.065625 23124 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.095414 19835 catalog_manager.cc:5697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 2 to 5, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 5 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:04.065580 22834 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:04.116375 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55976.3us
I20260709 13:01:04.108520 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:04.117867 20013 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.082527 23228 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.097465 23293 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:04.080365 23198 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12.
I20260709 13:01:04.110585 23088 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.115069 23151 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:04.142621 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.124466 23207 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:04.107188 22744 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.149812 20851 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:04.122642 23299 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:04.149786 22755 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.152640 23066 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:04.153141 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 58036.1us
I20260709 13:01:04.152845 23327 consensus_queue.cc:1059] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:04.154104 23079 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.155521 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:04.150045 22967 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.644s	user 0.028s	sys 0.001s
I20260709 13:01:04.149008 19835 catalog_manager.cc:5697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:04.143268 23332 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.166602 20015 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:04.147200 23293 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:01:04.171545 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 116472us
W20260709 13:01:04.172214 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46443us
I20260709 13:01:04.134183 22834 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.170430 23079 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.181182 20010 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:04.184038 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:04.168215 23066 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.184903 20014 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:04.185458 20082 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:04.186512 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:04.186915 23323 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:04.181182 23269 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:04.189002 23269 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.189644 23232 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:04.184237 19840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32772 (request call id 151) took 20003 ms (client timeout 15000 ms). Trace:
I20260709 13:01:04.189759 23241 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.183467 23079 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.192361 20084 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:04.156438 23299 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:04.155182 23228 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.192525 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:04.161610 23151 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.210763 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.191910 23232 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:04.219303 23351 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:04.242642 23094 consensus_meta.cc:287] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.889s	user 0.017s	sys 0.013s
I20260709 13:01:04.204281 23332 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.243811 20086 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:01:04.244236 20086 pending_rounds.cc:85] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 5
I20260709 13:01:04.244479 20086 pending_rounds.cc:107] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:01:04.244741 20086 raft_consensus.cc:2889] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:04.245296 23332 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.213894 23198 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.246444 23198 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:04.247639 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 119017us
W20260709 13:01:04.248492 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 56247.1us
I20260709 13:01:04.217999 19835 catalog_manager.cc:5697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 4 to 9, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 9 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:04.255299 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:04.234827 23348 log.cc:927] Time spent T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:01:04.178354 22755 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.217944 23239 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.254954 20013 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:04.194418 23088 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:04.192338 19840 rpcz_store.cc:269] 0709 13:00:44.196782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.263104 (+ 66322us) service_pool.cc:224] Handling call
0709 13:01:04.183794 (+19920690us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:55.828401 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:55.829398 (+   997us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:55.829540 (+   142us) write_op.cc:435] Acquired schema lock
0709 13:00:55.829568 (+    28us) tablet.cc:662] Decoding operations
0709 13:00:55.831656 (+  2088us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:55.831744 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:55.831772 (+    28us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:00:56.090292 (+258520us) tablet.cc:701] Row locks acquired
0709 13:00:56.090347 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:00:56.090554 (+   207us) write_op.cc:270] Start()
0709 13:00:56.090713 (+   159us) write_op.cc:276] Timestamp: P: 1783602056090469 usec, L: 0
0709 13:00:56.090741 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:00:56.091623 (+   882us) log.cc:844] Serialized 9389 byte log entry
0709 13:00:57.644285 (+1552662us) op_driver.cc:464] REPLICATION: finished
0709 13:00:58.375384 (+731099us) write_op.cc:301] APPLY: starting
0709 13:00:58.375518 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:58.376263 (+   745us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:58.376473 (+   210us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:58.421988 (+ 45515us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:58.423670 (+  1682us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 13:00:58.423893 (+   223us) write_op.cc:312] APPLY: finished
0709 13:00:58.426669 (+  2776us) log.cc:844] Serialized 163 byte log entry
0709 13:00:58.546108 (+119439us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:58.568092 (+ 21984us) write_op.cc:454] Released schema lock
0709 13:00:58.594573 (+ 26481us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:58.711119 (+116546us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:58.711155 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":715249,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":720238,"apply.run_cpu_time_us":68497,"apply.run_wall_time_us":278894,"num_ops":24,"prepare.queue_time_us":320820,"prepare.run_cpu_time_us":27195,"prepare.run_wall_time_us":579800,"raft.queue_time_us":649532,"raft.run_cpu_time_us":45516,"raft.run_wall_time_us":656668,"replication_time_us":1521868,"thread_start_us":1356294,"threads_started":5,"wal-append.queue_time_us":471418,"wal-append.run_cpu_time_us":44400,"wal-append.run_wall_time_us":2924596}]]}
W20260709 13:01:04.260948 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 47889.2us
I20260709 13:01:04.251536 23351 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:04.283653 20084 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:04.267575 20085 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
W20260709 13:01:04.316107 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.330149 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:04.344731 22835 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.331215 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:04.255090 22744 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:04.256907 23165 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.920s	user 0.007s	sys 0.022s
W20260709 13:01:04.332013 23349 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.179s	user 0.006s	sys 0.000s
I20260709 13:01:04.329001 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:01:04.331246 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 125481us
W20260709 13:01:04.334898 23254 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.793s	user 0.020s	sys 0.000s
I20260709 13:01:04.333050 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:04.329274 23351 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.364614 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:04.365975 20084 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:01:04.338162 23194 consensus_meta.cc:287] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.783s	user 0.014s	sys 0.009s
I20260709 13:01:04.330245 19835 catalog_manager.cc:5697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 4 to 6, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.252949 23269 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:04.281471 23347 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:01:04.264217 22693 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.353142 20010 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:04.326169 20130 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.281435 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:04.268910 22834 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.271507 23151 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.270488 23228 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.283881 23094 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.357007 23150 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.278637 23198 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.272516 22755 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.367153 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.276362 22729 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:04.368041 23165 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.371345 20857 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:04.371515 23124 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.372189 20013 raft_consensus.cc:2393] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:04.373139 19835 catalog_manager.cc:5697] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 4 to 5, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 5 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:04.376605 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47851.2us
W20260709 13:01:04.387244 20851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 978) took 1428 ms (client timeout 1642 ms). Trace:
I20260709 13:01:04.387890 23088 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.382561 22744 raft_consensus.cc:2764] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:04.354416 22835 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.421836 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:04.423245 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 209208us
I20260709 13:01:04.269528 23232 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.405268 20014 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
W20260709 13:01:04.416280 20851 rpcz_store.cc:269] 0709 13:01:02.958614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.963159 (+  4545us) service_pool.cc:224] Handling call
0709 13:01:04.384589 (+1421430us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54006ea854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:04.387125 (+  2536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2449314176}
I20260709 13:01:04.390069 23202 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:04.369218 23254 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.435850 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.276340 23299 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.442325 23088 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:04.442957 20082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 536) took 2279 ms (client timeout 2429 ms). Trace:
I20260709 13:01:04.442233 19835 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 6 to 7, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 7 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.444316 20852 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:01:04.446269 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.446171 20854 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:04.438645 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.437898 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:04.449105 20082 rpcz_store.cc:269] 0709 13:01:02.163401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.163518 (+   117us) service_pool.cc:224] Handling call
0709 13:01:04.442702 (+2279184us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54005a1254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:04.442866 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4647349120}
I20260709 13:01:04.455775 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.456555 20855 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:04.415664 22729 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:04.417253 22834 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.459290 20085 raft_consensus.cc:2410] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:04.391490 22693 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.465059 20853 raft_consensus.cc:2393] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:04.466660 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.465649 20859 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 6.
I20260709 13:01:04.471930 20091 raft_consensus.cc:2368] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13.
I20260709 13:01:04.477228 20858 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:04.450531 23088 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.424228 23124 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.478446 19835 catalog_manager.cc:5697] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 3 to 5, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 5 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:04.477674 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:04.483419 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 223683us
I20260709 13:01:04.484402 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:04.485405 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 152607us
I20260709 13:01:04.482224 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.435240 23150 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.489580 20084 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:04.486588 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.490108 20852 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:04.473173 23164 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.480576 23088 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.505975 23066 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.506358 23150 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.476753 22755 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:04.485536 23207 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.452900 23293 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.489665 22834 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:04.518620 23165 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:04.519236 23124 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:04.520349 23354 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.190s	user 0.001s	sys 0.001s
I20260709 13:01:04.523332 23202 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:04.523838 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 138505us
I20260709 13:01:04.493964 23198 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.485507 23326 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:01:04.527645 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74757.1us
I20260709 13:01:04.535826 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:04.536537 20091 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:04.538683 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 61114us
I20260709 13:01:04.542963 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:04.543294 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:04.544281 20086 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.549703 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.550089 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.550786 20088 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:04.552292 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.549609 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.555372 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.555778 20082 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:04.557026 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.557888 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.558585 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.559471 20090 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:04.561771 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.562461 23165 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:04.562896 23165 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.563084 22744 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.563311 23165 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:04.563587 22744 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.564121 22744 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.538895 22755 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.569514 22755 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:04.526661 23207 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.570565 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.572063 23207 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:04.575562 22744 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.529632 23232 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.572762 20086 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
W20260709 13:01:04.572533 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47515.9us
I20260709 13:01:04.598407 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.600771 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:04.600594 22729 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:04.608721 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 69799.4us
I20260709 13:01:04.608739 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:04.609843 20159 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:04.603724 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 986) took 1090 ms. Trace:
I20260709 13:01:04.601866 23254 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:04.600790 20162 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:01:04.610606 20857 rpcz_store.cc:276] 0709 13:01:03.513863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.514221 (+   358us) service_pool.cc:224] Handling call
0709 13:01:03.514934 (+   713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.415246 (+900312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:04.603434 (+188188us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54000078d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:04.603637 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967486720}
I20260709 13:01:04.607856 23207 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.609783 22729 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:04.612033 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.600904 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.599556 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:04.615638 20091 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:04.596762 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:04.525352 23164 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.612088 23207 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.619994 23207 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:04.626077 23124 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.639753 23124 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.640231 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.641502 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.642001 23124 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.642416 20851 raft_consensus.cc:2410] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:04.642529 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.580322 23198 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.643805 23326 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:04.644554 23017 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:04.646126 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46858.1us
W20260709 13:01:04.647468 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.648275 23094 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.648881 23094 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.650297 20165 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:04.640304 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.651140 23094 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.650363 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.653108 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:04.653528 23094 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.653882 20159 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.654119 23094 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.656121 20156 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:04.657665 23094 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:04.659030 23356 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:04.660249 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.661473 22693 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.662309 20158 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:04.663631 23164 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:04.664350 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.664363 22835 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.665434 23164 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:04.665897 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.666260 23164 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.668431 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.668655 23164 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:04.669178 23165 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.669850 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.670660 20858 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:04.673731 20859 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:04.677959 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.678735 23299 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.680236 23352 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:04.680922 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.681805 20855 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:04.682652 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.683498 23198 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.683807 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:04.690752 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 90114.4us
I20260709 13:01:04.690055 19982 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.709535 20855 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:04.716939 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 248200us
I20260709 13:01:04.698589 23088 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.709437 20159 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:04.633785 22834 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:04.733819 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.757625 20130 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.759032 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.775797 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:04.688174 23207 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:04.750506 23124 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.715463 23165 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.763015 23198 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:04.791210 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 116316us
I20260709 13:01:04.767048 23228 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:04.806694 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 44811.2us
I20260709 13:01:04.785458 23124 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.801980 19981 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:04.790905 22835 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.768543 22834 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.779165 22755 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:04.813190 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.795307 23352 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.817818 20130 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.827174 20858 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:04.831732 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:04.834815 22744 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.809629 23228 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.835994 22834 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.836510 22755 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.801646 23198 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.837483 23228 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.836920 23088 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.819990 23124 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.823379 23194 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:04.846674 23164 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.850621 20162 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:04.828325 23352 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.852620 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:04.843582 22744 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:04.860152 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 115796us
W20260709 13:01:04.863162 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 83249.5us
I20260709 13:01:04.826516 22835 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.842097 23326 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.866151 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.868587 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:04.872642 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.809948 23207 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:04.864290 23323 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:04.882870 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 82125.7us
I20260709 13:01:04.883682 20162 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:04.849965 22755 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:04.920733 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 75119.3us
I20260709 13:01:04.921315 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:04.924067 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:04.928602 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 176578us
I20260709 13:01:04.881471 23239 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:01:04.932135 23323 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:04.880858 23352 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.933635 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.934543 23323 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:04.936456 23362 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:01:04.937083 23351 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:04.937276 22835 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.937726 23351 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:04.938028 20016 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:04.930287 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.939380 20165 raft_consensus.cc:2410] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:04.863250 23164 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:04.939110 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:04.953153 23124 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.885813 23198 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.956809 23088 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:04.950454 20015 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:04.957647 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:04.961319 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.965420 20858 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:04.934695 23207 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:04.924796 22693 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:04.976748 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:04.882797 23269 consensus_queue.cc:1059] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:01:04.981630 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:04.978688 20859 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:04.982935 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 129584us
I20260709 13:01:04.928268 23194 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:04.988144 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:04.964958 23088 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.995515 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:04.994688 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:04.998844 20165 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:04.940830 23326 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:04.977604 22693 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:05.012367 23378 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:01:05.016280 20161 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.996454 22744 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.019425 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.020081 20157 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:05.014832 23375 raft_consensus.cc:3055] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:04.978842 23198 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:05.023533 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 83646.2us
I20260709 13:01:05.024417 23017 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:04.996923 23088 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.027108 23124 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:05.028196 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:05.054248 20013 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:05.009475 20163 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:05.055219 20160 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:05.055451 23370 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.056031 23370 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.056303 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.045287 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.046169 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:05.048877 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59443.4us
I20260709 13:01:05.005282 23351 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:04.998530 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.056201 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.057741 20086 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.040052 22693 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.062584 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.054347 20084 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:05.058017 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:05.058574 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 297723us
I20260709 13:01:05.061106 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.066893 20016 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:05.062289 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 63870us
I20260709 13:01:05.053032 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.071587 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.071749 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:05.072428 20160 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.062608 23198 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.076313 23358 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:05.077455 23254 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.050513 23375 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:05.077704 23198 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:05.077893 23165 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:05.078336 23165 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.078701 23165 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.078613 23375 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.061496 23150 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:05.080032 23375 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.080353 23150 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:05.081296 23150 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:05.083220 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:05.072448 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.085202 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:05.085613 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.063864 23017 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.088617 22744 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.089937 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:05.091166 22755 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:05.091717 22755 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:05.069922 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.090271 23017 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.090629 23124 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:05.098572 20086 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:05.100214 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.101006 20159 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:05.105109 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:05.105407 23228 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:05.105800 20013 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.107738 20087 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:05.105134 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.110563 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.111418 20165 raft_consensus.cc:2410] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:05.112524 20156 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:05.107151 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.116401 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:05.121304 23269 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:05.063380 23381 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:05.126030 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.127583 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.081234 23326 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.149426 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.139242 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:05.140038 20012 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:05.150147 20087 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
W20260709 13:01:05.144722 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.105811 23228 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.113914 23358 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:05.168318 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.123960 23276 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.169209 20162 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:05.159891 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:05.126571 23381 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:05.180720 20859 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:05.181161 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:05.112152 23164 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:01:05.122184 23269 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.157588 23228 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.184520 20165 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:05.185505 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:05.185592 20016 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:05.186038 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.157317 23358 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:05.126353 22834 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:05.159560 23326 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.210593 20161 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:05.192458 20160 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:05.195043 20087 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:05.192559 20157 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:05.175541 23276 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.184911 23017 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:01:05.208334 20744 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.212816 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:05.216148 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.183580 23381 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.224076 23384 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:01:05.229838 23017 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:05.233687 23379 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:01:05.236634 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:05.214334 23276 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.238873 23377 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:05.239348 23377 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.239723 23377 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.243512 23207 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.244194 23207 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:05.244726 23207 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:05.270457 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.223543 20084 raft_consensus.cc:2393] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:05.275704 20016 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:05.198247 23358 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:05.291278 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 133814us
I20260709 13:01:05.292722 20013 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:05.140966 22755 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.310215 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:05.312316 20084 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.168722 23194 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:05.312573 23232 consensus_meta.cc:287] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.684s	user 0.013s	sys 0.000s
I20260709 13:01:05.294520 23124 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.282528 23370 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.247726 23326 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:05.318557 22729 consensus_meta.cc:287] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.706s	user 0.010s	sys 0.003s
W20260709 13:01:05.335186 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 135645us
I20260709 13:01:05.339190 20089 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:05.340811 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:05.343014 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 185480us
W20260709 13:01:05.349586 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:05.337666 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 176046us
I20260709 13:01:05.359092 20088 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:05.331732 23384 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:05.360170 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.422793 23232 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:05.424974 23372 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 13:01:05.425390 23389 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:01:05.426777 20010 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:05.427309 23254 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.429695 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.433040 23165 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.435274 23165 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.341435 23124 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:05.437232 23326 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:05.437642 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.438463 20157 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:05.439299 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.439534 23384 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:05.441833 23384 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:05.443931 22693 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.451436 20084 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:05.418982 19981 leader_election.cc:400] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13.
I20260709 13:01:05.452308 19981 leader_election.cc:403] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:01:05.340471 23239 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.358778 23151 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:05.414361 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:05.415352 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46994.6us
W20260709 13:01:05.466506 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 106440us
I20260709 13:01:05.467777 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:05.468446 20157 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:05.427784 23254 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.467049 19982 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:05.445204 23392 log.cc:927] Time spent T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.102s	user 0.002s	sys 0.000s
W20260709 13:01:05.478266 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.483537 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:05.486194 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:05.473351 23326 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.495250 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.495867 20084 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:05.492098 19982 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:05.472970 23384 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:05.489405 23400 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:05.512562 20010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:05.516047 23390 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:01:05.519184 20084 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:05.472410 23239 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.472528 23151 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.479764 23088 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.494067 22834 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:05.548756 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62625.7us
I20260709 13:01:05.559648 23394 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.562413 23381 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.569350 23394 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.569808 23394 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.570784 23151 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:05.571265 23381 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:05.572170 22693 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.573016 20855 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:05.574044 23124 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:05.577299 23239 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:05.577811 23386 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
W20260709 13:01:05.580058 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 220513us
I20260709 13:01:05.582710 23228 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:05.584709 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.585361 20084 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
I20260709 13:01:05.585670 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:05.586297 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.586941 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:05.587368 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:05.587553 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.588186 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.588601 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.589335 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.589973 20083 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 7.
W20260709 13:01:05.592644 23352 consensus_meta.cc:287] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.000s
I20260709 13:01:05.586941 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.593786 20090 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.564047 23397 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:05.524324 23299 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:05.568578 23388 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:01:05.525753 23400 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.585373 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:05.637986 20852 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.589332 20084 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:05.610656 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 113167us
I20260709 13:01:05.620440 20130 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:05.633728 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:05.499082 23326 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.636255 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.643064 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:05.523902 23384 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.621958 23397 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.651779 20083 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:05.678743 23397 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:05.638443 23269 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:05.658982 23384 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:05.663061 20086 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:05.685089 23088 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:05.640477 23400 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.671638 23228 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:05.672000 23380 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:05.691619 20090 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:05.695410 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.712460 20085 raft_consensus.cc:2393] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:05.696164 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.713431 20091 raft_consensus.cc:2376] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10. Re-sending same reply.
I20260709 13:01:05.698884 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:05.712752 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:05.716215 20858 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.692266 23384 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:05.717058 23401 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:01:05.716992 23384 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:05.717850 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.719179 23323 consensus_queue.cc:1059] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:05.720167 23323 raft_consensus.cc:3055] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:05.695873 23377 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.725898 20858 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:05.595373 23352 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:05.638037 23254 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:05.731158 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 311032us
I20260709 13:01:05.729641 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:05.738209 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:05.720505 23323 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:05.719203 23384 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:05.723903 23380 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:01:05.746024 23375 consensus_meta.cc:287] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.666s	user 0.002s	sys 0.011s
I20260709 13:01:05.766196 20158 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:05.760265 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.674s (SENT) [suppressed 12 similar messages]
W20260709 13:01:05.761188 23207 consensus_meta.cc:287] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.002s
W20260709 13:01:05.765504 23276 consensus_meta.cc:287] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.527s	user 0.013s	sys 0.000s
I20260709 13:01:05.796118 20852 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:05.800141 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1148) took 1230 ms. Trace:
I20260709 13:01:05.800601 20088 rpcz_store.cc:276] 0709 13:01:04.557053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.560088 (+  3035us) service_pool.cc:224] Handling call
0709 13:01:05.787388 (+1227300us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b54006658d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:05.787540 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1809906432}
W20260709 13:01:05.805030 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 61738.8us
I20260709 13:01:05.780684 22693 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:05.806484 23326 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.818655 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:05.824942 23380 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:05.811991 23395 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:05.828423 23395 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:05.829859 23395 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.833338 23377 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:05.836196 23254 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.808794 23194 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.842658 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:05.843389 20856 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.837807 23194 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.797501 23207 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:05.821343 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106941us
I20260709 13:01:05.799547 23276 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:05.852201 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:05.860975 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:05.847151 23326 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:05.869355 20850 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:01:05.869853 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 96384.7us
I20260709 13:01:05.807128 23394 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:05.871975 23326 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:05.877473 23415 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:01:05.813726 23323 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:05.881799 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.888029 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:05.894307 22693 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:05.894449 23418 consensus_queue.cc:1059] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:05.895463 22693 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:05.896905 23276 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:05.903137 20856 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:05.863039 23352 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:05.923018 23417 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:05.932516 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:05.933537 22729 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.933707 23248 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:05.934657 23248 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.933753 23370 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.935870 23370 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:05.936607 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:05.934599 23228 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.939131 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:05.942233 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:05.943001 20011 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:05.944598 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:05.945464 20008 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:05.933976 22744 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:05.947872 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:05.935062 23393 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.938267 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:06.100095 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 153608us
I20260709 13:01:05.939489 23228 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.110015 20015 raft_consensus.cc:2368] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
W20260709 13:01:06.110725 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 174929us
I20260709 13:01:06.112259 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.113168 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.118432 20009 raft_consensus.cc:2410] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:05.938267 23370 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.091777 22834 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13.
I20260709 13:01:06.133428 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.134394 20009 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:05.935099 22729 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.194s
I20260709 13:01:05.923815 23417 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:06.112811 23419 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:06.107429 22744 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:06.150137 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.032464 23400 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.135964 23042 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.117791 23393 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:06.161675 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.161798 23194 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:06.164875 22744 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.170428 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.172591 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.143379 22693 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:01:06.173679 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.174901 20090 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:06.175099 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.172880 20009 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:06.175675 20084 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 5.
I20260709 13:01:06.177094 20014 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:06.177410 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.178210 20084 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.180867 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.181682 20085 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.182475 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.183261 20088 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:06.184041 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.160132 23326 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:06.185479 23358 consensus_meta.cc:287] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.856s	user 0.013s	sys 0.001s
I20260709 13:01:06.195222 20017 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:06.197136 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.198472 20010 raft_consensus.cc:606] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.198931 20010 raft_consensus.cc:614] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:06.199043 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.199311 20016 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:06.204663 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.199021 23358 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.185127 23248 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.205555 20086 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:06.217341 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.218070 20016 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:06.223806 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.224516 20015 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:06.225535 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.226472 23400 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.226608 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.227061 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.227764 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.229032 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.229658 20012 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:06.227900 20016 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:06.230870 20008 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:01:06.232051 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.224516 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.232879 20013 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:06.231410 20008 pending_rounds.cc:85] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 2
I20260709 13:01:06.235832 20008 pending_rounds.cc:107] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:01:06.236162 20008 raft_consensus.cc:2889] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:06.236117 20084 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:06.237597 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.242830 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.244527 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.250241 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.250864 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.220881 23395 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.236114 23269 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:06.254426 23358 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.254204 23417 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:06.264132 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:06.246723 23397 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.567s	user 0.013s	sys 0.000s
I20260709 13:01:06.270912 20008 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:06.288285 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.292924 23397 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:06.294275 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.296315 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.298323 20085 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:06.261214 23384 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:06.271489 23228 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.299265 22755 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.299779 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:06.304359 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 700001us
I20260709 13:01:06.318117 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.319110 20011 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:01:06.320062 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 106854us
I20260709 13:01:06.305465 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:06.322422 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.322914 20164 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:06.261579 23377 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:06.311846 23351 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:06.328297 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.330789 20089 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:06.282670 23395 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.337674 23395 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:06.338653 23429 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:01:06.323232 20013 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:06.324244 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.316388 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.326126 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.332965 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.345276 20086 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:06.345464 20016 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.335125 23207 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:06.339015 23431 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:06.283262 23299 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:06.269586 23269 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.268569 23194 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:01:06.345938 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:06.360373 20160 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:06.324010 23377 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.357295 20011 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:06.354911 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:06.364172 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 88337.4us
W20260709 13:01:06.373660 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 122845us
I20260709 13:01:06.375070 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.355399 23419 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:06.381611 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.356274 23094 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:06.353724 23228 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.389900 20163 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:06.361099 23269 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.401005 20016 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:06.409137 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:06.367269 23377 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:06.414093 20165 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:06.360008 23351 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:06.416714 20011 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:06.377138 23417 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:06.425447 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 68975.2us
I20260709 13:01:06.408852 23094 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:06.442871 20160 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:06.416975 23326 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:06.419015 23384 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:06.458016 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.410774 23393 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.450047 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.458484 20744 leader_election.cc:400] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:06.450977 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.459046 20744 leader_election.cc:403] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:06.459141 20160 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:06.451735 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.456796 20156 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:06.391204 23207 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:01:06.461716 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.462433 20165 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:06.463117 20163 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:06.463429 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:06.451453 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:06.464491 20159 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:06.460453 20157 raft_consensus.cc:2410] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:01:06.460592 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 85387.3us
W20260709 13:01:06.444726 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 91030.4us
I20260709 13:01:06.463562 20011 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:06.427965 23395 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:01:06.467715 20743 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.468214 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.469950 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.465981 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.470619 20156 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
W20260709 13:01:06.471517 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 96508.8us
I20260709 13:01:06.473546 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.472572 23207 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:06.419551 23228 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.470193 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:06.475654 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.489871 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:06.493026 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:06.500949 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.502807 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.505319 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.472688 23400 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.475163 23194 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:06.506611 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57018.5us
I20260709 13:01:06.472771 23430 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:06.494572 23393 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.482017 23417 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.507242 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.518952 20160 raft_consensus.cc:2368] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11.
I20260709 13:01:06.510115 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:06.504213 23432 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:01:06.470050 23434 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:01:06.507759 22744 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:06.518318 23207 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:06.510146 23326 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:06.506351 23435 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.509253 22834 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.525782 19982 leader_election.cc:400] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11.
I20260709 13:01:06.572454 19982 leader_election.cc:403] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:06.523788 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55691.6us
I20260709 13:01:06.526687 20013 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:06.528307 23418 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.525820 23207 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.576516 23207 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:06.565681 22744 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:06.578603 23435 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.578955 22744 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.512224 23351 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:06.565462 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 99545.8us
I20260709 13:01:06.580770 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:06.555241 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:06.470271 23299 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:06.527876 23326 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.523530 23384 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.573568 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.593587 20158 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:06.581962 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:06.564254 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51061.9us
I20260709 13:01:06.582541 20160 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.522626 23445 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:06.579093 23435 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.607681 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:06.611277 23419 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:06.636346 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:06.632232 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106589us
I20260709 13:01:06.638783 20160 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:06.561691 23440 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.638010 20853 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:06.590974 23370 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:06.599939 23326 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.649997 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.650808 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.611804 23393 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:06.653141 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 143097us
I20260709 13:01:06.521485 23430 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:06.599848 23434 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:06.655607 20129 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:06.630052 23445 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.658998 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:06.659826 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.631325 23432 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.650653 23370 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.665670 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.662114 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:06.576524 23418 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:06.664021 23380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.016s	sys 0.022s
I20260709 13:01:06.657086 22834 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11.
I20260709 13:01:06.675009 19981 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.656390 23395 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:06.675261 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:06.675859 20855 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.675793 23395 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.678179 23395 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.683485 23228 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.684226 23228 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:06.684582 23228 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.684937 23228 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:06.684791 23419 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.670255 23370 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:06.660678 23434 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:06.688709 23434 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.698369 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:06.666955 23440 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:06.709060 20855 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:06.709777 20164 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:06.664948 23445 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:06.714618 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:06.669742 23276 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:06.709364 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:06.673664 23380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.016s	sys 0.022s
I20260709 13:01:06.720817 20852 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:06.722771 20858 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:06.673722 23400 raft_consensus.cc:2764] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:06.661444 23393 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:06.729616 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 168695us
W20260709 13:01:06.730126 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50582.3us
I20260709 13:01:06.683493 23435 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:06.732209 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.735838 20129 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:06.734839 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 105302us
I20260709 13:01:06.708695 23377 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.724167 23276 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:06.742323 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.742980 20008 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:06.744692 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.745208 20088 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:06.746309 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.746964 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.748420 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:06.751468 20008 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:06.744112 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.727433 23431 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 13:01:06.752219 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 178755us
I20260709 13:01:06.753314 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:06.738963 23352 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:06.756798 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.756987 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.757555 20016 raft_consensus.cc:2376] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13. Re-sending same reply.
I20260709 13:01:06.754421 23446 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:01:06.749783 22744 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:06.759436 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.741027 23276 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:06.742627 23435 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:06.766119 23352 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:06.773217 23352 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:06.764247 23441 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.177s	user 0.001s	sys 0.001s
I20260709 13:01:06.776027 23450 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:06.776750 23450 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.776979 23418 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.777187 23450 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:06.778129 23430 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:06.779168 23202 raft_consensus.cc:3055] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:06.779680 23202 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:06.780532 23377 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.781152 23454 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:06.780612 23202 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:06.781613 23454 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.782058 23454 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:06.789580 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45543.1us
I20260709 13:01:06.742769 23400 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:01:06.791038 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.804035 23393 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.804476 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:06.805439 20857 raft_consensus.cc:2410] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:06.794422 23435 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:06.806068 23435 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:06.807874 20016 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:06.815608 20088 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:06.793206 23456 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.767477 22744 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.807000 23207 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:06.824820 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.832429 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.787428 23269 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.836225 23377 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:06.839092 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.840633 20854 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:06.828491 23460 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:06.837410 23202 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:06.852663 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 94757.4us
I20260709 13:01:06.857789 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.841446 23377 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:06.862942 23458 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:06.865964 23276 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:06.868340 23228 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:06.871424 23228 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:06.872778 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.873504 20088 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:06.857101 23460 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:06.876776 23207 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:06.877568 23377 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.877148 23460 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:06.880434 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.878918 23454 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.888383 23419 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:06.888404 23326 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:06.914016 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.916723 23326 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:06.917539 20015 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:06.911108 23419 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:06.920269 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.920428 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.920661 23419 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.921414 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.921592 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.922542 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.922825 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.924109 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.924284 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.925123 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.925303 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.926201 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:06.927420 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.919766 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.928820 20083 raft_consensus.cc:2410] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:06.911532 23207 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:06.923730 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:06.890426 23440 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:01:06.897356 23276 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.898629 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:06.932971 20016 raft_consensus.cc:2393] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:06.903685 23269 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.908555 23194 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.931385 22744 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.890429 20088 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:06.923714 20091 raft_consensus.cc:2376] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7. Re-sending same reply.
W20260709 13:01:06.944612 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88053.4us
I20260709 13:01:06.951144 20856 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.958020 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:06.960388 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 547) took 1027 ms. Trace:
I20260709 13:01:06.978475 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:06.978482 20014 rpcz_store.cc:276] 0709 13:01:05.933465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.101706 (+168241us) service_pool.cc:224] Handling call
0709 13:01:06.116341 (+ 14635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.177481 (+ 61140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.960137 (+782656us) spinlock_profiling.cc:243] Waited 59.4 ms on lock 0x7b54009fa6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:06.960326 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136560896}
I20260709 13:01:06.979772 20857 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:06.966017 23450 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.981860 23450 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.983718 23370 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:06.986019 23370 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.984167 23194 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.943696 23430 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:06.986539 20129 leader_election.cc:336] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.506s (SENT)
I20260709 13:01:06.998595 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.987942 23454 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:06.976887 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:06.992482 20090 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:06.992586 20082 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:06.993189 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:06.978627 20856 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:06.997852 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:07.017218 20013 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 6: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:07.017051 23194 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:07.018019 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:07.018340 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:07.017347 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:07.021612 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.022737 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:07.023546 20014 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:07.024267 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:07.028513 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.029412 20082 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:01:07.032661 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 46158.9us
I20260709 13:01:07.025247 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:07.033444 20129 leader_election.cc:336] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.529s (SENT)
I20260709 13:01:07.035105 20015 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:07.038003 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:06.996075 23450 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:07.020627 23269 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:01:07.041132 23393 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:06.999738 23377 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:07.041414 23269 consensus_queue.cc:1243] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:07.019987 23400 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.997741 23207 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:07.019620 23194 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:06.998116 22744 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:07.044914 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:07.039597 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 62886.4us
I20260709 13:01:07.045840 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.027974 23375 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:07.050366 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.051367 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.032686 23456 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.052220 20159 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:01:07.053318 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.055179 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.053735 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.042200 23269 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:07.055625 20857 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:07.057760 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.058688 20013 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:07.059391 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:07.059726 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.032106 23419 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:07.060622 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.060875 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.022562 23435 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.062000 23419 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:06.995476 23434 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.066269 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.067210 20014 raft_consensus.cc:2368] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10.
I20260709 13:01:07.068598 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:07.069455 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.070220 20013 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.056532 23207 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:07.069602 20011 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:07.074470 20744 leader_election.cc:400] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10.
I20260709 13:01:07.076231 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.078262 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.079278 20014 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.081820 23377 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:07.082492 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.083546 20859 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:07.057868 23269 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:07.084462 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.057751 23194 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:01:07.084908 23269 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:07.086319 23269 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.087522 20158 raft_consensus.cc:1240] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:07.088810 23434 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:07.090735 23207 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:07.095497 20016 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:07.096478 23435 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:07.057824 22744 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.098547 23397 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:07.098769 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.099382 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.091475 23445 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:07.101084 20858 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:01:07.101858 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.103423 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.103523 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.104339 20085 raft_consensus.cc:2393] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:07.105360 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.106405 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.107367 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.108053 23445 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:07.109093 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.111377 20085 raft_consensus.cc:2368] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:07.111259 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:07.112375 20089 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:07.109898 20013 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:07.048314 23450 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.117866 20014 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:07.121482 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.121950 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.126892 20089 raft_consensus.cc:2393] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:07.122002 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:07.122920 20129 leader_election.cc:336] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.623s (SENT)
I20260709 13:01:07.122735 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.129179 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.120124 23435 consensus_queue.cc:1059] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:01:07.129421 20011 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:07.133921 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.119621 23445 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:07.135568 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.148646 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:07.149540 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.137081 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.196805 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.196581 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.151062 20161 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 10. Ops: []
I20260709 13:01:07.199800 20014 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:07.137707 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.165123 20164 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:07.201572 20744 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:07.145823 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.144285 20159 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:07.144345 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:07.196761 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 73870.3us
I20260709 13:01:07.137697 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.202970 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:07.203014 20009 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.204049 20165 raft_consensus.cc:1240] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:07.204083 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.199231 20089 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
I20260709 13:01:07.206395 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.152046 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.208390 20744 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:07.208833 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.210088 20085 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:07.210889 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.205363 20743 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.211898 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.137676 23434 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.222189 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.223655 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.222805 20089 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.226825 20014 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:07.229372 20082 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:07.229615 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.230650 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.233227 20085 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:07.235399 20084 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:07.237172 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.136442 23445 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:07.222832 23474 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:01:07.232729 23454 consensus_queue.cc:1059] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:01:07.248551 23454 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:07.249130 23454 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:07.249527 23466 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.311s	user 0.002s	sys 0.000s
I20260709 13:01:07.251230 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.221101 23377 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:07.252009 20012 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:07.253391 20012 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:07.253829 20014 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:07.255203 20014 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:07.255617 20012 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:01:07.239323 20084 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:07.221498 23375 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:07.258672 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:07.260428 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.139567 23094 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.245631 23445 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.261252 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.261900 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.221865 23460 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:07.241704 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.265344 20851 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:07.258760 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:07.266285 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.267860 23094 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:07.268193 23375 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:07.269675 20156 raft_consensus.cc:2410] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:07.269248 23375 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.270705 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:07.270987 20156 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:07.264719 23400 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:07.277226 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:07.278996 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.279152 23445 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:07.281541 23445 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:07.282743 20014 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:07.282966 20857 raft_consensus.cc:2410] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:07.192576 23435 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:07.304211 20859 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:07.239967 23207 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:07.230207 23351 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.293572 23394 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:07.241607 23384 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:07.317112 23384 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:07.317680 23384 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.318380 20089 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:07.284592 23460 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.320389 23460 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.301159 23293 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:07.301669 23017 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:07.302388 23398 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:07.308207 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 66582.4us
I20260709 13:01:07.313390 20016 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:01:07.314100 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 67143.1us
I20260709 13:01:07.315590 20009 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:07.326963 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.328564 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:07.250140 23454 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.334208 23454 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:07.338894 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 115324us
I20260709 13:01:07.340764 20743 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.343043 20743 leader_election.cc:400] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:07.344152 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.345570 20743 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.326391 23165 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:01:07.349268 22755 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.350003 23269 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:07.354331 23430 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.368171 23451 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:07.366081 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62059.4us
W20260709 13:01:07.374919 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 48065.6us
I20260709 13:01:07.375648 20129 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:07.376175 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47736.8us
I20260709 13:01:07.377031 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:07.376974 23442 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.380863 20082 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:07.381258 20082 pending_rounds.cc:85] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 6
I20260709 13:01:07.381496 20082 pending_rounds.cc:107] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:07.381722 20082 raft_consensus.cc:2889] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:07.378448 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:07.383459 23094 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.385324 23487 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:07.369176 23398 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:07.392798 23398 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:07.393782 23398 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.396766 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:07.396842 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.408290 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:07.409312 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.409271 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.413654 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:07.413705 20009 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:07.414690 20858 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:07.416946 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.418546 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.372938 23377 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:07.420812 20014 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:01:07.418004 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:07.424227 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:07.421051 23442 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.428503 20858 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:07.433267 23489 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:07.434293 23442 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:07.434742 23442 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.435073 23442 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:07.435134 23489 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:07.437501 23447 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:07.412606 23293 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:07.420796 23487 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:07.445075 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:07.449800 23440 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:07.450428 23440 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:07.455564 23484 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.456257 23484 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.457304 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.459875 23484 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:07.460198 23293 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:07.461315 23293 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.460299 23450 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:07.471314 20055 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.473719 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.479225 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:07.480046 23419 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:07.452665 23254 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:07.481051 23042 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:07.481638 23042 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:07.482697 23042 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.483505 23194 raft_consensus.cc:3055] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:07.484073 23194 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:07.484337 23042 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.484967 23194 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:07.485790 20856 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:07.486002 23194 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.488098 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:07.489051 23484 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.489873 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.490655 23484 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.491811 20743 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:07.494549 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:07.479997 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 86913.8us
I20260709 13:01:07.423782 23384 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:07.497951 20011 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:01:07.500365 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:07.500856 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.502442 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.499014 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.512568 20011 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:07.451416 23440 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:07.501312 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.525785 20158 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.526443 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.527530 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:07.517292 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 108055us
I20260709 13:01:07.519459 20011 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:07.522845 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60530 (request call id 32) took 2011 ms. Trace:
I20260709 13:01:07.524113 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.529438 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.524780 20854 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:07.524966 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.530514 20161 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.517931 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:07.527619 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:07.533933 20014 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.528846 20010 rpcz_store.cc:276] 0709 13:01:05.511885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.512377 (+   492us) service_pool.cc:224] Handling call
0709 13:01:07.522605 (+2010228us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b5400398d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ac853 00007fa03377a60a 00007fa02cd007d4 00007fa02cd00762 00007fa02cd00691 00007fa02cd0060c 00007fa02ccff8c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:07.522738 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546406272}
I20260709 13:01:07.545290 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.550148 20853 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:07.552181 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:07.552641 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.553049 20159 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:07.548652 23254 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.517416 23419 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:07.557819 20743 leader_election.cc:336] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.651s (SENT)
I20260709 13:01:07.577935 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:07.508145 23446 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:07.558629 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:07.581562 20158 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:07.581135 20014 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:07.584076 20164 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:07.584477 20088 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:07.549863 23400 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.560859 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.506937 23484 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:07.519040 23435 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:07.568230 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:07.508982 23450 consensus_queue.cc:1059] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:01:07.549752 23417 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.560366 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.591346 20161 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:07.593065 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.596730 23202 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.572182 23380 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:07.577381 23017 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:07.604115 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.606346 20164 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:01:07.606762 20164 pending_rounds.cc:85] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 4
I20260709 13:01:07.607177 20164 pending_rounds.cc:107] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:01:07.607466 20164 raft_consensus.cc:2889] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:07.585420 23410 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.580701 23446 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:07.600653 23269 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.601202 23419 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.599746 23451 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:07.617921 23483 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:01:07.618124 23409 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:01:07.568141 23487 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:07.577636 23232 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:01:07.537086 23377 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.580013 23447 consensus_queue.cc:1059] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:01:07.615278 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 64628.3us
W20260709 13:01:07.642673 20743 leader_election.cc:336] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.674s (SENT)
I20260709 13:01:07.622870 22835 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.579694 23254 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:07.631702 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.609781 23435 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:07.623989 23446 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:07.628837 23017 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.647964 23450 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:07.624923 23494 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:01:07.645043 23496 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:01:07.645921 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.699975 23442 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.753906 23446 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.959059 23450 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:07.959245 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 313428us
I20260709 13:01:07.960794 23442 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:07.960892 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:07.893353 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 303096us
I20260709 13:01:07.962047 23450 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:07.962567 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:07.933264 23475 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:07.963595 23358 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:07.964488 23450 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:07.663369 23435 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:07.736100 23504 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:07.722671 23460 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.965902 23504 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:07.966755 23504 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:07.967639 23450 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:07.967736 23504 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.968303 23450 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:07.969065 23358 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:07.969515 23358 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:07.970561 23358 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:07.970971 23450 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:07.977121 20014 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:07.971724 23358 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:07.980618 23487 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:07.982081 23487 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:07.982564 23487 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:07.983700 23487 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:08.044531 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:08.046633 23450 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:08.047216 23450 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:08.048311 23450 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.048703 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:08.051883 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.065229 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 1014) took 1322 ms. Trace:
I20260709 13:01:08.065913 20855 rpcz_store.cc:276] 0709 13:01:06.743128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.743282 (+   154us) service_pool.cc:224] Handling call
0709 13:01:06.749933 (+  6651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.765328 (+ 15395us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:06.790533 (+ 25205us) log.cc:844] Serialized 14 byte log entry
0709 13:01:06.792644 (+  2111us) log.cc:844] Serialized 14 byte log entry
0709 13:01:06.792804 (+   160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:06.792847 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:06.792890 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.064857 (+1271967us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b540000aad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:08.064936 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.065147 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44649,"prepare.queue_time_us":32673,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3597,"raft.queue_time_us":7698,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":8437,"spinlock_wait_cycles":36261504}
I20260709 13:01:08.083459 23384 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.093905 23516 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:08.098398 23430 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:08.102766 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.106891 20855 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:08.112154 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.113356 19981 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:08.114205 19981 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:08.114877 23516 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:08.114986 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.115613 23516 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:08.116063 23522 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:08.116290 23450 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:08.117372 20010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.017s	sys 0.012s
W20260709 13:01:08.117805 20010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.018s	sys 0.013s
I20260709 13:01:08.117883 23522 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:08.117658 23516 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.118824 23522 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.116241 20091 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:08.133155 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:08.134238 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 568) took 1201 ms. Trace:
I20260709 13:01:08.134662 20087 rpcz_store.cc:276] 0709 13:01:06.933074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.132774 (+1199700us) service_pool.cc:224] Handling call
0709 13:01:08.134164 (+  1390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.135703 23417 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:08.142115 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:08.143147 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.156306 23443 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:08.156908 23443 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:08.157835 23443 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.159374 23450 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.159368 23443 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:08.160018 23450 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.162865 23450 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.158301 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:08.166306 23323 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:08.176095 23351 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.177275 23351 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:08.179216 20090 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.013s	sys 0.005s
I20260709 13:01:08.179574 23351 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:08.179665 20090 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.013s	sys 0.005s
I20260709 13:01:08.181565 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:08.182291 20083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 567) took 1256 ms (client timeout 1651 ms). Trace:
W20260709 13:01:08.182549 20083 rpcz_store.cc:269] 0709 13:01:06.926000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.181283 (+1255283us) service_pool.cc:224] Handling call
0709 13:01:08.182235 (+   952us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:08.183797 23522 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.184458 23522 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.186813 23522 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:08.186869 23398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.014s	sys 0.015s
W20260709 13:01:08.187561 23398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.015s	sys 0.016s
I20260709 13:01:08.190913 20165 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:08.202428 20012 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:08.205639 23490 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:08.206892 23490 consensus_queue.cc:1243] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:08.207635 23398 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:08.208181 23398 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:08.210003 20161 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:01:08.211302 20011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.010s	sys 0.009s
W20260709 13:01:08.211762 20011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.010s	sys 0.009s
I20260709 13:01:08.212917 23398 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.214337 23398 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:08.215452 23520 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:08.208348 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:08.222555 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.224040 23515 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:08.224704 23351 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:08.225455 23351 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.227814 23394 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:08.228065 23351 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:08.228319 23394 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:08.229242 23394 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.230527 23394 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.244653 20011 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:08.245254 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1168) took 1322 ms. Trace:
I20260709 13:01:08.245700 20082 rpcz_store.cc:276] 0709 13:01:06.922784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.231365 (+308581us) service_pool.cc:224] Handling call
0709 13:01:07.245977 (+ 14612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.382159 (+136182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.245023 (+862864us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540078f854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:08.245197 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309303040}
I20260709 13:01:08.245255 22755 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.247557 22755 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.247007 23405 consensus_queue.cc:1059] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:08.249138 23398 raft_consensus.cc:3055] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:08.249528 23398 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:08.250429 23398 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:08.251415 23398 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.251720 23515 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.252302 23515 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.258343 23254 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:08.264722 23515 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:08.265087 23094 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.265630 23094 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.266402 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:08.272833 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1016) took 1014 ms. Trace:
I20260709 13:01:08.273838 20853 rpcz_store.cc:276] 0709 13:01:07.258320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.272651 (+ 14331us) service_pool.cc:224] Handling call
0709 13:01:07.272924 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.556025 (+283101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.272598 (+716573us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54000032d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:08.272770 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637139840}
I20260709 13:01:08.275578 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1170) took 1352 ms. Trace:
I20260709 13:01:08.281394 20085 rpcz_store.cc:276] 0709 13:01:06.923410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.234389 (+310979us) service_pool.cc:224] Handling call
0709 13:01:07.234783 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.236005 (+  1222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.236033 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.246023 (+  9990us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.462915 (+216892us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b18001347a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:01:07.524022 (+ 61107us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:07.621065 (+ 97043us) log.cc:844] Serialized 14 byte log entry
0709 13:01:07.622143 (+  1078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.623972 (+  1829us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:07.675974 (+ 52002us) raft_consensus.cc:1685] finished
0709 13:01:08.275164 (+599190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.275392 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1474,"prepare.queue_time_us":148347,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":1551,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":26868,"spinlock_wait_cycles":59007360,"thread_start_us":126230,"threads_started":2,"wal-append.queue_time_us":343436}
I20260709 13:01:08.277364 20851 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:08.283262 20086 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:08.275728 23094 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:08.290867 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1152) took 1372 ms. Trace:
I20260709 13:01:08.291471 20086 rpcz_store.cc:276] 0709 13:01:06.912659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.912778 (+   119us) service_pool.cc:224] Handling call
0709 13:01:08.285072 (+1372294us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540087a354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:08.285210 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3123849472}
W20260709 13:01:08.299542 20129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.011s	sys 0.006s
I20260709 13:01:08.296108 23515 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 13:01:08.300529 20129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.013s	sys 0.006s
W20260709 13:01:08.301201 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.304008 23380 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:08.306869 23380 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:08.306951 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:08.307873 20164 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:08.308238 23380 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.312724 23380 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:08.316411 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.279181 23394 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.325201 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:08.328065 23405 consensus_queue.cc:1059] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:08.329079 23490 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:08.329634 23490 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:08.330914 23490 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.337136 20857 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:08.341349 20853 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:01:08.341799 20853 pending_rounds.cc:85] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 5
I20260709 13:01:08.342020 20853 pending_rounds.cc:107] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:01:08.342244 20853 raft_consensus.cc:2889] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:08.342499 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.345152 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.343750 20857 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 4.
I20260709 13:01:08.347301 23017 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:08.350199 20852 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:08.323698 23394 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.361877 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:08.366183 23017 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:01:08.364912 23391 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:08.369825 23391 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.370767 23391 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.368016 23405 consensus_queue.cc:1059] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:01:08.374241 23017 raft_consensus.cc:3055] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:08.374830 23017 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:08.376784 23017 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.385389 23375 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.385751 22755 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.386358 22755 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.386240 23375 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.386956 22755 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.387099 23391 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.388979 23394 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.389214 23391 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:08.393807 23375 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:08.396646 22744 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:08.399860 23446 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:08.410774 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.417586 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.419381 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:08.420459 20852 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:08.424685 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.425166 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:08.426342 20164 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:08.383714 23460 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:08.427894 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.429073 20856 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:08.425760 20850 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:08.436467 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.427313 23017 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:08.435813 23042 consensus_meta.cc:287] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.951s	user 0.017s	sys 0.000s
I20260709 13:01:08.539309 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1249) took 2218 ms. Trace:
I20260709 13:01:08.393904 23432 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:01:08.529209 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.436949 20853 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:08.437809 23293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.021s	sys 0.007s
I20260709 13:01:08.533450 20852 raft_consensus.cc:2468] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 5.
W20260709 13:01:08.535835 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 82856.3us
I20260709 13:01:08.548182 23506 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:08.552235 20857 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:08.562688 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:08.564154 20129 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:08.569813 23299 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.576058 23299 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.580674 23490 raft_consensus.cc:2804] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:01:08.582465 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:08.583757 20009 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:01:08.585882 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.587530 20164 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:08.588860 23391 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:08.591406 23490 raft_consensus.cc:697] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:08.569890 22744 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:08.592641 22744 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.551241 23432 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:08.592995 23490 consensus_queue.cc:237] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.594298 23432 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:08.593020 22744 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:08.600428 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:08.565248 23293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.023s	sys 0.014s
I20260709 13:01:08.602629 23276 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:08.607954 23351 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:08.614044 23432 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:08.631809 22755 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.637621 23506 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:08.613580 23391 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:01:08.627856 23299 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.629063 23276 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.654785 23276 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:08.656822 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.660816 20164 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:01:08.665493 20164 pending_rounds.cc:85] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 5
I20260709 13:01:08.665774 20164 pending_rounds.cc:107] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:01:08.645126 23506 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.666031 20164 raft_consensus.cc:2889] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:08.661456 23299 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.663049 23394 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.633262 23351 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.667956 23299 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.687283 23380 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.700958 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:08.709612 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:08.710757 20013 raft_consensus.cc:2393] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:08.676139 23394 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:08.714128 23534 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:01:08.717365 23405 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:08.717864 23405 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:08.718782 23405 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:08.711875 23520 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:01:08.711601 23228 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:08.723043 23228 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.723483 23228 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.729466 23394 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:08.731251 19871 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab200 after lost signal to thread 12412
W20260709 13:01:08.732481 19871 debug-util.cc:398] Leaking SignalData structure 0x7b0800368ba0 after lost signal to thread 19874
I20260709 13:01:08.694383 23380 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:08.709514 23299 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.746059 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.744935 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.741905 23405 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:08.752262 20859 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:08.752477 19871 debug-util.cc:398] Leaking SignalData structure 0x7b0800039820 after lost signal to thread 20041
W20260709 13:01:08.757136 19871 debug-util.cc:398] Leaking SignalData structure 0x7b08000f8920 after lost signal to thread 20115
W20260709 13:01:08.757472 19871 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8980 after lost signal to thread 20116
W20260709 13:01:08.758337 19871 debug-util.cc:398] Leaking SignalData structure 0x7b08003b6600 after lost signal to thread 20189
W20260709 13:01:08.759284 19871 debug-util.cc:398] Leaking SignalData structure 0x7b08000faae0 after lost signal to thread 20888
W20260709 13:01:08.759784 19871 debug-util.cc:398] Leaking SignalData structure 0x7b08001aa4c0 after lost signal to thread 23326
I20260709 13:01:08.762660 20013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:08.763373 20013 raft_consensus.cc:606] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.763718 20013 raft_consensus.cc:614] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:08.765026 20129 leader_election.cc:336] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.612s (SENT)
I20260709 13:01:08.765511 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:08.768579 23390 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:08.768843 23351 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:08.773701 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.780141 20163 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:01:08.785866 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:08.788085 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.788789 20011 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:08.790942 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:08.797302 20163 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:08.797958 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:08.796885 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:08.799086 20157 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:08.804311 23506 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:08.805254 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.805721 20013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:08.806282 20013 raft_consensus.cc:606] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.806617 20013 raft_consensus.cc:614] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:08.808405 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:08.808492 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:08.809005 23506 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:08.809931 20165 raft_consensus.cc:2410] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:08.773885 23394 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.817536 19982 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:08.819525 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:08.824677 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.825959 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.826742 23515 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:08.828052 23520 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:08.821784 23394 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.828255 23517 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:01:08.836231 23394 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.840262 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.841053 20850 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:08.842844 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:08.844079 23394 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:08.844578 23394 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.845022 23394 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:08.845049 23380 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:08.853359 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.858714 20008 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
W20260709 13:01:08.849202 23535 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.157s	user 0.002s	sys 0.002s
I20260709 13:01:08.868098 20013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "8b5b4e2a92b046d096f2661c54321d41"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:08.869465 20013 raft_consensus.cc:606] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Received request to transfer leadership to TS 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:08.828665 23352 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.871120 22744 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.872310 23352 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:08.874601 22744 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.875069 23352 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:08.824951 23506 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:08.875200 20013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:08.876641 20013 raft_consensus.cc:606] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.877127 20013 raft_consensus.cc:614] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:08.878660 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:08.879233 23394 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.881698 23394 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.882846 23487 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:08.896641 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:08.887212 23228 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:08.892140 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 64414.5us
I20260709 13:01:08.896401 23375 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.902704 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:08.908390 20858 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:08.903709 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:08.909340 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:08.914650 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:08.915763 20156 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:08.910530 23544 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:01:08.917809 23446 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.930186 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:08.930866 20851 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.933573 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:08.924464 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:08.910876 23460 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:08.541570 20017 rpcz_store.cc:276] 0709 13:01:06.320870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.321021 (+   151us) service_pool.cc:224] Handling call
0709 13:01:06.321420 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.323600 (+  2180us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:06.425050 (+101450us) log.cc:844] Serialized 14 byte log entry
0709 13:01:08.538317 (+2113267us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b18001c4b60. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a
0709 13:01:08.538800 (+   483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:08.538872 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:08.538943 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.539045 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.539232 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140,"prepare.queue_time_us":120376,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":5660,"raft.queue_time_us":1456,"spinlock_wait_cycles":13105920,"thread_start_us":118741,"threads_started":2,"wal-append.queue_time_us":102178,"wal-append.run_cpu_time_us":2582,"wal-append.run_wall_time_us":1217261}
I20260709 13:01:08.934952 23435 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:08.952442 23435 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:08.954625 23435 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:08.933014 23520 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:08.931830 23017 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:08.913312 23515 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:01:08.960851 20162 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:08.913932 23375 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:08.918736 23380 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:01:08.936654 23446 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:08.974545 23545 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:08.975126 23545 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.975476 23545 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:09.010000 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.016616 23228 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:08.966382 23017 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:09.020056 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 110787us
I20260709 13:01:09.014276 20851 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:09.023066 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:09.024585 23375 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:09.024904 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:09.025172 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:09.025558 20856 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.026755 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:09.027730 23551 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:09.029963 22755 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:09.032517 22755 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:09.032203 23551 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:09.033492 22755 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.034286 23551 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:09.022743 23520 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.039613 20858 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:09.020607 23460 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:09.051293 20856 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:09.038731 23375 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:09.043771 23435 raft_consensus.cc:2764] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:01:09.076552 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:09.076447 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45500.6us
I20260709 13:01:09.070674 23446 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:09.083521 19835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35376 (request call id 97) took 24930 ms (client timeout 15000 ms). Trace:
I20260709 13:01:09.086642 23409 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:09.087196 23409 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:09.088018 23409 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:09.089480 23409 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:09.078797 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:09.084272 19835 rpcz_store.cc:269] 0709 13:00:44.170815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.176179 (+  5364us) service_pool.cc:224] Handling call
0709 13:01:09.083366 (+24907187us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:04.820287 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:04.951352 (+131065us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:04.951412 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:01:04.951429 (+    17us) tablet.cc:662] Decoding operations
0709 13:01:04.952617 (+  1188us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:04.952729 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:04.952761 (+    32us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:01:04.953775 (+  1014us) tablet.cc:701] Row locks acquired
0709 13:01:04.953823 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:01:04.954061 (+   238us) write_op.cc:270] Start()
0709 13:01:04.954216 (+   155us) write_op.cc:276] Timestamp: P: 1783602064953919 usec, L: 0
0709 13:01:04.954242 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:01:04.955020 (+   778us) log.cc:844] Serialized 3999 byte log entry
0709 13:01:05.541680 (+586660us) op_driver.cc:464] REPLICATION: finished
0709 13:01:05.826344 (+284664us) write_op.cc:301] APPLY: starting
0709 13:01:05.826454 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:05.826753 (+   299us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:05.826918 (+   165us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:05.831878 (+  4960us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:05.832314 (+   436us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:01:05.832377 (+    63us) write_op.cc:312] APPLY: finished
0709 13:01:05.832773 (+   396us) log.cc:844] Serialized 76 byte log entry
0709 13:01:05.836932 (+  4159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:05.837118 (+   186us) write_op.cc:454] Released schema lock
0709 13:01:05.837200 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:05.864125 (+ 26925us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:05.864171 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5212470,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":251038,"apply.run_cpu_time_us":4198,"apply.run_wall_time_us":30419,"num_ops":10,"prepare.queue_time_us":178823,"prepare.run_cpu_time_us":16515,"prepare.run_wall_time_us":199055,"raft.queue_time_us":411582,"raft.run_cpu_time_us":47507,"raft.run_wall_time_us":315183,"replication_time_us":546600,"spinlock_wait_cycles":37120,"thread_start_us":518562,"threads_started":4,"wal-append.queue_time_us":38105,"wal-append.run_cpu_time_us":24054,"wal-append.run_wall_time_us":1805030}]]}
I20260709 13:01:09.092434 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:09.097429 23390 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.098519 23447 raft_consensus.cc:3055] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:09.098752 23554 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:09.099123 23447 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:09.075282 23375 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:09.103223 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.107443 20085 raft_consensus.cc:2410] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:09.104032 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.105041 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.108266 20084 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:09.102106 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:09.117781 23554 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:09.119290 23554 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.123309 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.110646 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:09.124222 20083 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:09.126134 20087 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:09.127472 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.128306 20086 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:09.126157 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:09.133855 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:09.134009 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:09.129640 23548 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:09.136796 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.137235 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:09.137530 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:09.142590 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:09.138945 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 590) took 1468 ms. Trace:
I20260709 13:01:09.141647 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.145182 20090 rpcz_store.cc:276] 0709 13:01:07.662021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.662232 (+   211us) service_pool.cc:224] Handling call
0709 13:01:07.664208 (+  1976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.664425 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:08.180528 (+516103us) log.cc:844] Serialized 14 byte log entry
0709 13:01:09.129734 (+949206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.129793 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:09.129840 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.129903 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.130009 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":948938,"prepare.queue_time_us":8146,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":954219,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":515414,"threads_started":1}
I20260709 13:01:09.148731 20156 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:09.148618 20087 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:09.135530 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 202059us
I20260709 13:01:09.107666 23446 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:09.152537 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.153465 20087 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.155391 23447 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.125381 23544 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:09.138226 23548 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:09.156761 23544 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:09.157248 23447 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.157806 23544 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.157487 23548 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.140765 23017 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:09.159704 23558 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:01:09.155494 23446 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.161864 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.162751 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:09.164255 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.165603 20082 raft_consensus.cc:2410] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:01:09.076397 23460 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.171326 20083 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:01:09.110284 23553 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:09.166286 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.125221 23232 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:09.173821 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.134449 23521 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.098886 23390 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.137049 19838 catalog_manager.cc:5697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 8, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 8 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:09.198740 20090 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:01:09.203533 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.172523 23428 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.141744 23352 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:01:09.186331 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:09.208616 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.200006 23521 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.199743 23446 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:09.220340 23323 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:09.220839 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.220338 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.222308 19838 catalog_manager.cc:5697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 8, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:09.223733 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.224243 23451 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.224622 23442 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:09.232003 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.186805 23553 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:09.140460 23375 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.225215 23419 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:09.251619 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:09.185665 23557 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:09.253561 20160 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:09.195241 23042 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.311233 23042 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.311940 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.312366 23521 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:09.313606 23042 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:09.263100 20083 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:09.315855 23443 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:09.283774 23323 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:09.316998 19838 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 7, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:09.263170 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:09.318176 23323 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:09.295208 23394 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:09.319046 23394 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.319541 23394 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:09.318773 19838 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 8 to 9, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:09.265420 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:09.265880 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 79642.3us
W20260709 13:01:09.274880 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 54665.2us
I20260709 13:01:09.343775 20008 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:09.275377 20086 raft_consensus.cc:2410] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:09.186829 23202 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:09.250027 23254 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.249888 23440 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.351343 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:09.352133 23254 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.297142 23451 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:09.189975 23549 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 13:01:09.278870 23545 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.189039 23232 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.324723 23390 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:09.325157 23553 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.265082 23428 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:09.330953 20091 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:09.255180 20158 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:09.318589 20858 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:09.334594 20851 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:09.335302 23487 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.338594 20087 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:09.296319 23446 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:09.344810 19981 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:09.298707 23442 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.350296 23556 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:09.350634 23557 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:09.373808 20009 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:09.357235 20856 raft_consensus.cc:1275] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:09.354524 23254 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:09.364383 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 108489us
I20260709 13:01:09.351300 23440 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:09.427191 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 43821.9us
I20260709 13:01:09.429715 20087 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:09.434964 20090 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:09.436916 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.437572 20084 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:09.438405 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.438979 20084 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:09.366276 19838 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 7, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 7 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
W20260709 13:01:09.368978 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 145156us
I20260709 13:01:09.440215 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.373186 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.441098 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.441340 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.442605 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.444311 20090 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.439771 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.445359 20084 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:09.446643 20084 raft_consensus.cc:3055] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:01:09.447151 20084 raft_consensus.cc:740] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:09.450320 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 77218.8us
I20260709 13:01:09.449462 19838 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 7 to 8, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:09.458719 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.366298 23446 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.359436 23202 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:09.469170 20858 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:09.470005 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:09.475081 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.471555 19838 catalog_manager.cc:5697] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 6, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 6 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:09.472995 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:09.475904 20082 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:09.479740 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:09.473656 20087 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:09.455371 20130 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.433545 23380 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.488195 20090 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:09.488687 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:09.488454 19838 catalog_manager.cc:5697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 9, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 9 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:09.435986 23435 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.484696 23299 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.495796 20857 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:09.497654 23017 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.497974 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.495579 19838 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 4 to 7, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 7 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:09.504678 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:09.505100 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.506460 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:09.507361 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:09.519176 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.509686 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.448886 23489 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.368964 23405 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:01:09.522620 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.368589 23487 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.523331 20090 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:09.525228 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.526432 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.530539 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:09.479825 20084 consensus_queue.cc:260] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.533689 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.463629 23228 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.537725 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.528260 19838 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 8 to 9, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:09.540258 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.535571 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:09.430392 23409 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:01:09.478811 23446 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.549086 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:09.478266 23419 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:09.547318 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.550880 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:09.449121 23254 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:01:09.444223 23520 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:01:09.473316 23544 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:01:09.559237 19838 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 6, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:09.571364 19838 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 7, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 7 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:09.573412 19838 catalog_manager.cc:5697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:09.562075 20082 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:09.521778 23017 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:09.576326 23017 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.564718 20055 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.542253 23496 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:09.581831 23496 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:09.582911 23496 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.542490 23432 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:09.585529 23432 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:09.586083 23432 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.575517 19838 catalog_manager.cc:5697] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:09.597556 23540 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:09.553972 23515 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:09.535266 23435 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.613582 20090 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:01:09.614334 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 115463us
I20260709 13:01:09.616452 20082 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:09.563410 22835 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.537721 23299 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.596953 23544 consensus_queue.cc:1243] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:01:09.629247 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.570516 23228 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.720029 20082 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:09.618176 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 148995us
I20260709 13:01:09.537809 23489 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:09.747972 20857 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:09.751055 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 162257us
I20260709 13:01:09.607230 23540 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:09.736523 23228 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.596789 23323 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
W20260709 13:01:09.735975 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 388152us
W20260709 13:01:09.765671 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 201028us
I20260709 13:01:09.761154 19838 catalog_manager.cc:5697] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:09.745502 23447 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:09.765316 23540 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:09.766645 23150 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:09.775210 23490 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:09.735006 22835 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.793376 23232 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:09.600486 23557 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:09.792681 23451 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:09.769550 23405 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 13:01:09.841953 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52053.9us
I20260709 13:01:09.803844 23545 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:09.819106 23442 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:09.846458 19838 catalog_manager.cc:5697] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:09.826279 23552 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:01:09.855367 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:09.857092 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:09.835446 23515 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.826524 23574 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.882552 20157 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:09.945982 23554 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:09.950158 23558 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.833232 23490 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.825737 22755 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:09.844914 23202 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:09.975791 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 92943.9us
I20260709 13:01:09.835147 23428 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:09.837498 23568 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.832387 23323 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:09.988833 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:09.986732 19838 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:10.021765 23299 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.023339 23228 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.024955 19838 catalog_manager.cc:5697] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 6, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 6 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:10.031072 20851 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:10.029364 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:09.975400 23490 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:09.993407 23564 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
W20260709 13:01:09.995893 23575 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:01:10.054781 19838 catalog_manager.cc:5697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:10.113178 19838 catalog_manager.cc:5697] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:09.986285 23572 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.111153 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:10.117532 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 347531us
W20260709 13:01:10.118023 23553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.007s	sys 0.008s
I20260709 13:01:09.997496 23487 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:10.118629 19980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.005s	sys 0.011s
I20260709 13:01:10.024506 23409 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.001366 23323 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:10.125784 19980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.005s	sys 0.011s
I20260709 13:01:09.967466 23515 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.133440 20055 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:10.122629 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:09.970989 23554 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:09.975400 23558 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.151595 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:09.989718 23442 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.131s
W20260709 13:01:10.159245 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.995092 23496 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.120409 23555 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.179630 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:10.182346 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:10.167910 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1198) took 1017 ms. Trace:
W20260709 13:01:10.221711 20088 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.012s	sys 0.030s
I20260709 13:01:10.184155 23589 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.184724 20015 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:10.133132 23553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.008s	sys 0.008s
I20260709 13:01:10.185863 23544 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.186676 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:10.174057 23595 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:10.132731 23572 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.192545 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:10.194242 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:10.194891 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:10.195093 23588 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:10.196309 23323 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.198122 22744 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:10.198762 23545 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.159328 23232 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.174220 23375 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:10.158967 23042 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.140276 23583 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.205045 23352 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:10.210441 23569 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.252s	user 0.000s	sys 0.000s
I20260709 13:01:10.221659 20162 rpcz_store.cc:276] 0709 13:01:09.150642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.169204 (+ 18562us) service_pool.cc:224] Handling call
0709 13:01:09.169542 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.169971 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:09.365483 (+195512us) log.cc:844] Serialized 14 byte log entry
0709 13:01:09.969381 (+603898us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:09.969937 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:01:09.970110 (+   173us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:09.970153 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.970235 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.167590 (+197355us) raft_consensus.cc:1685] finished
0709 13:01:10.167621 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.167813 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21007,"prepare.queue_time_us":27618,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":139311,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":15068,"thread_start_us":777228,"threads_started":3,"wal-append.queue_time_us":627000}
I20260709 13:01:10.141198 23409 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:10.222642 20088 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.012s	sys 0.031s
I20260709 13:01:10.267287 23202 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.974294 23574 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.140795 23390 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.248363 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:09.996366 22755 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:10.249493 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 99873us
I20260709 13:01:10.338281 20015 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:10.253007 20010 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:10.241252 23564 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:01:10.165378 23487 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.141075 23596 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:10.284042 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 83803.4us
I20260709 13:01:10.299777 20090 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
W20260709 13:01:10.316007 19981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.027s	sys 0.051s
W20260709 13:01:10.385780 19981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.027s	sys 0.051s
W20260709 13:01:10.386425 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 913400us
I20260709 13:01:10.175396 23489 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.174226 23447 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 13:01:10.228008 20157 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
W20260709 13:01:10.329345 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 177480us
I20260709 13:01:10.330367 22835 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.330500 23602 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:10.267594 23435 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.242036 23555 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:10.402446 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:10.323666 23580 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:10.332340 23254 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.253827 23544 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:10.427372 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 113616us
I20260709 13:01:10.428839 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:10.430867 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.299677 23232 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.299962 23545 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.300024 22744 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.249848 23589 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.307600 23042 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.433185 23409 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.324332 23583 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:10.497059 23560 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.019s	user 0.001s	sys 0.003s
I20260709 13:01:10.394421 23390 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.457186 23580 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.394371 23558 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.324512 23352 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:10.417068 22835 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.470069 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1197) took 1319 ms. Trace:
I20260709 13:01:10.462266 23254 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.275830 23588 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:10.518993 20158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33788 (request call id 700) took 2305 ms (client timeout 2143 ms). Trace:
I20260709 13:01:10.506768 20160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 1094) took 1373 ms. Trace:
W20260709 13:01:10.528662 20158 rpcz_store.cc:269] 0709 13:01:08.201475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.202096 (+   621us) service_pool.cc:224] Handling call
0709 13:01:10.500556 (+2298460us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b5400573354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:10.507208 (+  6652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299071744}
I20260709 13:01:10.528755 20160 rpcz_store.cc:276] 0709 13:01:09.133472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.133655 (+   183us) service_pool.cc:224] Handling call
0709 13:01:10.506570 (+1372915us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400244d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:10.506702 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245197568}
I20260709 13:01:10.405828 23487 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:10.532191 20012 raft_consensus.cc:2410] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:10.524180 20165 rpcz_store.cc:276] 0709 13:01:09.150269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.159481 (+  9212us) service_pool.cc:224] Handling call
0709 13:01:09.160261 (+   780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.170096 (+  9835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.170135 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:09.188351 (+ 18216us) log.cc:844] Serialized 26 byte log entry
0709 13:01:09.373168 (+184817us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:09.373208 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.373269 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.469832 (+1096563us) raft_consensus.cc:1685] finished
0709 13:01:10.469850 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.470006 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9419,"raft.queue_time_us":9914,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":7450,"thread_start_us":94939,"threads_started":1,"wal-append.queue_time_us":598020}
I20260709 13:01:10.537978 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:10.544687 23581 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.544714 23487 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.545451 23607 consensus_queue.cc:1048] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:10.546141 23591 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 13:01:10.547153 23617 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:10.547629 23617 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.547739 23608 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.548046 23617 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:10.548255 23608 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.550523 23608 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:10.545636 23600 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:10.535390 23432 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.949s	user 0.012s	sys 0.001s
I20260709 13:01:10.561550 23432 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.400620 23596 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:10.564651 20157 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:10.565426 23588 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.565828 23596 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.545497 23581 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.546432 23232 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.552030 23548 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:10.551664 22755 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:10.560220 23380 consensus_meta.cc:287] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.046s	user 0.011s	sys 0.002s
W20260709 13:01:10.560860 23540 consensus_meta.cc:287] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.000s
I20260709 13:01:10.570724 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 591) took 1074 ms. Trace:
I20260709 13:01:10.524453 23375 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:01:10.538822 23394 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.198s	user 0.025s	sys 0.004s
I20260709 13:01:10.580220 20083 rpcz_store.cc:276] 0709 13:01:09.496820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.517470 (+ 20650us) service_pool.cc:224] Handling call
0709 13:01:09.544876 (+ 27406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.563706 (+ 18830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.563740 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:09.596532 (+ 32792us) log.cc:844] Serialized 26 byte log entry
0709 13:01:09.834021 (+237489us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:09.834065 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.834150 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.570440 (+736290us) raft_consensus.cc:1685] finished
0709 13:01:10.570471 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.570648 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37474,"raft.queue_time_us":18941,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":32557,"thread_start_us":87933,"threads_started":1,"wal-append.queue_time_us":706750}
I20260709 13:01:10.595548 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:10.596158 20083 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:10.533738 22835 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.605916 23254 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.605787 23042 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.607218 20089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 596) took 1108 ms. Trace:
I20260709 13:01:10.580892 23515 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.578718 23545 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:10.610368 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 181781us
I20260709 13:01:10.610056 20089 rpcz_store.cc:276] 0709 13:01:09.498499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.576947 (+ 78448us) service_pool.cc:224] Handling call
0709 13:01:09.577346 (+   399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:09.577765 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.577804 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:09.578811 (+  1007us) log.cc:844] Serialized 52 byte log entry
0709 13:01:09.845184 (+266373us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:09.951715 (+106531us) log.cc:844] Serialized 14 byte log entry
0709 13:01:09.987290 (+ 35575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.987390 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.606996 (+619606us) raft_consensus.cc:1685] finished
0709 13:01:10.607015 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.607154 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69666,"prepare.queue_time_us":113602,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":10691,"raft.queue_time_us":213363,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":10541,"thread_start_us":258906,"threads_started":1,"wal-append.queue_time_us":966186}
I20260709 13:01:10.599308 23432 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:10.630203 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 171419us
I20260709 13:01:10.599468 23496 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.632773 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:10.636411 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:10.600258 23380 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:10.639099 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 210306us
I20260709 13:01:10.642702 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:10.646708 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.648416 20158 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:10.643554 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:10.646785 20165 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:10.651401 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:10.652729 20160 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:10.653138 20164 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:10.593688 23540 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.624912 23042 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:10.650764 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:10.658712 20162 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:10.661531 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:10.600908 23394 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.635528 23232 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:10.653496 23545 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:10.658004 23446 consensus_meta.cc:287] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.099s	user 0.006s	sys 0.008s
I20260709 13:01:10.650755 23572 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.652724 22755 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:10.667549 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:10.661578 23515 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:10.666436 23380 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:10.673058 23394 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:10.673588 23601 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.132s
I20260709 13:01:10.678222 20858 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:10.678889 23598 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:01:10.713141 20163 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:10.679358 20851 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:10.662020 23574 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:10.663666 23042 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.680114 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.680509 23352 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.681131 22835 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.681967 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:10.684288 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:10.684800 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.685875 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.720642 20163 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:10.725601 23609 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.726254 23609 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.726640 23440 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.727243 22744 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:10.728693 23581 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.729954 23580 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:10.730700 23594 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:10.731258 23594 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.732295 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.733274 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.734361 20163 raft_consensus.cc:2410] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:10.732445 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.685726 23380 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:10.741528 23042 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.712719 23555 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.754980 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.743203 23609 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:10.669322 23446 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.665199 23375 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.744226 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.744501 23627 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:10.744982 23621 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:10.745402 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.745734 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.714287 23515 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:10.860137 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.741617 23594 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.722414 22755 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.745606 23626 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:10.815279 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.865345 20856 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:01:10.817683 20009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 1297) took 1649 ms (client timeout 1852 ms). Trace:
W20260709 13:01:10.866241 20009 rpcz_store.cc:269] 0709 13:01:09.162361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.162498 (+   137us) service_pool.cc:224] Handling call
0709 13:01:10.811424 (+1648926us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540073f0d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:10.811633 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471373056}
I20260709 13:01:10.667066 23232 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.866864 23616 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:01:10.736743 23551 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:10.869385 23232 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.751972 23380 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:10.873919 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:10.721624 23017 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 1.145s	user 0.014s	sys 0.001s
I20260709 13:01:10.874888 20083 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:10.875150 23017 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:10.877295 23017 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:10.877846 23446 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.879024 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:10.879719 20083 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:10.879876 20164 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.860831 20159 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:10.865778 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.880740 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:10.881408 20165 raft_consensus.cc:2410] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:10.881492 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.882153 20159 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:10.884850 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.886005 20163 raft_consensus.cc:2410] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:01:10.888921 20156 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:10.855144 23627 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:10.891861 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.893253 20165 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:10.893486 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:10.895202 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:10.897003 20083 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:10.899667 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:10.900360 20165 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:10.901795 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:10.904048 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:10.905031 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 157716us
I20260709 13:01:10.877204 23487 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:10.879163 23594 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.910943 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:10.863708 23611 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:01:10.916447 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.919421 20852 raft_consensus.cc:2410] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:01:10.925072 23631 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:10.862763 23621 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:10.930598 23621 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:10.898568 23627 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:10.884735 22755 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:10.937894 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.943142 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.943867 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:10.912462 23592 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.912813 23254 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:10.908313 23487 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:10.961400 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.961866 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.961668 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:10.962440 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:10.962610 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:10.955367 23254 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.965130 23254 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:10.973636 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:10.974735 23551 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:10.945955 23094 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.989656 23624 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:10.990103 23611 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:10.953441 23592 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.920537 23594 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:10.993464 23094 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:10.996793 23592 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.997052 23094 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.998522 23594 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.000674 23624 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:10.975310 23551 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.003495 23624 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:11.003495 23551 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:10.902951 23515 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.011144 20083 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:11.012223 20083 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:11.011142 23617 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:11.008942 23490 consensus_meta.cc:287] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.889s	user 0.008s	sys 0.005s
W20260709 13:01:11.013641 20087 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1189) took 1575 ms (client timeout 1898 ms). Trace:
W20260709 13:01:11.037606 20087 rpcz_store.cc:269] 0709 13:01:09.438350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.440866 (+  2516us) service_pool.cc:224] Handling call
0709 13:01:11.013462 (+1572596us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b54004c6154. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:11.013582 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72959360}
I20260709 13:01:11.029613 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.040524 20159 raft_consensus.cc:2410] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:11.032369 20162 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:11.033936 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.043187 20157 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:11.032248 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.045219 20158 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:11.019323 23609 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:01:11.050866 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.051088 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.051158 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.052060 20009 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:11.052140 20014 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:11.052225 20157 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:11.055693 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:11.034610 20083 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:11.056576 20087 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:11.073227 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:11.074244 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:11.074880 20087 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:01:11.051285 20056 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 1.543s (SENT) [suppressed 24 similar messages]
I20260709 13:01:11.079052 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.079310 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.097363 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.097595 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:11.013160 23428 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:11.098189 20015 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:11.098338 20087 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:01:11.016654 22755 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:11.018618 23390 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.110497 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:11.031222 23604 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:11.112135 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:11.032747 23611 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:01:11.055796 23506 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:11.012630 23551 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:01:11.020100 23487 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.123813 20087 pending_rounds.cc:85] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 3
I20260709 13:01:11.129560 20087 pending_rounds.cc:107] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:01:11.129997 20087 raft_consensus.cc:2889] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:11.056139 23490 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.116525 23601 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:11.119959 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.102645 23428 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:11.116338 23604 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.077135 23616 consensus_queue.cc:1059] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:01:11.153258 23638 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:01:11.161571 20160 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:11.162855 23490 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.169540 23617 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.172377 23555 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:11.172986 23555 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.173553 23555 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:11.174683 23614 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.628s	user 0.009s	sys 0.022s
I20260709 13:01:11.112892 23460 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:01:11.128847 23506 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.147684 23487 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.147011 23604 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:11.192968 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.177249 23254 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.209987 23094 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.210837 23506 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.178814 23609 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:11.182068 23490 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:11.216985 23641 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:01:11.217211 23551 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:11.217658 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 113446us
I20260709 13:01:11.218990 23094 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.193493 23380 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.219233 23592 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.183432 23586 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:11.235929 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.236305 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.232496 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.238935 20165 raft_consensus.cc:2393] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:11.238765 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 119103us
I20260709 13:01:11.210871 23254 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.214814 23609 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:01:11.243456 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 192174us
I20260709 13:01:11.241356 19982 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.227203 23551 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:11.244451 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:11.248521 20009 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:11.252362 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.252647 20743 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:11.250042 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.257108 20743 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:11.262852 20743 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:11.230487 23490 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.273677 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.275100 23094 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:11.241340 23586 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.275731 23094 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.278234 23586 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.278579 23568 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:11.279047 23568 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.279273 23094 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.279975 23592 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.242717 23601 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:11.281486 23490 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:11.281368 23601 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:11.284654 23565 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.890s	user 0.005s	sys 0.000s
I20260709 13:01:11.285580 23380 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.264292 23515 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:11.286883 23515 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.287348 23515 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:11.289091 23599 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.744s	user 0.000s	sys 0.002s
I20260709 13:01:11.290724 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1341) took 2188 ms. Trace:
I20260709 13:01:11.291123 20088 rpcz_store.cc:276] 0709 13:01:09.102250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.109905 (+  7655us) service_pool.cc:224] Handling call
0709 13:01:09.125122 (+ 15217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.125340 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.125364 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:09.542120 (+416756us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b2c001392a8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:01:09.542588 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.323262 (+780674us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:10.323318 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.323414 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.288523 (+965109us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:11.290468 (+  1945us) raft_consensus.cc:1685] finished
0709 13:01:11.290494 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.290641 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65943808,"thread_start_us":715642,"threads_started":1,"wal-append.queue_time_us":978347}
I20260709 13:01:11.215580 23428 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:11.293373 23428 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.260798 23460 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:11.296593 23460 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.297117 23460 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:11.299485 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.302866 23323 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:11.292176 23644 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.107s	user 0.000s	sys 0.002s
I20260709 13:01:11.270109 23621 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:11.304481 23621 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.304827 23621 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.279363 23568 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:11.308697 23645 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:11.310937 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.265429 23626 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:11.316658 23626 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:11.269836 22755 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:11.317174 23554 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:11.322047 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.322801 20158 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.325012 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 1071) took 1297 ms. Trace:
I20260709 13:01:11.325512 20857 rpcz_store.cc:276] 0709 13:01:10.027249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.028794 (+  1545us) service_pool.cc:224] Handling call
0709 13:01:10.108182 (+ 79388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.417700 (+309518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.417734 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.417778 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.417815 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.324686 (+906871us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54000050d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:11.324750 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.324936 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711291264}
I20260709 13:01:11.341780 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.342836 20014 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:11.344568 20130 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.346021 20130 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:11.354645 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:11.355319 20014 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.360790 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:11.364581 23647 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:11.372427 23621 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.372814 23551 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.374195 23551 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.375010 23621 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.317184 23626 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.376820 23626 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.380643 23627 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.380486 23460 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:11.383021 23627 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.383849 23554 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.384420 23460 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:11.384645 23554 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.389645 20014 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:11.321408 23609 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.393844 23609 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:11.395555 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.317790 22755 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:11.396831 22755 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:11.405853 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 59927.7us
I20260709 13:01:11.410171 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.400699 23554 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.408500 20130 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.413616 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.352845 23490 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.427548 20158 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:11.430898 23555 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.433645 23555 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.421360 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.406946 23624 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:11.419098 23165 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.457216 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.427102 23017 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:11.450829 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.449688 23390 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:11.484191 23428 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.457376 23165 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.450632 23490 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.484306 23017 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.497563 22755 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.488127 23390 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.506794 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.509713 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.489563 23428 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:11.493568 23581 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:01:11.581453 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 78003.8us
W20260709 13:01:11.582242 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 319483us
I20260709 13:01:11.604346 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:11.502899 23446 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.023s	sys 0.035s
I20260709 13:01:11.607621 20158 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:11.500259 23625 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:11.607877 20743 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:11.496500 23165 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.501425 23017 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.505623 23390 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.619297 20162 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:01:11.609113 23446 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.023s	sys 0.036s
I20260709 13:01:11.607894 23581 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.609942 23232 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:11.646733 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 161416us
I20260709 13:01:11.646926 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1188) took 2208 ms. Trace:
I20260709 13:01:11.648833 20085 rpcz_store.cc:276] 0709 13:01:09.438082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.443156 (+  5074us) service_pool.cc:224] Handling call
0709 13:01:09.443411 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.535835 (+ 92424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.535866 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.535908 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:09.535984 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.646679 (+2110695us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b540047d1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:11.646732 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.646864 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212072704}
I20260709 13:01:11.630613 23581 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:11.648733 20130 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.638931 23232 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.668411 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:11.643376 23604 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.672587 23659 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:11.674917 23607 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:11.675621 23583 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:11.676017 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1365) took 1003 ms. Trace:
I20260709 13:01:11.676735 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:11.677266 23558 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:11.680073 23656 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:11.685977 23490 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.687744 22755 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.650748 23568 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.696506 23544 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.699486 23428 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:11.700569 23601 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:11.699569 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 1010) took 1515 ms. Trace:
I20260709 13:01:11.701887 20085 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:11.702287 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:11.704349 20088 rpcz_store.cc:276] 0709 13:01:10.672811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.292801 (+619990us) service_pool.cc:224] Handling call
0709 13:01:11.293247 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:11.296262 (+  3015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.296294 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:11.298041 (+  1747us) log.cc:844] Serialized 52 byte log entry
0709 13:01:11.500690 (+202649us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:11.601558 (+100868us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.602907 (+  1349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.603122 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.675744 (+ 72622us) raft_consensus.cc:1685] finished
0709 13:01:11.675765 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.675954 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3762,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":1743,"raft.queue_time_us":9694,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":4291,"thread_start_us":202204,"threads_started":1,"wal-append.queue_time_us":375078}
I20260709 13:01:11.715224 20012 rpcz_store.cc:276] 0709 13:01:10.184339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.186408 (+  2069us) service_pool.cc:224] Handling call
0709 13:01:11.699277 (+1512869us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54000faad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:11.699485 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669706112}
W20260709 13:01:11.745445 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.697477 23515 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:11.762837 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.741031 23601 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.770824 23625 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:11.771389 23625 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:11.771886 23604 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:11.773666 23589 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.776296 23583 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.776822 23583 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.778429 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:11.779119 23583 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.779320 20089 raft_consensus.cc:2368] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
I20260709 13:01:11.779858 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.781354 20743 leader_election.cc:400] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
I20260709 13:01:11.781644 20743 leader_election.cc:403] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:11.783043 23616 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.783226 23652 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.784926 23586 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:11.785470 23094 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:11.786161 23094 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:11.788303 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.788722 23375 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:11.789124 20012 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:11.790202 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.791342 23540 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.791601 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.792312 20014 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:11.792631 20009 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:11.786460 20088 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.793673 23616 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8.
W20260709 13:01:11.794873 23487 consensus_meta.cc:287] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.000s
I20260709 13:01:11.797117 20016 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:11.798287 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1363) took 1126 ms. Trace:
I20260709 13:01:11.801967 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 1070) took 2262 ms. Trace:
I20260709 13:01:11.802456 20856 rpcz_store.cc:276] 0709 13:01:09.539768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.541744 (+  1976us) service_pool.cc:224] Handling call
0709 13:01:09.555613 (+ 13869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.865889 (+1310276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.801711 (+935822us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000087d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:11.801906 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010882432}
I20260709 13:01:11.789157 23375 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:11.803869 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 606) took 1624 ms. Trace:
I20260709 13:01:11.804313 20090 rpcz_store.cc:276] 0709 13:01:10.179086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.179274 (+   188us) service_pool.cc:224] Handling call
0709 13:01:11.803702 (+1624428us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b54006a8454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:11.803814 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186895616}
I20260709 13:01:11.713176 23656 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.806131 20743 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:11.807765 23604 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:11.807936 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.808405 23604 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:11.808887 23232 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.810284 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.811357 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:11.812234 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.798741 20091 rpcz_store.cc:276] 0709 13:01:10.671542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.181186 (+509644us) service_pool.cc:224] Handling call
0709 13:01:11.181587 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.442859 (+261272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.442902 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:11.442955 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:11.443002 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.798049 (+355047us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540078fd54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:11.798090 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.798221 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":708,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":599789312}
I20260709 13:01:11.813117 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:11.815003 23232 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:11.808872 23604 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:11.815781 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.817124 23651 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.817934 23651 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:11.820117 23621 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:11.820946 23621 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:11.821729 23568 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.821995 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.823097 23621 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:11.810343 23654 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:01:11.826556 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 1074) took 1798 ms. Trace:
I20260709 13:01:11.804175 23375 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.828461 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1009) took 1644 ms. Trace:
I20260709 13:01:11.828740 23375 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:11.828873 20008 rpcz_store.cc:276] 0709 13:01:10.183978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.199272 (+ 15294us) service_pool.cc:224] Handling call
0709 13:01:10.199636 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.531528 (+331892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.531563 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.531608 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:10.531641 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.828213 (+1296572us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b540073e6d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:11.828259 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.828380 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762704768}
I20260709 13:01:11.712393 23607 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:11.835134 20017 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:11.808871 20160 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:11.837214 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.837962 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.841109 20087 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:11.841794 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:11.842370 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.842813 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.842906 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.844866 20009 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:11.838828 20011 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:11.845013 20160 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:11.841496 20014 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:11.839524 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.821367 20091 raft_consensus.cc:1240] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:11.849117 20160 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:11.850659 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.851361 20089 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:11.851393 20014 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:11.855023 20085 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:11.859789 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.864884 20015 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:11.861289 20012 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 6.
I20260709 13:01:11.853752 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:11.868803 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.870851 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.871805 20012 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:11.826985 20850 rpcz_store.cc:276] 0709 13:01:10.028221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.029878 (+  1657us) service_pool.cc:224] Handling call
0709 13:01:10.030303 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.150321 (+120018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.150356 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:10.150899 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.406483 (+255584us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.406578 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.826195 (+1419617us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54000055d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:11.826254 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.826298 (+    44us) raft_consensus.cc:1685] finished
0709 13:01:11.826310 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.826499 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9404,"raft.queue_time_us":276344,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":30930,"spinlock_wait_cycles":253499136,"thread_start_us":211178,"threads_started":1,"wal-append.queue_time_us":313743,"wal-append.run_cpu_time_us":3811,"wal-append.run_wall_time_us":1092830}
I20260709 13:01:11.883615 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.884599 20012 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:11.870787 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.888787 20016 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:11.835947 23656 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:11.891279 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.892861 20160 raft_consensus.cc:2410] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:11.839376 23601 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:11.893543 20088 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:11.787631 23094 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:11.895970 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.898469 23515 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:11.901970 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:11.903721 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50049us
I20260709 13:01:11.783816 23652 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:11.906539 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:11.785465 23586 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.908438 20162 raft_consensus.cc:2393] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:11.915570 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.917788 20160 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:11.918176 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.918948 20156 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:11.919443 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 270811us
I20260709 13:01:11.919775 20016 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:11.923249 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:11.924088 20009 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:11.925426 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.926175 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.922912 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.926213 20164 raft_consensus.cc:2393] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:11.925355 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.930557 20014 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:11.929158 20015 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:01:11.933889 20085 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:11.933682 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:11.937181 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:11.938277 20162 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:11.939882 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:11.942957 20011 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:11.946684 20088 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:11.946735 20012 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:11.950630 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:11.966308 20016 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:11.907512 23666 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:11.907217 23604 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:11.952978 23651 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:11.978847 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 45585.9us
I20260709 13:01:11.917200 23446 consensus_queue.cc:1059] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:01:11.934592 23576 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.944566 23624 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:11.979000 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:11.931471 23549 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:01:11.990406 23567 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:11.942677 23435 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:11.990736 23666 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:12.001017 23666 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.004323 23604 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:12.004745 23551 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.630s	user 0.012s	sys 0.001s
I20260709 13:01:12.005517 23576 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:11.969669 23652 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.012485 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:12.009779 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.012006 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.006731 23669 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:01:11.995311 23567 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:12.022359 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:12.023411 23567 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:12.024257 23549 raft_consensus.cc:3055] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:12.006738 23659 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.024871 23549 raft_consensus.cc:740] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:12.025202 23659 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.025944 23549 consensus_queue.cc:260] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.006156 23656 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:01:12.028282 23435 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:11.936210 23607 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.029780 23435 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.006546 23624 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.032305 23624 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:12.040778 20009 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:12.046975 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 615) took 1058 ms. Trace:
I20260709 13:01:12.049268 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.050853 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.053776 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.056420 20858 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:12.058226 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.060079 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.061188 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.064138 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:12.066707 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.069440 20084 raft_consensus.cc:3060] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:12.074904 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.076143 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.077284 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:12.027832 23549 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.992373 23017 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.075567 20854 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:12.083773 20160 raft_consensus.cc:1240] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 4. Current term is 5. Ops: []
I20260709 13:01:12.088703 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.092907 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.094127 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:12.095160 20085 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:12.096909 20084 raft_consensus.cc:1275] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:01:12.097848 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1192) took 2658 ms. Trace:
I20260709 13:01:12.097779 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.098888 20087 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 8.
I20260709 13:01:12.057363 23659 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.100625 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.105911 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.058593 23567 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:12.106244 20017 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.057407 23232 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:12.106544 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.111478 20087 raft_consensus.cc:2410] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:12.098631 20084 rpcz_store.cc:276] 0709 13:01:09.445484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.446107 (+   623us) service_pool.cc:224] Handling call
0709 13:01:09.446495 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.291249 (+1844754us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b18000bbda0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ed646 00007fa02e72758e 00007fa02e7a8f9e 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:01:12.069201 (+777952us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b580066c8d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:01:12.097506 (+ 28305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.097775 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729201280}
I20260709 13:01:12.014667 23165 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:12.110168 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.115468 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.116168 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.117472 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.117735 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:12.121142 20160 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:12.119387 20852 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:12.121384 20085 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:12.068315 23409 consensus_queue.cc:1059] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:01:12.124326 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:12.100708 23626 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.004s
I20260709 13:01:12.137449 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.100062 23568 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.124745 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.125815 20158 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:12.130647 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:12.140903 20160 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.142730 20164 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:12.143806 20017 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:12.147228 20091 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:12.150939 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.151829 20017 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:12.154568 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.156769 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.157459 20853 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:12.158602 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:12.156916 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:12.199956 23584 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:12.200644 23584 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.201064 23584 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:12.204692 23672 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:01:12.205284 23576 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.126828 23390 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:12.207571 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.208455 23390 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:12.209216 23581 consensus_meta.cc:287] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 13:01:12.210248 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.210809 23651 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:12.210983 20161 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:12.211194 20852 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 8: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:12.211306 23651 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.211956 23651 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.156384 23515 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:01:12.212320 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.213203 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.214615 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.215975 23165 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.216526 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.218467 23447 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:12.220680 23674 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:12.220860 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:12.222369 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:12.222504 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.223449 23586 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:12.224037 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:12.224449 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:12.224779 20016 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:12.225229 23017 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:12.146169 23626 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.228338 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:12.144613 23568 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.118595 23567 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:12.240373 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.241364 23567 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:12.238631 23626 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.247066 20851 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:12.240790 20156 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
W20260709 13:01:12.248133 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.195722 19981 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:12.251188 20853 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:12.252270 20858 raft_consensus.cc:2410] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:01:12.263109 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:12.264569 23515 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.265291 23515 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.265616 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.266577 20089 raft_consensus.cc:2393] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:12.268689 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.268981 23593 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:01:12.269779 20016 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:12.271404 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.271838 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.273195 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:12.274436 20089 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:12.275012 23651 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.277611 23584 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:12.288489 23584 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:12.310803 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.311997 20852 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:12.314821 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.315663 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.316520 20012 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:12.249269 23581 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.333969 20160 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
W20260709 13:01:12.334959 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.335074 20012 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:12.381749 20856 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:12.383270 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:12.383536 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.413162 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 82773.3us
W20260709 13:01:12.414629 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69527.9us
I20260709 13:01:12.415009 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.415750 20087 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:12.431814 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.433203 20089 raft_consensus.cc:2410] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:12.436028 20091 raft_consensus.cc:2393] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:12.415870 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:12.439574 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 243946us
I20260709 13:01:12.440810 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:12.444700 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:12.456245 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.456445 23650 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:12.457060 20012 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:12.445873 23580 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:12.267784 23515 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:12.461037 23580 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:12.445689 23555 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:12.461900 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.462404 23651 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.462489 23584 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:12.461961 23555 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:12.464033 23555 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:12.461978 23580 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:12.465158 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.466539 23624 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:12.446094 23150 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:12.469173 23150 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:12.470017 23150 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.419771 23581 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:12.444597 23625 consensus_meta.cc:287] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.001s
I20260709 13:01:12.473115 23625 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.476302 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.476464 23625 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:12.476760 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:12.476925 20009 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:12.477725 20088 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:12.479832 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:12.480551 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.481999 23375 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:12.482240 23626 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:12.482550 23375 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.484619 23375 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.485713 20009 raft_consensus.cc:2468] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:01:12.488356 20130 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:12.489363 23017 raft_consensus.cc:2804] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:12.489869 23017 raft_consensus.cc:697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:12.490787 23017 consensus_queue.cc:237] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.492126 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:12.494514 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.494885 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.495326 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.495541 20856 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:12.497737 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.497874 20012 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:12.498426 20855 raft_consensus.cc:2393] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:12.495576 20852 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
W20260709 13:01:12.498797 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 417444us
I20260709 13:01:12.499146 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:12.501343 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.495697 20055 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.505152 20161 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:12.508270 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.508669 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.511648 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.514562 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:12.517670 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.513551 23611 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:12.521107 23611 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.513618 23638 raft_consensus.cc:3055] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:12.508725 23625 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:12.524441 23638 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:12.525663 23638 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.527053 23638 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.528196 23576 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:12.529153 23576 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.529729 23625 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:12.509888 23531 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.508347 20017 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:12.541018 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:12.531751 23576 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:12.549589 20852 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:12.550083 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.551551 20055 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.537245 23616 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:12.544085 23611 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:12.542218 23558 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:01:12.558741 23576 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.559438 23428 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:12.580489 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 271236us
I20260709 13:01:12.581574 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:12.549185 23375 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:01:12.586380 20055 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:12.592388 20854 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:12.581279 23611 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:12.567281 23435 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:01:12.584339 23659 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:01:12.558113 23625 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.612345 23454 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:12.579509 23604 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:12.581290 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:12.583326 23576 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:12.567040 23609 consensus_queue.cc:1059] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:01:12.563499 23616 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.593279 23409 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:12.635495 20129 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.634320 23454 raft_consensus.cc:697] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:12.643549 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.624645 23611 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:12.635875 23576 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:12.654560 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.636459 23409 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:12.657776 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:12.658529 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 215658us
I20260709 13:01:12.639655 23165 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:12.633631 23586 consensus_queue.cc:1059] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:01:12.634595 23604 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.663360 23409 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:12.665278 23017 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:12.653470 23454 consensus_queue.cc:237] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.661845 23669 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:12.669869 23017 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.674080 23428 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:12.680256 23659 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:12.677227 23651 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:01:12.700663 23017 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.668625 23611 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.711071 23531 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:12.692603 23669 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.718880 20083 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.054001 20090 rpcz_store.cc:276] 0709 13:01:10.988971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.805628 (+816657us) service_pool.cc:224] Handling call
0709 13:01:11.805900 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.806236 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:11.806515 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.046596 (+240081us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:12.046651 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:12.046701 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.046768 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.046900 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31746,"prepare.queue_time_us":8445,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":35583,"thread_start_us":207947,"threads_started":1}
I20260709 13:01:12.706465 23651 consensus_queue.cc:1243] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:01:12.739308 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 247924us
I20260709 13:01:12.743448 20129 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.668814 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.748119 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:12.751717 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 72222.2us
I20260709 13:01:12.737406 23611 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:12.755504 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 86777.2us
I20260709 13:01:12.757411 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.798691 23616 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:12.792616 23580 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:12.788060 23521 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:12.754010 23588 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:12.788066 23609 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:12.798607 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.753199 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:12.765091 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.813443 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 60323.7us
W20260709 13:01:12.809895 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 228703us
I20260709 13:01:12.816514 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.819823 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.821071 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.828217 20083 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:12.829340 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1360) took 2163 ms. Trace:
I20260709 13:01:12.829388 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.830137 20083 rpcz_store.cc:276] 0709 13:01:10.665632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.968370 (+302738us) service_pool.cc:224] Handling call
0709 13:01:11.006724 (+ 38354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.718434 (+1711710us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b58008ab3d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:01:12.829031 (+110597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.829283 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148810624}
I20260709 13:01:12.832456 23588 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.832921 23588 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:12.833307 23588 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.836252 23165 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:12.836406 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:12.837715 23595 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:01:12.839979 23581 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:12.824621 23651 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:12.819232 23409 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.842949 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.846385 23409 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:12.848873 23669 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:12.818571 23607 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:12.858322 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 101031us
I20260709 13:01:12.859516 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:12.861411 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:12.862695 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:12.872941 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:12.874007 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:12.810195 23454 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:12.878878 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:12.908526 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.830063 23228 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:12.913419 20744 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:12.891546 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.917856 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 128715us
I20260709 13:01:12.924358 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:12.925768 20856 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:12.939441 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.819028 23609 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:12.875753 23544 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:12.869354 23165 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:12.906992 23584 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:12.917793 23454 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:12.961975 23521 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:12.984830 23521 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:12.959798 23556 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:12.989178 23454 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.989972 23454 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:12.991716 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.997164 23521 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:12.961557 23638 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:01:13.002506 23703 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.003151 23703 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:13.003717 23202 raft_consensus.cc:3055] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:13.004179 23202 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:13.005213 23202 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.005650 23703 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:12.873574 23375 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:01:13.006376 23202 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.967231 23624 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:13.008376 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:13.009152 20853 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:13.010375 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:13.013549 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 742) took 1085 ms. Trace:
I20260709 13:01:13.014123 20158 rpcz_store.cc:276] 0709 13:01:11.928536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.003206 (+ 74670us) service_pool.cc:224] Handling call
0709 13:01:12.003754 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.140482 (+136728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.013166 (+872684us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54002445d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:13.013358 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280127360}
I20260709 13:01:13.014562 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:13.016217 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.020179 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 111781us
I20260709 13:01:13.021845 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.017778 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.043967 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.049486 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.051578 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:13.055963 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 234960us
I20260709 13:01:13.057144 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.058647 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.003520 23017 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:12.926288 23695 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 13:01:12.871829 23669 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:01:13.064994 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.052488 20859 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:12.997265 23687 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:13.046277 23409 raft_consensus.cc:2764] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:01:13.087435 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:12.960278 23696 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.146s
I20260709 13:01:13.050876 23521 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.047129 23702 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:13.062048 23460 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:13.061450 23556 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:13.070833 23634 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:13.078660 23375 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:13.108059 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:13.110929 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.074606 23607 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:13.117769 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.101253 23702 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:13.127362 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 277948us
I20260709 13:01:13.084455 23659 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:01:13.106598 23094 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.104719 23710 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.139748 23544 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.140565 23544 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:13.141251 23708 consensus_queue.cc:1048] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:13.129633 23375 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:13.144223 23709 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:13.144876 23709 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:13.145062 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.150197 23708 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.109601 23651 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:13.145279 23709 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.152200 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.158476 23454 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:13.160040 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.160261 23609 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:13.160941 23017 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:13.144939 23375 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:13.097994 23687 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:13.164400 23687 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:13.149734 23576 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:13.170096 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.125265 23669 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 13:01:13.167883 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.134933 23702 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.193907 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:13.196552 20856 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:13.174355 23708 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:13.198532 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 133665us
W20260709 13:01:13.201180 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 127442us
I20260709 13:01:13.203097 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.197438 23669 consensus_queue.cc:1243] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:13.195470 23706 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.215768 20161 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:13.213937 20157 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:13.197935 23299 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:13.215122 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:13.218947 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.221575 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.214978 23094 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.153256 23607 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.217684 23669 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.245550 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:13.252630 20016 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:13.253024 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 83037.6us
I20260709 13:01:13.253789 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.226017 23718 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.266350 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.264469 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.267763 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.269753 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.269742 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.279016 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:13.280622 20088 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:13.281512 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:13.282258 20091 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:13.284114 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:13.284925 20088 raft_consensus.cc:2410] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:13.241240 23722 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.221279 23299 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:13.286574 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:13.297242 20089 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
W20260709 13:01:13.297432 20164 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 1120) took 1379 ms (client timeout 1550 ms). Trace:
I20260709 13:01:13.256054 23202 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:13.228502 23611 consensus_queue.cc:1059] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:01:13.261740 23719 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:13.228312 23696 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:01:13.294121 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:13.304394 20087 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:13.298923 20088 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:13.259548 23454 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:13.297829 20164 rpcz_store.cc:269] 0709 13:01:11.917579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.939642 (+ 22063us) service_pool.cc:224] Handling call
0709 13:01:13.297232 (+1357590us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54004d7454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:13.297378 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463085440}
I20260709 13:01:13.306164 23719 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:13.259593 23588 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.307157 23719 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:13.308689 23588 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:13.309218 23454 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:13.289137 23627 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.309381 23588 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:13.310204 23627 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.310701 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:13.311064 20158 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:13.310879 23588 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:13.312135 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 43836.2us
I20260709 13:01:13.309388 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:13.303398 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 185700us
I20260709 13:01:13.275710 23624 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:13.314574 20089 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:13.315943 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:13.316699 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.317833 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.318671 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:13.318456 23711 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.255864 23669 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:13.296823 23094 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:13.326980 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:13.321931 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.324867 20089 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:01:13.327613 20012 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.327764 20089 pending_rounds.cc:85] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 5
I20260709 13:01:13.328074 20089 pending_rounds.cc:107] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:01:13.328303 20089 raft_consensus.cc:2889] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:13.328725 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.274621 23709 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:13.333133 20859 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:13.339125 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:13.345314 20084 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:13.347235 20084 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:01:13.357676 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48364.1us
I20260709 13:01:13.358752 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:13.358965 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.359436 20853 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:01:13.368201 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:13.319530 23624 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:13.371188 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:13.373200 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:13.374173 20855 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:13.325207 23696 consensus_queue.cc:1243] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:13.323939 23460 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:13.323843 23710 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.322330 23698 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:13.322091 23712 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.391250 20015 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:13.391687 20012 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:13.389304 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.382774 23520 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:01:13.393477 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.394701 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:13.396025 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.397043 23716 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:13.400853 23568 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:13.330024 23722 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:13.383641 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 55020.7us
I20260709 13:01:13.406410 23669 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:13.407311 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:13.408576 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:13.409128 20859 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.326326 23723 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:01:13.413604 23687 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:13.414134 20856 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:13.334295 23094 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.414610 20856 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:13.415390 23709 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:13.382984 23731 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:13.416316 23094 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:13.416405 23731 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:13.417714 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:13.418707 20017 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 7: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:01:13.418994 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 214755us
I20260709 13:01:13.420230 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:13.420976 20165 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:13.423028 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.424026 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:13.424461 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.421868 20856 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:13.368430 23576 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:13.436547 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:13.436717 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.436467 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:13.437721 20857 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:13.379005 23556 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.439090 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.376562 23586 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:13.381129 23624 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:13.445101 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:13.390698 23698 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.445791 20089 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.383008 23634 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:13.390466 23460 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:13.454152 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.465328 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:13.466542 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.388962 23725 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.469506 20087 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:13.383829 23611 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:01:13.475132 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:13.477643 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 89258.1us
I20260709 13:01:13.389863 23545 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:01:13.435576 23718 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.550920 20014 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:13.551993 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.479403 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.448782 23698 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.414157 23687 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:13.470206 23460 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:13.545480 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.554557 20017 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
W20260709 13:01:13.559350 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.481446 23503 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.470275 23726 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.531719 23712 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.031s
W20260709 13:01:13.589570 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 47304.1us
I20260709 13:01:13.591106 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.467998 23586 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.595254 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 597) took 1758 ms. Trace:
I20260709 13:01:13.595196 20089 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:13.590147 20087 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:13.594107 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.595844 20859 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:13.596661 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:13.448863 23556 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.595742 20008 rpcz_store.cc:276] 0709 13:01:11.837096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.838460 (+  1364us) service_pool.cc:224] Handling call
0709 13:01:11.838751 (+   291us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:11.933482 (+ 94731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.933513 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:12.853243 (+919730us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:01:12.853946 (+   703us) log.cc:844] Serialized 52 byte log entry
0709 13:01:13.078365 (+224419us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:13.078427 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.594965 (+516538us) spinlock_profiling.cc:243] Waited 93.4 ms on lock 0x7b54002463d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:13.595010 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.595053 (+    43us) raft_consensus.cc:1685] finished
0709 13:01:13.595065 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.595198 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":830,"raft.queue_time_us":5517,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":12341,"spinlock_wait_cycles":220457088,"thread_start_us":223728,"threads_started":1,"wal-append.queue_time_us":301033}
I20260709 13:01:13.600420 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:13.600651 20129 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.598460 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:13.602624 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.602784 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.593469 23299 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:13.603826 23576 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:13.604071 23669 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:13.604210 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.605607 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.606158 23556 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:13.606328 20014 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:13.609254 20008 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:13.610042 23726 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:13.610734 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 1089) took 1402 ms. Trace:
I20260709 13:01:13.611122 20858 rpcz_store.cc:276] 0709 13:01:12.208695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.212049 (+  3354us) service_pool.cc:224] Handling call
0709 13:01:13.610541 (+1398492us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b54002404d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:13.610679 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89608064}
I20260709 13:01:13.611724 23607 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:13.611761 23299 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:13.612243 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:13.612521 23299 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:13.612875 23607 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:13.613317 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:13.614586 23299 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:13.614699 20858 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:13.616040 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:13.603185 20089 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:13.566514 23719 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.618147 20014 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: [12.8-15.10]
I20260709 13:01:13.475499 23634 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.618664 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.618671 20089 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:13.542222 23228 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:01:13.617524 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:13.626106 20744 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.621445 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:13.627359 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.628021 23715 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:13.628401 23739 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:13.629186 23730 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:13.629254 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.630030 23739 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:13.629853 23730 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.630556 23739 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:13.632200 23576 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.633087 23576 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:13.635484 23671 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.518s	user 0.010s	sys 0.003s
I20260709 13:01:13.613916 23607 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:13.628115 23669 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:13.628732 23696 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.592553 23638 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.628682 23715 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:13.598181 23723 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.566269 23651 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:01:13.627511 20858 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:13.597234 23731 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:13.620290 23726 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:13.696949 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:13.697036 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:13.631629 23739 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:13.664255 23712 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:13.700909 20857 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:13.674824 23715 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:13.700697 23739 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.669590 23588 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:13.626453 23719 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:13.690452 23720 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:13.702515 23738 log.cc:927] Time spent T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:01:13.708963 20129 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:13.718266 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:13.723850 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 94669.5us
I20260709 13:01:13.724504 20014 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:13.726641 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:13.720194 23588 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.703748 23712 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:13.742734 20014 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:13.742690 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:13.754020 20160 raft_consensus.cc:1240] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:13.735633 23723 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:13.786983 20164 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:13.775676 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:13.715663 23720 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:13.789135 20160 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:13.793994 20164 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:01:13.744195 23730 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:13.784183 23723 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:13.797678 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:13.798569 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:13.735361 23638 consensus_queue.cc:1059] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 13:01:13.802266 23737 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:01:13.798445 23746 consensus_queue.cc:1059] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:01:13.797715 23723 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:13.804600 23741 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:01:13.822700 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:13.805706 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:13.807135 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:13.800400 23728 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:01:13.809724 20014 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:13.785609 23228 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:13.741998 23669 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:01:13.823819 20156 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:13.833038 22744 raft_consensus.cc:993] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 8b5b4e2a92b046d096f2661c54321d41 to start an election
I20260709 13:01:13.844836 23228 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:13.847795 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1195) took 3375 ms. Trace:
I20260709 13:01:13.850922 22744 raft_consensus.cc:1081] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Signalling peer 8b5b4e2a92b046d096f2661c54321d41 to start an election
I20260709 13:01:13.849234 20086 rpcz_store.cc:276] 0709 13:01:10.472363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.514538 (+ 42175us) service_pool.cc:224] Handling call
0709 13:01:11.831161 (+1316623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.831550 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.831577 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.832108 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:01:12.120816 (+288708us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:12.120874 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.120989 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.275300 (+154311us) raft_consensus.cc:1685] finished
0709 13:01:12.275329 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.847451 (+1572122us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54006a8450. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:13.847713 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4146,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12613,"spinlock_wait_cycles":2975693568,"thread_start_us":288263,"threads_started":1,"wal-append.queue_time_us":409514,"wal-append.run_cpu_time_us":2702,"wal-append.run_wall_time_us":1042207}
I20260709 13:01:13.847851 23228 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:13.850502 23734 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:01:13.738431 23726 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:13.930828 23737 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:01:13.901835 23634 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:14.245992 23715 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.003s
I20260709 13:01:14.251880 23712 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:14.293286 23739 consensus_meta.cc:287] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.000s
W20260709 13:01:14.300630 23460 consensus_meta.cc:287] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.001s
I20260709 13:01:14.259732 23715 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:14.311775 23711 consensus_meta.cc:287] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.982s	user 0.014s	sys 0.000s
I20260709 13:01:14.301612 23460 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:14.317970 23702 consensus_meta.cc:287] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 1.114s	user 0.005s	sys 0.010s
I20260709 13:01:14.329886 23702 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:14.333681 23460 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:14.553347 23460 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.554602 20009 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:14.554452 23460 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:14.430083 19982 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.004s	sys 0.004s
W20260709 13:01:14.558274 19982 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.004s	sys 0.004s
W20260709 13:01:14.559018 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 785558us
W20260709 13:01:14.561345 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.807628 20008 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:14.564497 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1348) took 1143 ms. Trace:
I20260709 13:01:14.564927 20008 rpcz_store.cc:276] 0709 13:01:13.421369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.807060 (+385691us) service_pool.cc:224] Handling call
0709 13:01:13.807314 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.563649 (+756335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.564264 (+   615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:13.836180 20016 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:14.471364 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1053) took 1048 ms. Trace:
W20260709 13:01:14.493577 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.567131 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1349) took 1144 ms. Trace:
I20260709 13:01:14.567742 20016 rpcz_store.cc:276] 0709 13:01:13.422222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.835290 (+413068us) service_pool.cc:224] Handling call
0709 13:01:13.835660 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.566525 (+730865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.566928 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:14.506589 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.568605 20858 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:14.509091 20083 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:01:14.525615 20157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.003s	sys 0.035s
W20260709 13:01:14.572278 20157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.004s	sys 0.035s
I20260709 13:01:14.572824 23228 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:14.535028 20160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.001s	sys 0.004s
W20260709 13:01:14.573787 20160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.001s	sys 0.005s
I20260709 13:01:14.575107 20857 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:14.575625 20857 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:14.575699 23460 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:14.546044 22744 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.008s	sys 0.011s
W20260709 13:01:14.579730 22744 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.008s	sys 0.011s
I20260709 13:01:14.566511 20014 rpcz_store.cc:276] 0709 13:01:13.423232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.838820 (+415588us) service_pool.cc:224] Handling call
0709 13:01:14.469754 (+630934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.470522 (+   768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:14.470597 (+    75us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:14.470694 (+    97us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:14.470774 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.470890 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.471094 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:14.582510 23232 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.582638 23460 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:14.583513 23232 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.586731 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:14.587493 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:14.573918 23228 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:14.546999 23150 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:14.588783 20854 raft_consensus.cc:2410] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:01:14.589196 23150 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:14.548486 23375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.002s	sys 0.007s
W20260709 13:01:14.592257 23375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.003s	sys 0.008s
I20260709 13:01:14.542315 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1048) took 1271 ms. Trace:
I20260709 13:01:14.582521 23556 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.596467 20015 rpcz_store.cc:276] 0709 13:01:13.270317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.270454 (+   137us) service_pool.cc:224] Handling call
0709 13:01:13.270721 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.391716 (+120995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.541631 (+1149915us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400567f54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:14.542112 (+   481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276763008}
I20260709 13:01:14.597209 23556 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.598294 23150 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:14.577076 23580 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.600483 23556 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:14.545864 20852 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:14.601234 23558 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:14.601038 23580 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:14.601740 23558 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:14.591459 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:14.603461 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1059) took 1245 ms. Trace:
I20260709 13:01:14.604362 23558 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:14.624696 23558 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:14.625314 23700 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.648s	user 0.018s	sys 0.040s
W20260709 13:01:14.625702 23760 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.282s	user 0.010s	sys 0.004s
I20260709 13:01:14.588303 20855 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:14.609391 23580 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:14.583904 23460 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:14.319597 23624 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.845s	user 0.013s	sys 0.000s
I20260709 13:01:14.577761 20857 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:14.326699 23698 consensus_meta.cc:287] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.727s	user 0.012s	sys 0.000s
I20260709 13:01:14.337556 23711 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:14.321455 23715 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:14.335824 23723 consensus_meta.cc:287] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.000s
W20260709 13:01:14.617408 23759 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.361s	user 0.000s	sys 0.010s
I20260709 13:01:14.707731 23715 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:14.618656 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 632) took 1339 ms. Trace:
I20260709 13:01:14.710973 23627 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:14.711437 23715 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:14.711222 20090 rpcz_store.cc:276] 0709 13:01:13.278843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.278972 (+   129us) service_pool.cc:224] Handling call
0709 13:01:13.279300 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.289620 (+ 10320us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:13.289652 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.290375 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.429589 (+139214us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:13.429626 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.429712 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:14.617985 (+1188273us) raft_consensus.cc:1685] finished
0709 13:01:14.618031 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.618320 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9671,"raft.queue_time_us":12585,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6582,"thread_start_us":106673,"threads_started":1,"wal-append.queue_time_us":298562}
I20260709 13:01:14.615767 20852 rpcz_store.cc:276] 0709 13:01:13.358136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.361448 (+  3312us) service_pool.cc:224] Handling call
0709 13:01:13.361845 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.601598 (+1239753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.603103 (+  1505us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540027e2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:14.603366 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2721165184}
I20260709 13:01:14.586719 23232 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:14.664373 23745 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.840s	user 0.007s	sys 0.005s
W20260709 13:01:14.664959 23749 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.690s	user 0.002s	sys 0.009s
W20260709 13:01:14.665159 23754 log.cc:927] Time spent T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.357s	user 0.000s	sys 0.007s
W20260709 13:01:14.665838 23755 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.484s	user 0.003s	sys 0.008s
I20260709 13:01:14.672113 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 630) took 1401 ms. Trace:
I20260709 13:01:14.719239 23586 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.719018 20082 rpcz_store.cc:276] 0709 13:01:13.270794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.270907 (+   113us) service_pool.cc:224] Handling call
0709 13:01:13.272310 (+  1403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.272692 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.272725 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.273225 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.360701 (+ 87476us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:13.360739 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.360800 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:14.671471 (+1310671us) raft_consensus.cc:1685] finished
0709 13:01:14.671533 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.671820 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1755,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":87086,"threads_started":1,"wal-append.queue_time_us":165591}
I20260709 13:01:14.672883 23725 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.722363 23725 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:14.677518 23757 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.497s	user 0.004s	sys 0.007s
I20260709 13:01:14.724781 23719 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.678982 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:14.726197 20858 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:14.681138 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:14.727445 23624 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.683300 23609 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.729140 23609 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:14.730319 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.731470 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1384) took 1195 ms. Trace:
W20260709 13:01:14.684435 23721 log.cc:927] Time spent T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 1.393s	user 0.002s	sys 0.021s
W20260709 13:01:14.685335 20851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 1082) took 2601 ms (client timeout 2187 ms). Trace:
I20260709 13:01:14.669147 23460 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:14.686702 23696 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.596271 23593 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.699388 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:14.668890 23710 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:14.705812 23760 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.079s	user 0.001s	sys 0.002s
I20260709 13:01:14.737607 20856 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.708804 23723 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:14.738713 20857 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.733798 23722 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:14.740419 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:14.734611 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 631) took 1463 ms. Trace:
I20260709 13:01:14.741447 20083 rpcz_store.cc:276] 0709 13:01:13.278457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.278722 (+   265us) service_pool.cc:224] Handling call
0709 13:01:14.734289 (+1455567us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540078fd54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:14.734513 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2824609152}
I20260709 13:01:14.734514 23696 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.744349 23698 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:14.735723 20085 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:14.735023 23593 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:14.747255 23698 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:14.736642 23710 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:14.748159 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1378) took 1418 ms. Trace:
I20260709 13:01:14.748713 20085 rpcz_store.cc:276] 0709 13:01:13.329481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.329637 (+   156us) service_pool.cc:224] Handling call
0709 13:01:14.735253 (+1405616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:14.746476 (+ 11223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.747890 (+  1414us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540078fd50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:14.748099 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3231513856}
W20260709 13:01:14.732507 20851 rpcz_store.cc:269] 0709 13:01:12.084203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.084376 (+   173us) service_pool.cc:224] Handling call
0709 13:01:14.684796 (+2600420us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54000091d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:14.685075 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308230016}
I20260709 13:01:14.749679 23710 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:14.736346 20859 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:14.756369 23702 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:14.758985 23731 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:14.760442 23731 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:14.740830 23722 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.760830 23731 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:14.698534 23580 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.731889 20091 rpcz_store.cc:276] 0709 13:01:13.536483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.610617 (+ 74134us) service_pool.cc:224] Handling call
0709 13:01:13.610916 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.611232 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.611261 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.615144 (+  3883us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.702226 (+ 87082us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:13.702268 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.702332 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:14.731001 (+1028669us) raft_consensus.cc:1685] finished
0709 13:01:14.731037 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.731301 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5697,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8811,"thread_start_us":86656,"threads_started":1,"wal-append.queue_time_us":129494}
W20260709 13:01:14.769587 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 71477.7us
I20260709 13:01:14.730355 23624 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.716970 23715 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.778170 23764 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:01:14.778383 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:14.784793 23609 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:14.785735 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 749) took 1581 ms. Trace:
I20260709 13:01:14.786322 23593 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:14.792364 20128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.002s	sys 0.007s
W20260709 13:01:14.792912 20128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.002s	sys 0.007s
I20260709 13:01:14.794690 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.794886 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:14.796432 23729 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.243s	user 0.009s	sys 0.011s
I20260709 13:01:14.798650 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:14.799062 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:14.799439 20851 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:14.800514 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.800709 23702 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.801610 23702 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:14.801971 23723 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:14.802233 20160 raft_consensus.cc:1217] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Deduplicated request from leader. Original: 14.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 13:01:14.802699 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.802230 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.804085 20159 raft_consensus.cc:2410] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:14.802871 20160 raft_consensus.cc:1240] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:14.794744 20055 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:14.803545 20157 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:14.786492 20158 rpcz_store.cc:276] 0709 13:01:13.204348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.205172 (+   824us) service_pool.cc:224] Handling call
0709 13:01:13.212622 (+  7450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.311501 (+ 98879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.784861 (+1473360us) spinlock_profiling.cc:243] Waited 98.2 ms on lock 0x7b540028d7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:14.785404 (+   543us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225973504}
I20260709 13:01:14.807935 20012 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:14.779090 23228 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.815263 20855 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:01:14.803458 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:14.811714 20859 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.813634 23586 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:14.815830 20855 pending_rounds.cc:85] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 5
I20260709 13:01:14.813637 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:14.818779 20855 pending_rounds.cc:107] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:14.825399 20855 raft_consensus.cc:2889] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:14.826313 23586 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.826812 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:14.827507 23232 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:14.828203 23232 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:14.828760 23232 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.834357 23581 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.840075 23711 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:14.840648 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.840993 23711 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.842792 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:14.834619 23634 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:01:14.844280 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:14.844820 23627 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:14.845292 23627 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.845515 23634 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:14.845690 23627 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:14.846428 23634 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:14.847055 23521 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:14.847568 23521 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:14.858712 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:14.859789 20159 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:14.860137 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:14.861399 23443 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:14.862187 23443 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:14.835270 23580 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:14.848929 23521 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:14.831120 23715 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:01:14.843756 23711 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.855993 23710 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.855544 23586 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.879740 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:14.882076 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.882380 23443 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:14.886199 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.886945 20159 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:14.887396 23443 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.888127 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.890184 20162 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:14.891636 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.892835 20156 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:14.893070 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55579.2us
I20260709 13:01:14.894387 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.896210 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:14.897042 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.897853 20160 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:14.899055 23580 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.900848 23634 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:14.904173 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 61554.7us
I20260709 13:01:14.904692 23731 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:14.908341 20744 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:14.909374 23722 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.920722 20859 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:14.923583 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:14.935148 23687 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:14.935441 23598 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.936183 23598 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.928795 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:14.938231 20164 raft_consensus.cc:2410] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:01:14.938961 23598 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:14.929208 23697 raft_consensus.cc:3055] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:14.940107 23697 raft_consensus.cc:740] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:14.941329 23697 consensus_queue.cc:260] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:14.895638 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:14.942931 23435 raft_consensus.cc:3055] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:14.943454 23634 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:14.943969 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48640.4us
I20260709 13:01:14.931772 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:14.878014 23725 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:14.919546 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:14.946295 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 65102.4us
W20260709 13:01:14.954974 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47113.2us
I20260709 13:01:14.888152 23443 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:14.957922 23611 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:14.961766 23611 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.962061 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 630) took 1064 ms. Trace:
I20260709 13:01:14.945065 23687 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.963528 23443 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:14.962564 20013 rpcz_store.cc:276] 0709 13:01:13.775471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.539672 (+764201us) service_pool.cc:224] Handling call
0709 13:01:14.541006 (+  1334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.541584 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:14.542207 (+   623us) log.cc:844] Serialized 14 byte log entry
0709 13:01:14.838274 (+296067us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:14.838343 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:14.838427 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.838501 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.839638 (+  1137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4872,"prepare.queue_time_us":88921,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":9142,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":1003,"thread_start_us":287237,"threads_started":1,"wal-append.queue_time_us":280455}
I20260709 13:01:14.962167 23611 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.965747 20157 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:14.972374 23521 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.943445 23435 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:14.977159 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:01:14.980338 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.980829 23702 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:14.985508 20743 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:14.988184 23609 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.990397 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:14.992699 20012 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:14.992882 20010 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:14.996199 23725 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:14.996685 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 119168us
I20260709 13:01:14.982218 23604 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:15.024649 23702 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:14.959133 20744 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:15.026845 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 67813.6us
I20260709 13:01:15.027590 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:15.029325 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:15.029894 23766 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.369s	user 0.002s	sys 0.000s
I20260709 13:01:15.020124 20856 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:01:15.030948 23666 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:01:15.000770 23697 raft_consensus.cc:3060] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:15.012349 20012 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:15.032500 20012 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:15.027626 23764 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:15.033943 23764 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:15.035153 23764 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.036479 23604 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:15.037031 20157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
 from {username='slave'} at 127.0.0.1:50786
I20260709 13:01:15.038210 23604 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:15.038254 20157 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:15.038689 23604 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:15.039433 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 1079) took 1633 ms. Trace:
I20260709 13:01:15.039845 20856 rpcz_store.cc:276] 0709 13:01:13.405888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.413662 (+  7774us) service_pool.cc:224] Handling call
0709 13:01:13.413977 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.736291 (+1322314us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b58007406d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:01:15.030637 (+294346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.039360 (+  8723us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902766336}
I20260709 13:01:15.041422 20162 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:15.042707 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.044373 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.045249 20162 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:15.016292 23720 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:15.014891 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.016705 20164 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:15.046288 20161 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:15.020354 23042 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:15.046968 23042 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:15.036234 23656 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.019327 20011 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:15.044417 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.048950 20159 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:15.048144 23656 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.022992 23609 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.050479 23150 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:15.021970 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:15.025264 23708 consensus_queue.cc:1059] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:15.052456 23708 raft_consensus.cc:3055] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:15.048334 23042 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.052915 23708 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:15.015012 23435 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.053907 23435 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.053942 23708 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.054809 23731 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.055567 23708 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.024693 23460 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:15.055826 23609 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:15.057145 23443 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:15.059186 23696 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.059216 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.025873 23586 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.050024 23656 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.063606 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.063832 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.064896 20857 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:15.064965 20091 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:15.065838 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 1091) took 1257 ms. Trace:
W20260709 13:01:15.066172 23659 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 13:01:15.066473 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 44649.5us
I20260709 13:01:15.067212 23764 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.052177 23150 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:15.045199 20156 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:14.982173 23232 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.033936 20012 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.038825 20157 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.062896 23711 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:15.081027 23711 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:15.081527 23711 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:15.082470 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 82130.3us
I20260709 13:01:15.062703 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.062633 23586 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.085700 20165 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:15.086403 23586 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:15.069824 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:15.071010 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1217) took 1646 ms. Trace:
I20260709 13:01:15.025074 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.066215 20857 rpcz_store.cc:276] 0709 13:01:13.808338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.808788 (+   450us) service_pool.cc:224] Handling call
0709 13:01:13.809055 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:14.574768 (+765713us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b54003a50a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 13:01:15.065504 (+490736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.065778 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74576640}
I20260709 13:01:15.008173 23581 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:15.083566 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:15.089061 20088 rpcz_store.cc:276] 0709 13:01:13.424116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.482129 (+ 58013us) service_pool.cc:224] Handling call
0709 13:01:13.545765 (+ 63636us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:13.556561 (+ 10796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.556590 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:13.594343 (+ 37753us) log.cc:844] Serialized 26 byte log entry
0709 13:01:13.730492 (+136149us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:13.751150 (+ 20658us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.751351 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.751426 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:15.067848 (+1316422us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b1800216520. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:15.070486 (+  2638us) raft_consensus.cc:1685] finished
0709 13:01:15.070621 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:15.070796 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24200,"prepare.queue_time_us":20514,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4249,"raft.queue_time_us":11040,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":58957,"spinlock_wait_cycles":26147200,"thread_start_us":39893,"threads_started":1,"wal-append.queue_time_us":141017}
I20260709 13:01:15.092844 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.058569 23443 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:15.087751 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1225) took 1301 ms. Trace:
I20260709 13:01:15.098170 20165 rpcz_store.cc:276] 0709 13:01:13.786542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.786650 (+   108us) service_pool.cc:224] Handling call
0709 13:01:15.087547 (+1300897us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b540001a9d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:15.087674 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121778944}
W20260709 13:01:15.101053 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.036191 23627 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.097958 23443 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:15.116508 23627 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:15.118185 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.118919 20156 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
W20260709 13:01:15.120057 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 93463us
I20260709 13:01:15.120277 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.124564 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.126664 23443 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.128293 23702 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:15.131577 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:15.131951 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.132339 20090 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.132695 20082 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:15.133999 23611 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.136555 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.137326 20082 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:15.060572 23604 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.131577 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.146344 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:15.138157 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 54783.5us
I20260709 13:01:15.160454 19835 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:15.138067 23232 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:15.165892 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59708.2us
I20260709 13:01:15.128355 23443 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.174793 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.189284 20744 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.188266 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:15.153656 23581 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.192209 20856 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:15.192749 20857 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:15.195442 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.198761 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.199645 20012 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:01:15.201951 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.151397 23627 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:01:15.203245 20011 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:15.201838 20012 pending_rounds.cc:85] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 5
I20260709 13:01:15.214023 20012 pending_rounds.cc:107] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:15.214442 20012 raft_consensus.cc:2889] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:15.216236 20085 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:15.221774 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.191555 23611 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.198011 23710 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:15.228905 20850 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:01:15.231051 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.203974 23723 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.222882 20855 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:15.234198 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.236521 20090 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:15.236922 20856 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
W20260709 13:01:15.234604 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 164940us
I20260709 13:01:15.234989 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.237067 20165 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.239868 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.245440 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:15.224133 23698 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:15.191803 23666 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:01:15.232124 23710 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.235586 23723 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.252100 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:15.252496 20012 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.256316 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.239205 23697 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.211724 23725 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:01:15.248513 23698 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:15.263720 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80743us
W20260709 13:01:15.266194 23776 log.cc:927] Time spent T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:01:15.253299 23710 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.251617 23723 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.274037 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.316813 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.317957 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.319255 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.273425 23698 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:15.269778 23697 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.329312 20091 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:15.260725 23558 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:01:15.332180 23521 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:15.332643 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:15.333220 23521 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:15.334378 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:15.335587 23652 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:15.336422 23652 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:15.336215 23711 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.342092 23711 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:15.263080 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.284540 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:15.286414 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:15.287665 20015 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:01:15.343832 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 47946.9us
I20260709 13:01:15.289310 20854 raft_consensus.cc:2410] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:01:15.293198 23775 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:15.275506 23611 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:15.356537 20012 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:01:15.305887 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.258561 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.318367 20087 raft_consensus.cc:2410] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:01:15.279474 20157 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:15.345237 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 88783.1us
I20260709 13:01:15.357966 20090 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:15.360157 20165 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:15.328485 23697 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.363183 20089 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.357017 20012 pending_rounds.cc:85] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 6
I20260709 13:01:15.369318 20012 pending_rounds.cc:107] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:01:15.369625 20012 raft_consensus.cc:2889] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:15.378901 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.384044 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.363276 20157 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:15.393105 23659 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:15.393882 23659 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.395025 20164 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:15.395337 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.386610 23042 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:01:15.401223 23627 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:15.402159 23627 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.286577 23656 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:15.405206 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46031.4us
I20260709 13:01:15.356663 23611 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:15.412596 20089 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:15.390283 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:15.388927 20013 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:15.428598 19835 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:15.386204 23521 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.419968 23711 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:15.435283 19835 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:15.436093 23711 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:15.437232 19835 catalog_manager.cc:5697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 4 to 7, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 7 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:15.439164 19835 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:15.448135 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:15.441045 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:15.451624 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 66625.3us
I20260709 13:01:15.403535 23696 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.453033 20159 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:15.457124 20157 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:15.456373 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 90895.1us
I20260709 13:01:15.466128 23650 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:01:15.466751 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.467293 23650 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.467751 23650 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:15.468801 23650 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.470121 23650 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.470489 20082 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:15.471971 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.472445 23616 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:15.467530 20090 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:15.452507 23558 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.474398 20744 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.474704 23558 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.474428 23581 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:15.476563 23581 consensus_queue.cc:1243] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:15.476866 23558 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:15.476130 23435 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:15.477386 23580 consensus_queue.cc:1175] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e5eb27db496b47129df3e689e546d94a" responder_term: 7 status { last_received { term: 7 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 7 index: 5 } }
I20260709 13:01:15.478576 23435 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:15.479620 23435 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.481979 23627 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:15.457713 23042 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:15.473018 23616 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:15.485975 20163 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.487372 23616 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:15.485513 23659 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.472960 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.489120 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.490015 23094 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:15.490466 23094 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.490223 23611 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.490867 23094 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.491430 23611 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:15.493410 23443 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.496260 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.518039 20850 raft_consensus.cc:1240] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:15.497030 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.473412 23521 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:15.519001 20858 raft_consensus.cc:2410] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:15.519129 23521 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:15.497754 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:15.519543 23521 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.499586 20017 raft_consensus.cc:2410] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:15.520124 20852 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:15.500188 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.501686 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.506651 23604 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.509670 20130 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:15.522042 20164 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:15.511317 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.512315 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.539759 20089 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:15.539894 20085 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:15.514606 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:15.515972 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.516829 23725 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:15.541064 20090 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:15.449607 23150 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:01:15.542424 23725 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.542891 23725 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:15.543156 23150 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:15.543473 23725 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.517983 20851 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:15.543712 23150 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:15.500967 23764 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.456022 23696 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:15.545032 20854 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:15.544771 23764 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.545711 23696 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.546725 23696 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.547048 23764 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:15.548432 23659 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.549039 23659 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.550421 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.551223 20856 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:15.528851 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.521282 23443 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:15.555979 23722 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:15.556160 23708 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:15.556593 23722 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.556831 23708 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:15.533732 23739 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:15.561281 23739 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:15.562302 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.537258 20859 raft_consensus.cc:1275] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:15.536460 23723 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.500519 23656 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.538968 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:15.522500 19835 catalog_manager.cc:5697] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:15.465169 23758 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:01:15.537603 23593 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:15.452847 23711 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:15.544813 23150 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:15.525851 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:15.557595 23443 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.565389 20857 raft_consensus.cc:2393] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:15.572546 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.569474 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.592425 20164 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:15.571964 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:15.572283 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:15.579335 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.565076 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.594965 20159 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.595670 23656 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:15.596186 23656 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.596522 23656 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.564173 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.588168 19835 catalog_manager.cc:5697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:15.600502 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 74792.5us
I20260709 13:01:15.604048 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.604842 20160 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.612002 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:15.551347 23659 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:15.618522 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:15.559463 23708 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:15.564731 23580 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:15.641227 20851 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.619520 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.649562 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:15.648780 20857 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:15.639577 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.656800 20014 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:15.570420 23624 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:01:15.639938 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.643253 19835 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 10, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 10 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:15.547744 23094 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.572656 23723 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.724538 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.725443 20090 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:15.657078 20852 pending_rounds.cc:85] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 6
I20260709 13:01:15.727948 20852 pending_rounds.cc:107] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:15.728325 20852 raft_consensus.cc:2889] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:15.676468 23652 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:15.562923 23739 consensus_queue.cc:1059] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:01:15.680599 20091 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:15.680755 23793 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:15.684316 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:15.688408 23042 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:15.688984 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.692034 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.694789 20088 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:15.701407 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.704038 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.707125 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.708153 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.710209 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.745157 20015 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:15.712548 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.713546 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.716373 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.717208 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.717941 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:15.566918 23598 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:01:15.577073 23698 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:15.566639 23758 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:15.748998 20086 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:15.639938 23720 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.606681 23731 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:15.651548 23580 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:15.614867 23702 consensus_queue.cc:1059] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
W20260709 13:01:15.741796 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 187007us
I20260709 13:01:15.749181 20157 raft_consensus.cc:2393] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:15.782589 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:15.782630 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 170757us
I20260709 13:01:15.784574 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:15.743772 23793 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:15.723249 23150 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.753242 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.754558 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:15.759423 20857 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:15.757401 19835 catalog_manager.cc:5697] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:15.770054 20008 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:15.788075 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:15.788703 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 179451us
I20260709 13:01:15.730839 20159 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:15.743578 23042 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:15.748729 23758 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.812310 23764 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:15.812505 23758 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.812906 23764 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.812644 23580 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.813344 23764 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:15.813702 23698 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:15.814138 23580 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.816118 20083 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:15.816365 23785 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:01:15.817231 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.817643 23094 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:15.818027 23722 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:15.820083 20085 raft_consensus.cc:2368] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11.
I20260709 13:01:15.820410 20855 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:15.757982 23652 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:15.803013 20744 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:15.801146 20088 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:15.805752 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:15.809372 20130 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:15.811985 23042 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:15.839001 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.843725 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.845095 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:15.846633 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:15.847394 20159 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:15.851006 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.800086 23793 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.859598 20090 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:01:15.866074 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.863102 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:15.868778 20086 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:15.867764 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.808413 23460 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:15.770510 23720 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:15.879083 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:15.824452 23652 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:15.885488 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:15.846175 23722 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:15.853214 23723 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.882525 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:15.807968 23710 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:15.899572 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:15.953255 20082 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:01:15.958598 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.949695 20859 raft_consensus.cc:2410] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:15.857084 23624 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:01:15.876794 23581 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:15.963826 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 98724.5us
I20260709 13:01:15.887176 23611 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:15.888494 23460 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:15.933286 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:15.857311 23708 consensus_queue.cc:1048] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 13:01:15.971874 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61765.9us
I20260709 13:01:15.893587 23698 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:15.895879 23722 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:15.983646 20083 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:15.988379 20008 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:01:15.989405 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 370829us
I20260709 13:01:15.953850 23723 raft_consensus.cc:697] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:15.947844 23794 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:16.145542 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:15.887533 23521 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.015431 20851 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:16.004159 23652 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:16.044766 23797 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:01:16.055540 23726 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.195s
I20260709 13:01:16.169390 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:16.170084 20850 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:16.067631 23786 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.334s	user 0.002s	sys 0.000s
I20260709 13:01:16.085119 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:16.113711 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 241994us
I20260709 13:01:16.114108 20159 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:16.116222 23581 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:16.172839 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 87836.3us
I20260709 13:01:16.197319 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:16.198403 23042 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.166962 23604 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:16.198964 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:16.327862 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.175359 23460 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:16.180848 20743 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:16.324672 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.333567 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 152846us
W20260709 13:01:16.182534 23656 consensus_meta.cc:287] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.585s	user 0.007s	sys 0.006s
I20260709 13:01:16.200047 23723 consensus_queue.cc:237] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.162165 20854 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:01:16.170433 23697 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:16.232508 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 339423us
W20260709 13:01:16.283296 20017 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.013s	sys 0.010s
W20260709 13:01:16.341846 20017 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.013s	sys 0.017s
W20260709 13:01:16.252813 20160 consensus_meta.cc:287] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.003s
I20260709 13:01:16.173035 23739 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:16.192449 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:16.200673 20850 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
W20260709 13:01:16.231980 23696 consensus_meta.cc:287] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.685s	user 0.014s	sys 0.000s
I20260709 13:01:16.188599 23581 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:16.166170 19835 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 7, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 7 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:16.254093 23792 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.270s	user 0.000s	sys 0.003s
I20260709 13:01:16.148308 23802 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.216053 23652 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.202178 23625 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:16.216531 20084 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.201617 23720 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:16.189462 23710 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:16.343449 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:16.343907 20160 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:16.217993 23521 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:16.345963 23739 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:16.177325 20089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:16.164099 23722 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.347599 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.351531 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:16.357126 20857 raft_consensus.cc:2410] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:01:16.358632 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 193587us
W20260709 13:01:16.374840 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 175934us
I20260709 13:01:16.170526 23794 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.239146 23580 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:16.348963 23698 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:16.334766 23656 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.380781 20089 raft_consensus.cc:606] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.358273 23094 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:16.356019 23435 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:01:16.358315 23809 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:16.445477 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 102125us
W20260709 13:01:16.446836 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 99366.1us
I20260709 13:01:16.365589 23625 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:01:16.456507 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 264164us
I20260709 13:01:16.380424 23805 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.454648 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.438057 23697 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:16.389145 23739 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.456871 23659 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:01:16.471103 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.475345 20744 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.379998 23802 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.476897 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:16.463861 23435 consensus_queue.cc:1243] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:16.476291 23809 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:16.479677 23521 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:16.475440 23650 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 1.005s	user 0.016s	sys 0.000s
W20260709 13:01:16.475634 23758 consensus_meta.cc:287] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.661s	user 0.003s	sys 0.010s
W20260709 13:01:16.477265 23793 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.003s
W20260709 13:01:16.480119 23764 consensus_meta.cc:287] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.006s
I20260709 13:01:16.475968 23659 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.487304 20055 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.485796 23521 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:16.499588 23521 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.499799 23818 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:16.500475 23818 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:16.474633 23805 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.501461 23818 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.503299 23805 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:16.504118 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.505026 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.512807 20744 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.513190 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:16.496476 23803 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:01:16.525928 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:16.532090 20850 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:16.509006 23659 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:16.543987 23812 log.cc:927] Time spent T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:01:16.543057 23794 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:16.523327 23042 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:16.534061 23826 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:16.571674 23725 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.957s	user 0.016s	sys 0.000s
I20260709 13:01:16.573450 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.509392 23793 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.498824 23758 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.511257 23764 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.569902 23094 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:16.590684 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 77804.6us
I20260709 13:01:16.574925 23798 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.509637 23625 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.584100 23825 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.574409 23697 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:16.600680 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.601398 20016 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:16.588347 23826 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:16.591605 23827 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:16.608274 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 95561.8us
I20260709 13:01:16.589267 23725 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:16.660339 20089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:16.662874 20089 raft_consensus.cc:606] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.663344 20089 raft_consensus.cc:614] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:16.659790 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 134050us
I20260709 13:01:16.605997 23806 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:16.699364 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:16.700469 20158 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:16.701404 23826 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.703291 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:16.705538 23150 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.886s	user 0.004s	sys 0.008s
I20260709 13:01:16.707149 23822 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:16.707669 23822 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:16.707845 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:16.708184 23822 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.708413 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:16.707051 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:16.713132 23829 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.713768 23829 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.716517 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:16.716852 20009 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:16.718236 20083 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:16.718597 23819 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:01:16.611754 23625 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.599510 23798 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.725216 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:16.723308 20013 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:16.725585 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1255) took 1147 ms. Trace:
I20260709 13:01:16.728083 20156 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:16.569934 23809 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:16.730811 20010 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 7.
W20260709 13:01:16.734484 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 229558us
I20260709 13:01:16.736804 23827 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:01:16.737810 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 71232.4us
I20260709 13:01:16.737916 23827 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.739001 20853 raft_consensus.cc:1240] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:01:16.737833 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:16.740309 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:16.741542 23828 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:16.741643 20743 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:16.742162 23828 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.742684 23828 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:16.694144 23656 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:16.723636 23521 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.745278 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.745793 23094 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:16.746264 23831 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:01:16.746577 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.747094 23580 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:16.747639 23794 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.744550 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:16.757588 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.727581 23042 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:16.731387 20161 rpcz_store.cc:276] 0709 13:01:15.578307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.578465 (+   158us) service_pool.cc:224] Handling call
0709 13:01:15.603579 (+ 25114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.011223 (+407644us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:16.061839 (+ 50616us) log.cc:844] Serialized 14 byte log entry
0709 13:01:16.284044 (+222205us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:16.331270 (+ 47226us) log.cc:844] Serialized 26 byte log entry
0709 13:01:16.331515 (+   245us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:16.331564 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.331661 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:16.725357 (+393696us) raft_consensus.cc:1685] finished
0709 13:01:16.725379 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:16.725521 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32030,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":72360,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":34474,"thread_start_us":545180,"threads_started":2,"wal-append.queue_time_us":284423}
I20260709 13:01:16.751873 23652 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:16.768347 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1225) took 1090 ms. Trace:
I20260709 13:01:16.769783 20082 rpcz_store.cc:276] 0709 13:01:15.677876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.689802 (+ 11926us) service_pool.cc:224] Handling call
0709 13:01:15.690222 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.973531 (+283309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.765746 (+792215us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54005a1c54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:16.768292 (+  2546us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603859712}
I20260709 13:01:16.752480 23460 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:16.767848 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:16.744208 23829 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:16.765729 23650 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.781635 23521 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:16.782140 23822 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.783275 20014 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:16.784458 23822 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:16.785935 23791 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.740s	user 0.000s	sys 0.009s
I20260709 13:01:16.787678 19835 catalog_manager.cc:5697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 9, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
W20260709 13:01:16.793982 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 48803.5us
I20260709 13:01:16.797279 23758 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:16.772807 23818 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:16.798645 23764 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:16.799491 23793 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:16.800290 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 53799.5us
I20260709 13:01:16.765125 23094 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.801221 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:16.763139 23580 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.803012 20158 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:16.815864 20089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:16.816454 20089 raft_consensus.cc:606] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.816776 20089 raft_consensus.cc:614] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:16.788769 23697 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.818990 23580 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:16.821273 23725 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:16.822834 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 1149) took 1107 ms. Trace:
I20260709 13:01:16.812659 23460 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.827649 23460 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.829957 23460 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:16.830788 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:16.819046 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:01:16.833429 20744 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.833503 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:16.834908 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.836028 23558 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.836078 23832 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.836809 23558 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:16.836745 23832 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.837414 23558 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:16.837546 23829 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.838109 23829 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.838430 23558 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.839123 23832 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:16.839254 23611 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:16.840719 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:16.842324 23807 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.482s	user 0.001s	sys 0.001s
I20260709 13:01:16.842506 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:16.811791 23818 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:16.843561 20855 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:16.843430 23818 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:16.766280 23042 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.765836 23794 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.845099 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 1110) took 1105 ms. Trace:
I20260709 13:01:16.812430 23813 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.863617 20856 rpcz_store.cc:276] 0709 13:01:15.739674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.748303 (+  8629us) service_pool.cc:224] Handling call
0709 13:01:15.754404 (+  6101us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:15.779110 (+ 24706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:15.779140 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:15.837107 (+ 57967us) log.cc:844] Serialized 52 byte log entry
0709 13:01:16.076306 (+239199us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:16.120917 (+ 44611us) log.cc:844] Serialized 14 byte log entry
0709 13:01:16.167610 (+ 46693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:16.167714 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:16.797517 (+629803us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b18000b8e40. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:16.841601 (+ 44084us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:16.844868 (+  3267us) raft_consensus.cc:1685] finished
0709 13:01:16.844888 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:16.845043 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99140,"prepare.queue_time_us":34914,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":72778,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":44894,"spinlock_wait_cycles":73721344,"thread_start_us":176114,"threads_started":1,"wal-append.queue_time_us":409401}
W20260709 13:01:16.836947 20013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 1370) took 1265 ms (client timeout 1647 ms). Trace:
I20260709 13:01:16.800741 23833 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:16.866240 20013 rpcz_store.cc:269] 0709 13:01:15.571555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.571680 (+   125us) service_pool.cc:224] Handling call
0709 13:01:16.836744 (+1265064us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b5400527bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:16.836868 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2080176000}
I20260709 13:01:16.845923 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:16.847978 20161 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:16.839704 23611 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.823385 20156 rpcz_store.cc:276] 0709 13:01:15.715210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.716101 (+   891us) service_pool.cc:224] Handling call
0709 13:01:16.822603 (+1106502us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b5400449b54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:16.822756 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257975680}
I20260709 13:01:16.852717 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:16.868268 20859 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:16.773902 23652 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:16.848862 23796 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.856576 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.815335 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.862509 20084 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:16.869975 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:16.849493 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:16.874970 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:16.811616 23793 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:01:16.892726 23818 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:16.894224 23820 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:16.894815 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:16.896109 20158 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:16.896642 20856 raft_consensus.cc:2410] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:01:16.878062 23833 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.870497 23611 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:16.904450 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.564s (SENT) [suppressed 18 similar messages]
I20260709 13:01:16.906102 23232 raft_consensus.cc:3055] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:16.906603 23232 raft_consensus.cc:740] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:16.907547 23232 consensus_queue.cc:260] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.909170 23232 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.910017 23798 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:16.913491 23802 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:16.914815 20012 raft_consensus.cc:3055] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:16.915241 20012 raft_consensus.cc:740] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:16.915964 20012 consensus_queue.cc:260] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:16.917069 20012 raft_consensus.cc:3060] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.869103 23829 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:16.900542 23834 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:16.901273 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 53484.6us
I20260709 13:01:16.922076 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:16.899957 23840 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:01:16.927306 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 112077us
I20260709 13:01:16.931519 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.932129 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:16.936734 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.938972 20012 raft_consensus.cc:2468] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:16.940032 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34530 (request call id 1371) took 1368 ms. Trace:
I20260709 13:01:16.944804 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:16.946362 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:16.947038 20013 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:16.948282 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:16.952668 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:16.953265 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:16.951650 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:16.953866 20015 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:16.958026 22744 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:16.958467 22744 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.958871 22744 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:16.934448 23802 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.934288 23834 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:16.960688 20014 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:16.965781 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.936255 23829 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:16.928310 23833 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:16.954144 23731 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:01:16.965183 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:16.966800 23826 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:16.940521 20012 rpcz_store.cc:276] 0709 13:01:15.571841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.572020 (+   179us) service_pool.cc:224] Handling call
0709 13:01:16.914577 (+1342557us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b540046c920. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b5e18 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:16.939903 (+ 25326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701131136}
I20260709 13:01:16.963179 23711 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:16.967708 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:16.954694 23823 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:16.920692 23796 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:16.978327 23828 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:16.979391 23796 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:16.979934 23711 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.980690 23042 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:16.982069 23711 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:16.974295 23826 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.982951 23826 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:16.873152 23813 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:16.984483 20159 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:16.985595 23813 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:16.986389 23739 consensus_meta.cc:287] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.004s
I20260709 13:01:16.986806 23739 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:16.970525 23443 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.967258 23834 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.979499 20852 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:16.975696 23794 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:16.991526 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 87097.1us
I20260709 13:01:16.992648 20017 raft_consensus.cc:2393] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:16.968638 23829 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.992693 23833 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:17.001645 23829 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.994201 23823 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.013110 23702 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:17.016667 23823 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:17.019088 23722 consensus_meta.cc:287] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:01:17.019573 23722 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.007841 23042 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:01:17.011611 23833 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.030189 22744 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.035463 20082 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:17.015336 23831 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:17.038959 23739 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:17.047674 22744 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.043432 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:17.036636 23625 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.043849 23831 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.034864 23833 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:17.062407 20857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 1071) took 1219 ms (client timeout 1592 ms). Trace:
W20260709 13:01:17.057065 20084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 661) took 1546 ms (client timeout 1564 ms). Trace:
W20260709 13:01:17.063998 20857 rpcz_store.cc:269] 0709 13:01:15.843028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.858883 (+ 15855us) service_pool.cc:224] Handling call
0709 13:01:17.062168 (+1203285us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54000032d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:17.062325 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053121792}
I20260709 13:01:17.035674 23094 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.037504 23828 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:17.052373 23625 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.067114 23659 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:17.064821 20084 rpcz_store.cc:269] 0709 13:01:15.511010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.514229 (+  3219us) service_pool.cc:224] Handling call
0709 13:01:17.056769 (+1542540us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54009803d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:17.056976 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553823616}
I20260709 13:01:17.072145 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:17.072346 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.072844 20856 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.073524 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:17.074486 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.074640 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.028110 23702 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.077387 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.067767 23625 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.079229 23697 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.079697 23697 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.080094 23697 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.080598 23702 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:17.084215 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.086509 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:17.089951 20856 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:17.092473 20851 raft_consensus.cc:2368] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:17.092877 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:17.093958 23794 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:17.094573 23794 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.096741 23794 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.097245 23794 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:17.098531 23794 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:17.099179 23794 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.077544 23232 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:17.101629 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.047904 23652 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.101925 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:17.089258 23799 log.cc:927] Time spent T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.051s	user 0.001s	sys 0.003s
I20260709 13:01:17.101814 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.102660 20855 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:17.055784 23795 raft_consensus.cc:993] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:17.104756 23795 raft_consensus.cc:1081] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:17.106592 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1382) took 1258 ms. Trace:
I20260709 13:01:17.106333 23834 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:17.107054 20016 rpcz_store.cc:276] 0709 13:01:15.851049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.897938 (+ 46889us) service_pool.cc:224] Handling call
0709 13:01:15.957148 (+ 59210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.604364 (+647216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.106332 (+501968us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54006220d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:17.106530 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481044224}
I20260709 13:01:17.101373 23794 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.109046 23826 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.110369 20859 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:17.048979 23722 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.114205 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.113667 23720 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.116581 23720 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:17.119170 23720 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.125706 23443 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:17.126960 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:17.127907 20130 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:17.128492 23659 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.130070 23826 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:17.111255 23558 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:01:17.131265 23624 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:17.132105 23625 raft_consensus.cc:2804] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:17.132231 23823 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.113832 23831 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:17.132673 23625 raft_consensus.cc:697] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:17.133764 23625 consensus_queue.cc:237] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.136142 23833 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:17.136914 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.138792 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 98841.4us
I20260709 13:01:17.140448 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1074) took 1243 ms. Trace:
I20260709 13:01:17.145766 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:17.147018 20159 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.140750 20854 rpcz_store.cc:276] 0709 13:01:15.897291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.897488 (+   197us) service_pool.cc:224] Handling call
0709 13:01:15.951512 (+ 54024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.339709 (+388197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.140260 (+800551us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54000046d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:17.140399 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483612544}
I20260709 13:01:17.150645 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:17.152267 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:17.155440 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:17.155998 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.156533 20082 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:17.131765 23624 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:17.157424 20162 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:17.158444 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:17.160192 20856 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:17.161293 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:17.162946 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:17.165745 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.170643 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:17.171753 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:17.172389 20160 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:17.175061 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.176142 20850 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:17.186575 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:17.112625 23150 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:17.191823 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:17.185000 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.197925 23834 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.221767 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.198454 20011 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:17.136183 23813 consensus_queue.cc:1059] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:01:17.159322 23696 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:17.201743 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:17.158318 23624 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.212106 23611 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.213709 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.215866 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.241784 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:17.216159 23460 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:17.217097 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:17.219964 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.199536 20016 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:17.171191 23652 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:17.224583 20160 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:17.220857 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:17.159996 23809 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:17.230515 19835 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:17.232095 20159 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
W20260709 13:01:17.236461 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 120417us
I20260709 13:01:17.158185 23829 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.238523 20009 raft_consensus.cc:1217] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[7.5-11.7]   Dedup: 7.5->[9.6-11.7]
I20260709 13:01:17.257090 20009 pending_rounds.cc:85] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 5
I20260709 13:01:17.257349 20009 pending_rounds.cc:107] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:01:17.257588 20009 raft_consensus.cc:2889] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:17.239230 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.240478 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:17.241405 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.258912 20014 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:17.242311 20857 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:17.246894 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.247752 20161 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:17.248660 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.253623 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.259810 20015 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:17.260730 20012 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:17.254415 20852 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:17.262996 23809 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:17.246671 23460 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.264182 23809 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.249588 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.267709 20856 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
W20260709 13:01:17.268587 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.272894 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.249406 23834 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:17.274282 23722 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:17.274886 23722 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.277086 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:17.277570 19835 catalog_manager.cc:5697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:17.280144 20014 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:17.280273 20855 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:17.284207 20165 raft_consensus.cc:1240] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:17.284314 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:17.283806 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 70182.8us
I20260709 13:01:17.291009 20012 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:17.292398 19835 catalog_manager.cc:5697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 9, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 9 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.231994 23150 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.293643 20008 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:17.293645 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.296818 20856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44216
I20260709 13:01:17.297501 20856 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:17.297854 20856 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.299082 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:17.279124 23846 log.cc:927] Time spent T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:01:17.242321 23696 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.302707 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:17.254073 23813 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:17.317574 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:17.278934 23558 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:17.305899 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.321425 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:17.308224 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 65441.1us
I20260709 13:01:17.322520 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.323977 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.312870 20055 leader_election.cc:400] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:17.284130 23820 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:17.252559 23652 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:17.330297 19838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 159) took 25914 ms (client timeout 15000 ms). Trace:
I20260709 13:01:17.329432 23798 raft_consensus.cc:3055] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:17.353226 20015 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:17.325130 20055 leader_election.cc:403] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:01:17.291966 23833 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:17.320966 23834 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:01:17.332237 23697 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.367782 23828 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:17.369583 23697 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.370172 22744 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:17.370994 22744 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:17.371030 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.371533 22744 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:17.373018 23711 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:17.372594 22744 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.373468 23711 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.373873 23711 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:17.374277 22744 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:17.292868 23725 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:01:17.305258 23731 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:17.353314 23593 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:17.363137 23820 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.364926 23802 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:17.365274 23652 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.365434 23659 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:17.354108 23813 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:17.366847 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 71738.2us
I20260709 13:01:17.357226 23150 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:17.361434 19835 catalog_manager.cc:5697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 12, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 12 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:17.353478 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.391021 20014 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:17.438957 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.347173 20856 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:17.409989 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.430176 20161 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:01:17.433135 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 100055us
I20260709 13:01:17.443367 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:17.359643 23611 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.439869 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.352954 23798 raft_consensus.cc:740] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:17.367830 23443 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:17.453871 20856 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:17.441535 23829 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.449646 23521 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:17.437073 23558 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:17.413655 23726 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:01:17.460997 23831 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:17.461555 23834 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:17.472038 23795 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:01:17.475895 23232 consensus_queue.cc:1059] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:17.451807 23652 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:17.448943 23722 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.481742 19835 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 9, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 9 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:17.448167 23659 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.527895 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:17.529639 20014 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
W20260709 13:01:17.527895 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 84626.2us
I20260709 13:01:17.532498 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:17.481922 23521 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:17.483734 23696 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.490471 23042 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.567660 19835 catalog_manager.cc:5697] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 10. New cstate: current_term: 10 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:17.524751 23652 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.516520 23094 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:17.573422 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.578415 19835 catalog_manager.cc:5697] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:17.580928 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:17.581820 22744 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:17.582276 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.548899 23758 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:17.565706 23521 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:17.585067 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.600934 22744 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:17.601737 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.590377 19835 catalog_manager.cc:5697] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 8, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 8 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.594779 20016 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:17.576686 23652 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.624963 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:17.354148 19838 rpcz_store.cc:269] 0709 13:00:51.416099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.421319 (+  5220us) service_pool.cc:224] Handling call
0709 13:01:17.329982 (+25908663us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:10.809576 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:10.877309 (+ 67733us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:10.877347 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:01:10.877366 (+    19us) tablet.cc:662] Decoding operations
0709 13:01:10.924705 (+ 47339us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:10.924787 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:10.924831 (+    44us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:01:10.927204 (+  2373us) tablet.cc:701] Row locks acquired
0709 13:01:10.927260 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:01:10.927478 (+   218us) write_op.cc:270] Start()
0709 13:01:10.927797 (+   319us) write_op.cc:276] Timestamp: P: 1783602070927412 usec, L: 0
0709 13:01:10.927827 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:01:11.015287 (+ 87460us) log.cc:844] Serialized 7853 byte log entry
0709 13:01:11.816303 (+801016us) op_driver.cc:464] REPLICATION: finished
0709 13:01:12.242292 (+425989us) write_op.cc:301] APPLY: starting
0709 13:01:12.242409 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:12.242828 (+   419us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:12.243008 (+   180us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:12.245533 (+  2525us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:12.297107 (+ 51574us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 13:01:12.297163 (+    56us) write_op.cc:312] APPLY: finished
0709 13:01:12.297750 (+   587us) log.cc:844] Serialized 138 byte log entry
0709 13:01:12.297968 (+   218us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.298154 (+   186us) write_op.cc:454] Released schema lock
0709 13:01:12.298209 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:12.298781 (+   572us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.298828 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1158507,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":334942,"apply.run_cpu_time_us":5555,"apply.run_wall_time_us":105322,"num_ops":20,"prepare.queue_time_us":146983,"prepare.run_cpu_time_us":20293,"prepare.run_wall_time_us":364280,"raft.queue_time_us":436099,"raft.run_cpu_time_us":43527,"raft.run_wall_time_us":343760,"replication_time_us":888316,"thread_start_us":723696,"threads_started":4,"wal-append.queue_time_us":119441,"wal-append.run_cpu_time_us":102618,"wal-append.run_wall_time_us":2153183}]]}
I20260709 13:01:17.628839 19835 catalog_manager.cc:5697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:17.632782 22744 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:17.633906 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:01:17.637878 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 284474us
I20260709 13:01:17.637449 19835 catalog_manager.cc:5697] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 1 to 9, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 9 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:17.638923 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.591354 23593 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:17.599715 23702 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.642807 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.646548 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:17.646832 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.687158 20008 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.592032 23758 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:17.575131 23042 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:17.681358 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.681232 19835 catalog_manager.cc:5697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:17.682737 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.690902 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:17.691250 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:17.693677 20015 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.693755 20017 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:17.694216 20011 raft_consensus.cc:2393] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:17.686246 23702 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.695302 23702 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:17.700721 23858 log.cc:927] Time spent T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:01:17.693195 19835 catalog_manager.cc:5697] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 11, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 11 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
W20260709 13:01:17.702675 23856 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 13:01:17.705317 23855 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:01:17.708161 23809 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:17.708585 23659 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:17.708810 23809 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:17.709482 23820 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:17.632721 23739 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:17.583876 23094 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.710919 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:17.693018 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59172.4us
I20260709 13:01:17.648866 23794 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:17.598986 23558 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.709818 23809 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.688966 23731 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:17.721733 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.718618 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:17.782440 20015 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:17.785614 20009 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:17.710393 23813 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:17.783380 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.785881 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67377.4us
W20260709 13:01:17.782471 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 250060us
W20260709 13:01:17.790493 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 149188us
I20260709 13:01:17.791123 20850 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:17.720101 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:17.793637 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 361217us
I20260709 13:01:17.718981 23696 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:17.797937 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:17.800256 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.804555 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.807866 23720 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:01:17.813472 23702 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.817010 23042 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.805488 23758 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:17.818234 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:17.819002 20157 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:17.823076 23862 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:01:17.824023 20854 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:17.824411 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.827579 20161 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:17.828871 23702 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:17.831416 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 111401us
I20260709 13:01:17.779598 23831 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:17.837397 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:17.844285 20163 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:17.841101 20008 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
W20260709 13:01:17.860417 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 62542.1us
I20260709 13:01:17.866001 20851 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:17.867046 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:17.805059 23868 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:17.876432 20009 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:17.869190 20157 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:17.792059 23731 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.808992 23696 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:17.850648 23865 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:17.891719 20165 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:17.849313 23794 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:17.870393 23232 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:17.876171 23868 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:17.901331 20016 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:17.902100 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:17.903019 23831 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:17.908105 23868 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.904157 23875 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.905009 23460 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.910909 23875 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.911263 23787 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:17.913017 23875 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:17.899379 23865 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:17.907187 23731 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:17.917591 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 50693.5us
I20260709 13:01:17.914992 23798 consensus_queue.cc:260] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:17.918912 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:17.920923 20082 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:01:17.907720 23711 consensus_meta.cc:287] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.001s
I20260709 13:01:17.915686 23865 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:17.930850 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.935599 23711 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:17.939453 23711 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.939767 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:17.940754 20857 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:17.944202 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.932950 23875 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:17.934114 23798 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:17.952353 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:17.923545 23872 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:17.899802 23878 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:01:17.947585 23702 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:17.958478 23521 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:17.975256 23521 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:17.955557 23731 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.976117 23731 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:17.959532 23443 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.001s
I20260709 13:01:17.962517 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:17.950703 23795 consensus_queue.cc:1059] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:01:17.961630 23711 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:17.951256 23818 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:17.965821 23094 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:17.971359 20857 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:17.972429 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:17.978879 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:17.991268 23878 raft_consensus.cc:2764] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:17.953940 23696 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:17.994544 23150 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:01:17.998939 23652 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:18.000015 20855 raft_consensus.cc:2368] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10.
I20260709 13:01:17.987016 23826 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.988876 23443 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:18.001271 23731 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:17.990195 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 119211us
I20260709 13:01:18.007236 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:18.009379 20852 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:18.013880 23711 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:18.012552 23826 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:18.014492 23711 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:18.015477 23711 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:18.016291 23826 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:18.017676 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:18.019085 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:18.020856 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:18.021584 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:18.022361 20158 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.024776 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:18.025689 20159 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:18.026988 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:18.028190 20159 raft_consensus.cc:2393] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:18.030175 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.031046 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:18.012631 23878 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:01:18.046499 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:18.047863 20165 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:18.040813 20055 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:18.050307 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.051115 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.052091 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.052523 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.052870 20085 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:18.045966 23702 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:18.053990 23702 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:18.054800 23702 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:18.039685 23580 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:01:18.058815 20159 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:18.061429 20157 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:18.066936 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:18.068662 20744 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:18.069830 23833 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:18.072371 23833 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:18.073881 23697 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:18.074425 23697 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:18.075521 23869 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:18.080577 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.084859 20091 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:18.086205 20158 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:18.087184 20156 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:18.087607 20055 leader_election.cc:400] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10.
I20260709 13:01:18.088291 20055 leader_election.cc:403] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:18.089315 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:18.075276 23868 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:18.096546 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.100642 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:18.102548 20157 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.029803 23831 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.101475 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:18.092231 23150 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:01:18.113749 23831 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:18.117199 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.121236 23150 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:18.121686 23150 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:18.121840 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:18.086073 23802 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.016s
I20260709 13:01:18.104828 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:18.122922 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.122539 23150 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:18.123270 20158 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:18.109221 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:18.126221 20165 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:18.116786 20851 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:18.127859 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.111999 20852 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:18.122548 23802 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.013s	sys 0.018s
I20260709 13:01:18.133921 23802 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:18.134783 23802 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:18.136183 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:18.136726 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.137365 20083 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:18.137620 20856 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:18.115226 23868 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:18.140265 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:18.140801 20011 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:18.141168 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:18.141849 20015 raft_consensus.cc:2368] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:18.141711 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:18.142968 23798 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:18.143188 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:18.143471 23870 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:18.143592 23878 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:18.144425 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:18.144994 23798 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:18.145880 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:18.145973 23798 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:18.140864 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:18.146912 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.114813 23875 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.149827 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.152884 23652 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:18.153244 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:18.153748 20090 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.154896 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:18.155113 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45795.7us
I20260709 13:01:18.156446 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:18.155823 20013 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.155707 20130 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:18.158840 20083 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:18.163776 20130 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:18.100888 23872 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:18.170133 20013 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:18.104635 23702 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.172042 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:18.123886 23150 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.125430 23435 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:18.155473 23818 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.183789 20090 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:18.188786 20157 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:18.202221 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.205055 23796 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:18.205775 23796 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:18.206314 23796 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:18.075304 23697 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:18.124452 23831 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:18.162019 23625 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:18.189870 23696 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:18.189841 23460 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.211007 20090 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:18.201701 23872 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:18.163400 23094 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:01:18.226414 23696 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.225207 23892 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.230998 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.436228 20090 raft_consensus.cc:2376] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11. Re-sending same reply.
I20260709 13:01:18.174612 23873 consensus_queue.cc:1059] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:01:18.211413 23232 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.180s
I20260709 13:01:18.441957 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.444942 20090 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:18.232097 23625 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:18.391178 23731 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:18.222694 23818 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:18.473425 20090 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:18.395511 23435 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:18.436658 23696 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:18.549897 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.448594 23580 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.233860 23460 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:18.568013 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 366592us
W20260709 13:01:18.585269 20743 leader_election.cc:336] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.564s (SENT)
I20260709 13:01:18.573426 20130 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:18.645330 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72003.3us
I20260709 13:01:18.650657 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:18.553969 23813 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:18.655093 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.701882 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.712129 20013 raft_consensus.cc:1240] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:18.716017 20086 raft_consensus.cc:2368] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:01:18.691159 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:18.443641 23892 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:18.508819 23731 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:18.657145 23725 raft_consensus.cc:2764] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:01:18.688297 23232 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:18.768123 20083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.028s	sys 0.040s
I20260709 13:01:18.569116 23580 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:18.772677 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:18.726531 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.675606 23813 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:18.563001 23435 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:18.788084 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.811146 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:18.820894 23894 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.380s	user 0.001s	sys 0.000s
I20260709 13:01:18.824080 23892 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:18.824954 23892 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:18.825558 23892 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:18.827885 23895 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:01:18.780552 23625 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:18.844565 23893 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.613s	user 0.001s	sys 0.000s
I20260709 13:01:18.856557 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:18.864020 20128 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.020s	sys 0.054s
I20260709 13:01:18.859802 23094 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:18.865702 20128 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.020s	sys 0.054s
I20260709 13:01:18.774961 23652 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:18.673834 23872 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:18.779469 23232 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:18.918040 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:18.872798 23094 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:18.930498 23094 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:18.932356 23910 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:18.932819 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:18.932964 23910 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:18.935053 23910 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:18.940881 23580 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:18.943535 23880 log.cc:927] Time spent T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.990s	user 0.000s	sys 0.015s
I20260709 13:01:18.945737 23232 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:18.917366 23896 log.cc:927] Time spent T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:01:18.894460 23652 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:18.916764 23902 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:01:18.948544 23800 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:18.949287 23798 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:18.949738 23731 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:18.919641 23900 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:18.918692 23435 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:18.918659 23460 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:18.952632 23702 consensus_meta.cc:287] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.763s	user 0.008s	sys 0.003s
I20260709 13:01:18.907011 23813 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:18.954898 23796 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.000s
I20260709 13:01:18.955749 23892 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:18.962679 20008 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:18.966970 20015 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:18.970281 20087 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:18.997249 20085 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
W20260709 13:01:18.948395 23875 consensus_meta.cc:287] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.798s	user 0.011s	sys 0.000s
I20260709 13:01:19.007432 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.007805 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.010078 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 661) took 1364 ms. Trace:
I20260709 13:01:19.034548 20016 rpcz_store.cc:276] 0709 13:01:17.646030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.646165 (+   135us) service_pool.cc:224] Handling call
0709 13:01:19.009832 (+1363667us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540073fad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:19.009991 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493231488}
I20260709 13:01:19.036243 23813 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.021473 20014 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:19.038326 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.039556 20085 raft_consensus.cc:3055] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:19.039956 20085 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:19.024650 23798 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.026463 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.045182 20086 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.917502 23818 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:18.975854 23898 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:01:19.055190 23652 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:19.033824 23796 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:19.062244 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54300.5us
I20260709 13:01:19.031808 23580 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.050989 20085 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:19.071943 20085 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:19.043824 23875 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10.
W20260709 13:01:19.080440 23150 consensus_meta.cc:287] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.878s	user 0.010s	sys 0.002s
I20260709 13:01:19.100169 20085 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:19.101696 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.103396 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.103609 20090 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:19.104873 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.082057 23652 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:19.291754 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 114952us
W20260709 13:01:19.297232 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.297719 20855 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:19.069432 23580 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:19.293111 23731 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.332650 23722 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:01:19.299082 23652 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.370589 23460 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:19.369650 23833 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:19.380681 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.383419 20087 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
W20260709 13:01:19.385051 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.385968 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.302874 23903 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:01:19.393463 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.394675 20162 raft_consensus.cc:2410] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:19.394824 20163 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:19.397524 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.351114 23731 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:19.386956 23833 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:19.392328 23726 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.388522 23460 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:19.412761 23912 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:01:19.413247 23833 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.413661 23460 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.415086 23892 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:19.415693 23892 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:19.418084 23892 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:19.419862 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.420328 23796 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:19.421069 23521 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:19.421648 23521 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.423761 23521 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.426841 23726 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:19.429766 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 671) took 1306 ms. Trace:
W20260709 13:01:19.462670 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.561012 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:19.562031 20163 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:19.565492 20743 leader_election.cc:336] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.735s (SENT)
I20260709 13:01:19.566521 23873 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:19.567183 23818 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:19.567221 23873 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.567973 23818 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.570725 23873 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.571803 23726 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:19.573855 23731 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:19.575467 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:19.576601 20016 raft_consensus.cc:2368] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10.
I20260709 13:01:19.514070 23460 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:19.588069 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:19.589663 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:19.590960 20090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 677) took 1414 ms (client timeout 1824 ms). Trace:
W20260709 13:01:19.591414 20090 rpcz_store.cc:269] 0709 13:01:18.176404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.524182 (+347778us) service_pool.cc:224] Handling call
0709 13:01:19.590798 (+1066616us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b5400a5e954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:19.590902 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1271532416}
I20260709 13:01:19.592572 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.592868 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:19.593179 20090 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:19.594053 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.596153 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:19.600565 20163 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:19.602113 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.607540 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:19.608424 20016 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
W20260709 13:01:19.607704 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.609882 20090 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:19.610647 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:19.612001 20086 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.002s
I20260709 13:01:19.613713 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.614125 20086 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:19.615085 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:19.615329 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.617544 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.618854 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.603133 23731 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:19.615746 20013 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:19.607030 23892 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.599788 23460 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:19.613137 23652 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.615295 23818 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.630046 20017 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:19.630887 20013 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:19.647311 23580 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:19.652887 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:19.653749 20013 raft_consensus.cc:2410] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:19.631212 23892 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:19.648826 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 1170) took 1530 ms. Trace:
I20260709 13:01:19.659248 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.631724 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:19.655577 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:19.636590 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.663173 20016 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.663110 20086 raft_consensus.cc:2376] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11. Re-sending same reply.
I20260709 13:01:19.657644 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.660804 20855 rpcz_store.cc:276] 0709 13:01:18.118522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.118699 (+   177us) service_pool.cc:224] Handling call
0709 13:01:18.119072 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.303754 (+1184682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.648487 (+344733us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000055d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:19.648745 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705534208}
I20260709 13:01:19.627863 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.670099 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.666162 20851 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:19.670622 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.672974 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1242) took 1642 ms. Trace:
I20260709 13:01:19.673285 20088 rpcz_store.cc:276] 0709 13:01:18.030423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.030599 (+   176us) service_pool.cc:224] Handling call
0709 13:01:19.670473 (+1639874us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:01:19.672928 (+  2455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7562496}
I20260709 13:01:19.674477 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:19.675034 20856 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:19.677377 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.677953 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.678504 20855 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:19.678666 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.678825 20850 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:19.679234 20090 raft_consensus.cc:2376] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12. Re-sending same reply.
I20260709 13:01:19.678133 20086 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:19.680665 23930 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.680835 23928 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:19.681265 23930 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:19.681422 23928 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.682255 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.680562 23580 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:19.652364 23818 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:19.652165 23787 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:19.684731 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.683841 23930 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:19.689186 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:19.670611 23813 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.000s
I20260709 13:01:19.683645 23928 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.689785 20852 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:19.690987 23580 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.690987 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.689839 20088 raft_consensus.cc:2376] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10. Re-sending same reply.
I20260709 13:01:19.691744 23731 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:19.708889 23731 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.710490 23580 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.711398 23731 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.708273 20850 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:19.708218 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.702323 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.666594 23873 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.715265 20016 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:19.712103 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 1161) took 1686 ms. Trace:
I20260709 13:01:19.716132 20156 rpcz_store.cc:276] 0709 13:01:18.025459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.025623 (+   164us) service_pool.cc:224] Handling call
0709 13:01:18.086787 (+ 61164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.087650 (+   863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.711841 (+1624191us) spinlock_profiling.cc:243] Waited 60.8 ms on lock 0x7b54004d6f50. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:19.712026 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139809536}
I20260709 13:01:19.718406 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.721446 23460 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:19.725732 20089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1243) took 1695 ms. Trace:
I20260709 13:01:19.726325 20089 rpcz_store.cc:276] 0709 13:01:18.030703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.032323 (+  1620us) service_pool.cc:224] Handling call
0709 13:01:19.724704 (+1692381us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa03378bfc0 00007fa033776929 00007fa0337763d2 00007fa02ccedbd4
0709 13:01:19.725006 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.725266 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.725305 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.725362 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:19.725409 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.725487 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.725650 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33947392}
W20260709 13:01:19.728565 23042 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:01:19.728466 23873 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:19.728880 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.731287 23873 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:19.688861 23521 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:19.706907 23925 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:01:19.732249 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:19.733464 23901 log.cc:927] Time spent T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.802s	user 0.004s	sys 0.000s
I20260709 13:01:19.732249 23521 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.736472 23521 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:19.736593 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 1164) took 1693 ms. Trace:
I20260709 13:01:19.737216 20161 rpcz_store.cc:276] 0709 13:01:18.043058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.044247 (+  1189us) service_pool.cc:224] Handling call
0709 13:01:18.044666 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.046564 (+  1898us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:18.091214 (+ 44650us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.461248 (+370034us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.532094 (+ 70846us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.666671 (+134577us) log.cc:844] Serialized 26 byte log entry
0709 13:01:18.666860 (+   189us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:18.667153 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 13:01:18.945994 (+278841us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b18001dc7a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:01:18.946179 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.946353 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.732813 (+786460us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:19.736387 (+  3574us) raft_consensus.cc:1685] finished
0709 13:01:19.736409 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.736536 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64599,"prepare.queue_time_us":67663,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":123,"raft.queue_time_us":481727,"raft.run_cpu_time_us":874,"raft.run_wall_time_us":21074,"spinlock_wait_cycles":283330688,"thread_start_us":283077,"threads_started":1,"wal-append.queue_time_us":740052}
W20260709 13:01:19.739374 23907 log.cc:927] Time spent T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.811s	user 0.000s	sys 0.002s
I20260709 13:01:19.740375 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.742081 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1259) took 1619 ms. Trace:
I20260709 13:01:19.744711 20083 rpcz_store.cc:276] 0709 13:01:18.122461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.159547 (+ 37086us) service_pool.cc:224] Handling call
0709 13:01:18.173489 (+ 13942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.209127 (+ 35638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.209164 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.209722 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:01:18.778614 (+568892us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:19.000886 (+222272us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.001224 (+   338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.001339 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.738776 (+737437us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:19.741739 (+  2963us) raft_consensus.cc:1685] finished
0709 13:01:19.741843 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.742013 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":232775,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":779,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":23722,"spinlock_wait_cycles":263554944,"thread_start_us":673860,"threads_started":2,"wal-append.queue_time_us":690325}
I20260709 13:01:19.747790 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:19.748544 23897 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.736s	user 0.004s	sys 0.014s
I20260709 13:01:19.748962 20083 raft_consensus.cc:2376] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13. Re-sending same reply.
I20260709 13:01:19.750262 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.750694 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.751098 20083 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:19.751379 20086 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:19.753334 20855 raft_consensus.cc:2410] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:19.754953 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.756263 20089 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:19.757894 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.761261 23802 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.784353 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.771750 20856 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:19.774816 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:19.768100 23696 consensus_meta.cc:287] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.184s	user 0.012s	sys 0.012s
I20260709 13:01:19.789933 20090 raft_consensus.cc:2376] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15. Re-sending same reply.
I20260709 13:01:19.791254 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.704139 23787 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:19.793995 23787 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:19.796904 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.797824 20082 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
W20260709 13:01:19.798445 20082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1250) took 1764 ms (client timeout 1625 ms). Trace:
W20260709 13:01:19.798898 20082 rpcz_store.cc:269] 0709 13:01:18.034364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.057646 (+ 23282us) service_pool.cc:224] Handling call
0709 13:01:19.796690 (+1739044us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:01:19.798337 (+  1647us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54148992}
I20260709 13:01:19.804435 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.805647 20084 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:19.806206 20084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36126 (request call id 1241) took 1776 ms (client timeout 2226 ms). Trace:
W20260709 13:01:19.806682 20084 rpcz_store.cc:269] 0709 13:01:18.030126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.032221 (+  2095us) service_pool.cc:224] Handling call
0709 13:01:19.804220 (+1771999us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b580057de98. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:01:19.806128 (+  1908us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34526592}
I20260709 13:01:19.809669 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1072) took 1671 ms. Trace:
I20260709 13:01:19.810261 20014 rpcz_store.cc:276] 0709 13:01:18.138187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.142499 (+  4312us) service_pool.cc:224] Handling call
0709 13:01:18.142775 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.037400 (+894625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.809302 (+771902us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54000f96d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:19.809587 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020329728}
I20260709 13:01:19.812026 23834 raft_consensus.cc:3055] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:19.812481 23834 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:19.813349 23834 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:19.813930 23758 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:19.814400 23834 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:19.826218 23720 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:19.826989 23720 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:19.833629 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 673) took 1668 ms. Trace:
I20260709 13:01:19.834256 20087 rpcz_store.cc:276] 0709 13:01:18.165022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.917783 (+752761us) service_pool.cc:224] Handling call
0709 13:01:19.833342 (+915559us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b54006a8454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:19.833534 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107147008}
I20260709 13:01:19.430596 20085 rpcz_store.cc:276] 0709 13:01:18.123789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.292047 (+1168258us) service_pool.cc:224] Handling call
0709 13:01:19.292509 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.292795 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:19.312345 (+ 19550us) log.cc:844] Serialized 14 byte log entry
0709 13:01:19.429413 (+117068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.429472 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:19.429534 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.429599 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.429711 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19750,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":2640,"raft.queue_time_us":1167,"thread_start_us":116631,"threads_started":1}
I20260709 13:01:19.701133 23818 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:19.861515 20090 raft_consensus.cc:2376] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9. Re-sending same reply.
I20260709 13:01:19.862026 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.839888 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.879689 20085 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:19.881282 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.882520 20085 raft_consensus.cc:2410] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:01:19.835723 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.883229 20087 raft_consensus.cc:2376] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12. Re-sending same reply.
I20260709 13:01:19.920557 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:19.921787 20084 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
W20260709 13:01:19.859890 23899 log.cc:927] Time spent T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.950s	user 0.003s	sys 0.000s
W20260709 13:01:19.795902 23916 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.772s	user 0.000s	sys 0.002s
W20260709 13:01:19.848050 23915 log.cc:927] Time spent T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.523s	user 0.002s	sys 0.000s
I20260709 13:01:19.938402 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:19.941327 20085 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:19.952039 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 676) took 1779 ms. Trace:
I20260709 13:01:19.952409 20085 rpcz_store.cc:276] 0709 13:01:18.172167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.923946 (+1751779us) service_pool.cc:224] Handling call
0709 13:01:19.924310 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.951736 (+ 27426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.951980 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.953182 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:19.953451 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.953881 20161 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:19.954069 20090 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:19.941701 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 683) took 1544 ms. Trace:
I20260709 13:01:19.849937 23611 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:19.956073 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:19.956710 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 681) took 1729 ms. Trace:
I20260709 13:01:19.957300 20084 rpcz_store.cc:276] 0709 13:01:18.227086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.939604 (+1712518us) service_pool.cc:224] Handling call
0709 13:01:19.956044 (+ 16440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.956233 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.956270 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.956330 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:19.956430 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.956504 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.956641 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.955204 20082 rpcz_store.cc:276] 0709 13:01:18.397558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.940156 (+1542598us) service_pool.cc:224] Handling call
0709 13:01:19.940688 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.941005 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.941137 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.941235 (+    98us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:19.941294 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.941382 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.941613 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.943611 20087 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:19.871199 23872 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.854799 23798 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:19.849247 23652 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.962167 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 672) took 1797 ms. Trace:
I20260709 13:01:19.953766 20085 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:01:19.965955 20085 pending_rounds.cc:85] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 4
I20260709 13:01:19.966427 20085 pending_rounds.cc:107] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:01:19.965945 20087 rpcz_store.cc:276] 0709 13:01:18.164582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.923329 (+1758747us) service_pool.cc:224] Handling call
0709 13:01:19.943390 (+ 20061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.961753 (+ 18363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.961959 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.966707 20085 raft_consensus.cc:2889] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:19.940052 23908 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.957008 20156 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 6.
I20260709 13:01:19.790431 23802 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:19.984325 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.985651 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.884793 23696 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:19.989125 20084 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:19.991633 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 685) took 1485 ms. Trace:
I20260709 13:01:19.991947 20082 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:19.992055 20852 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:19.992272 20084 rpcz_store.cc:276] 0709 13:01:18.506786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.958603 (+1451817us) service_pool.cc:224] Handling call
0709 13:01:19.986881 (+ 28278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.991272 (+  4391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.991562 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.989604 20850 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:19.992744 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 689) took 1274 ms. Trace:
I20260709 13:01:19.995746 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.996270 20082 rpcz_store.cc:276] 0709 13:01:18.718876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.982632 (+1263756us) service_pool.cc:224] Handling call
0709 13:01:19.983064 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.992557 (+  9493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.992699 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.997829 20082 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:19.989602 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 684) took 1529 ms. Trace:
I20260709 13:01:19.999225 20085 rpcz_store.cc:276] 0709 13:01:18.450776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.953152 (+1502376us) service_pool.cc:224] Handling call
0709 13:01:19.953447 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.980189 (+ 26742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.980501 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.003048 20087 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:20.003764 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 691) took 1223 ms. Trace:
I20260709 13:01:20.004181 20087 rpcz_store.cc:276] 0709 13:01:18.780238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.982554 (+1202316us) service_pool.cc:224] Handling call
0709 13:01:19.995571 (+ 13017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.003523 (+  7952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.003712 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.006886 20858 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:01:19.972958 23929 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:19.972715 23604 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.010636 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:19.990859 23904 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.981608 23908 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.032205 23904 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:20.034197 23908 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.012347 20084 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:19.990828 23833 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.011269 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 811) took 1870 ms. Trace:
I20260709 13:01:20.024255 20090 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:19.990928 23659 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.038633 20158 rpcz_store.cc:276] 0709 13:01:18.140491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.148082 (+  7591us) service_pool.cc:224] Handling call
0709 13:01:18.148424 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.148783 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.148809 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.150169 (+  1360us) log.cc:844] Serialized 26 byte log entry
0709 13:01:18.394682 (+244513us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:18.394730 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.394812 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.819312 (+1424500us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:19.995126 (+175814us) raft_consensus.cc:1685] finished
0709 13:01:19.995146 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.011201 (+ 16055us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":783,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":328437504,"thread_start_us":244098,"threads_started":1,"wal-append.queue_time_us":625543}
I20260709 13:01:19.940737 23720 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:19.969507 23872 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:20.042696 23042 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:20.044504 23833 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:20.045852 23802 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:19.974895 23652 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.046965 23833 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:20.038488 23604 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.048708 23652 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:20.051198 23604 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.052052 23659 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.054430 23659 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.055148 23094 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:20.057709 23042 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:20.058149 23042 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:20.059123 23042 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:20.034543 23904 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:20.080466 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 378482us
I20260709 13:01:20.037966 23892 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:20.082553 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 381261us
I20260709 13:01:20.094605 20085 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:01:20.095033 20085 pending_rounds.cc:85] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 4
I20260709 13:01:20.095307 20085 pending_rounds.cc:107] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:20.095566 20085 raft_consensus.cc:2889] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:20.096165 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.069787 23042 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.101263 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.103636 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.094206 23720 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.106112 20165 raft_consensus.cc:2410] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:20.104439 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:20.035446 23929 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:20.107270 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 514452us
I20260709 13:01:20.118089 20056 leader_election.cc:400] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:20.119544 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.132895 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:20.133929 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.135669 20090 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:20.139109 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.141090 23873 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.142274 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1273) took 1136 ms. Trace:
I20260709 13:01:20.142627 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:20.142697 20159 rpcz_store.cc:276] 0709 13:01:19.006222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.006541 (+   319us) service_pool.cc:224] Handling call
0709 13:01:19.006855 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.070692 (+ 63837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.070743 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.086532 (+ 15789us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.738154 (+651622us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b1800114640. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:01:19.950932 (+212778us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:19.963686 (+ 12754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.963756 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.141832 (+178076us) raft_consensus.cc:1685] finished
0709 13:01:20.141852 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.142201 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71278,"raft.queue_time_us":64037,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10335,"spinlock_wait_cycles":2468096,"thread_start_us":212577,"threads_started":1,"wal-append.queue_time_us":1006459}
I20260709 13:01:20.144850 20855 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:20.146461 23873 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:20.147076 23873 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:20.147130 23802 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:20.147637 23802 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.148010 23802 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.148044 23873 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:20.150588 23905 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.234s	user 0.003s	sys 0.000s
I20260709 13:01:20.152212 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.152701 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:20.153154 23818 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:20.154090 23872 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:20.154939 23731 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:20.155335 23731 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:20.155699 23731 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:20.155709 23928 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:20.156019 23521 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.157503 20056 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.158810 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:20.160121 23813 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.160658 23813 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:20.161237 23929 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.162621 23696 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.167111 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36126 (request call id 1258) took 2045 ms. Trace:
I20260709 13:01:20.169484 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.166620 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.167541 20091 rpcz_store.cc:276] 0709 13:01:18.122131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.150754 (+ 28623us) service_pool.cc:224] Handling call
0709 13:01:18.151073 (+   319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:18.166029 (+ 14956us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:18.166079 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:18.173510 (+  7431us) log.cc:844] Serialized 52 byte log entry
0709 13:01:18.686679 (+513169us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:18.686739 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.686838 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.149836 (+1462998us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:20.165027 (+ 15191us) raft_consensus.cc:1685] finished
0709 13:01:20.165057 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.167046 (+  1989us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7084,"raft.queue_time_us":10836,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7388,"spinlock_wait_cycles":329631616,"thread_start_us":456820,"threads_started":1,"wal-append.queue_time_us":685768}
I20260709 13:01:20.105093 23892 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:01:20.174059 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.176882 20016 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:20.177626 23521 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:20.179139 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.179637 23928 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.180081 20014 raft_consensus.cc:2410] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:20.176025 23652 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:20.181274 23652 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.184712 23834 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:20.185408 23834 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.189210 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:20.189783 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 203787us
I20260709 13:01:20.192649 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.193181 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.193384 20159 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:20.195431 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 674) took 2030 ms. Trace:
I20260709 13:01:20.196146 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.196318 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.200383 20855 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:20.202248 20858 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:20.171191 23872 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:20.202854 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 324065us
I20260709 13:01:20.175611 23934 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.173621 23875 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.224687 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.193785 20854 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:20.228096 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.224606 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.215340 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:20.227489 20164 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
W20260709 13:01:20.232122 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.199664 23795 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:20.236557 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.203373 23813 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:20.203994 23874 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:20.225579 23604 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.223973 23652 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:20.233884 23875 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:20.250941 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:20.214519 23929 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:01:20.253708 23868 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:20.256266 20858 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:01:20.278334 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:20.278746 20858 pending_rounds.cc:85] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 5
I20260709 13:01:20.278987 20858 pending_rounds.cc:107] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:01:20.279207 20858 raft_consensus.cc:2889] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:20.245206 23874 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:20.258579 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.280556 20850 raft_consensus.cc:2410] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:20.259352 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.232380 23872 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:20.261329 23936 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.153s	user 0.002s	sys 0.000s
I20260709 13:01:20.237182 23933 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.240531 23930 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:01:20.245591 23917 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:20.285526 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:20.276225 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:20.287453 20012 raft_consensus.cc:2376] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8. Re-sending same reply.
I20260709 13:01:20.277832 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.288199 20015 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:20.258270 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.288199 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:20.282759 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.252358 23940 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.289196 20855 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:20.285580 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:20.234159 23934 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:20.293582 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:20.290135 23917 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:20.280886 23150 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:20.296470 23580 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.296236 23150 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.296918 23580 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.297324 23580 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.298475 23150 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.299782 23917 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:20.291790 23892 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:01:20.300848 23892 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:20.301227 23892 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:20.274875 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:20.302057 23892 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.304302 20016 raft_consensus.cc:2376] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10. Re-sending same reply.
I20260709 13:01:20.304703 23892 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.290462 23940 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.305321 23940 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.290294 23933 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:20.284004 23739 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:20.308835 23739 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:20.309265 23739 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.271095 23604 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.314253 23604 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.195907 20088 rpcz_store.cc:276] 0709 13:01:18.165317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.931257 (+1765940us) service_pool.cc:224] Handling call
0709 13:01:19.946294 (+ 15037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.946642 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:19.967853 (+ 21211us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.193414 (+225561us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.193584 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.193627 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:20.193677 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.195250 (+  1573us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.195375 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17441,"prepare.queue_time_us":24149,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":24335,"thread_start_us":206059,"threads_started":1}
I20260709 13:01:20.280066 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:20.277907 23696 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:20.252987 23869 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:20.278226 23875 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:20.325747 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 47152.6us
I20260709 13:01:20.323616 20011 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:20.200079 23834 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.331138 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.331933 20011 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
W20260709 13:01:20.288252 23897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.016s	sys 0.025s
I20260709 13:01:20.331789 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:20.326750 23696 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:20.326578 23868 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:20.307898 23933 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:20.347088 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.354434 20012 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:20.348760 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.348433 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.344609 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.358314 20156 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:20.359803 19981 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.358458 20011 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:20.361040 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:20.391642 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.364002 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:20.399704 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 51004.1us
I20260709 13:01:20.361912 20156 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
W20260709 13:01:20.430246 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 74051.7us
I20260709 13:01:20.433584 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.332712 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:20.439640 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106970us
W20260709 13:01:20.345137 23897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.025s
I20260709 13:01:20.347227 23720 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:20.363484 23951 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.448554 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:20.448940 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:20.446084 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.452677 20130 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:20.427680 23950 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.468051 23460 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.467888 23918 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:20.465483 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:20.578400 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 113028us
I20260709 13:01:20.484598 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:20.485989 23795 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.488051 23873 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:20.339241 23580 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.455996 23872 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.443367 23929 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.443284 23042 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.444496 23934 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:20.498989 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.450608 23720 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.342276 23731 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.432044 23834 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.342576 23652 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:01:20.347597 23802 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.561709 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.562956 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.345911 23696 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.592829 23726 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.564834 20083 raft_consensus.cc:1275] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:20.345614 23604 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
W20260709 13:01:20.566054 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 67909.4us
W20260709 13:01:20.586764 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 102421us
I20260709 13:01:20.528642 23933 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.333760 23874 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.592904 20091 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:01:20.618731 20091 pending_rounds.cc:85] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 6
I20260709 13:01:20.619150 20091 pending_rounds.cc:107] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:01:20.618803 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.619561 20091 raft_consensus.cc:2889] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:20.596444 23869 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:20.596436 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 147611us
W20260709 13:01:20.601393 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 241577us
I20260709 13:01:20.596951 23958 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:20.588590 23813 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:20.582655 23460 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:20.604897 20086 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:20.564345 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.628798 23802 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:20.593680 23918 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:20.613961 23042 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:20.630436 23813 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.631356 23042 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:20.630687 23918 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.614406 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.601146 20088 raft_consensus.cc:2393] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 10: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:20.602695 23720 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:20.621536 20090 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:20.623214 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.649507 20084 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:20.624956 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.624208 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:20.627730 23460 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.633678 23940 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.640245 23731 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:20.642513 20743 leader_election.cc:336] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.198s (SENT)
W20260709 13:01:20.667588 20743 leader_election.cc:336] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.546s (SENT)
I20260709 13:01:20.642359 23580 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:20.612387 20744 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:20.598217 23739 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:20.673719 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 48900.9us
I20260709 13:01:20.565461 23950 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.668228 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:20.652454 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:20.671588 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 57269us
W20260709 13:01:20.671584 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59332.1us
I20260709 13:01:20.625478 23958 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.674441 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:20.630899 23813 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:20.674922 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:20.681785 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.677695 23731 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.676158 23892 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.681439 23958 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.676700 23802 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:01:20.685308 23604 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:20.685510 20743 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.505s (SENT)
W20260709 13:01:20.691398 23949 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:01:20.670855 23652 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:20.677347 23722 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:20.627005 23869 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.665855 23929 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.695629 20129 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:20.675215 23697 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:20.700636 23952 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:20.700305 23731 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:20.700537 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 48226.6us
I20260709 13:01:20.702752 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.702917 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.704298 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.720517 23940 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:20.722527 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.761456 20164 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
W20260709 13:01:20.738000 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 57088.3us
I20260709 13:01:20.742993 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.744786 23968 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.751688 23739 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:20.702844 23957 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:20.771457 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 68790.6us
I20260709 13:01:20.759807 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:20.774478 20165 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.776369 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:20.777949 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.778813 20159 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:20.763589 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.722923 23652 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.767102 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.780269 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:20.781248 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:20.781780 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.784140 20086 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:20.784442 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:20.785007 20854 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.785715 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:20.786082 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:20.786607 20086 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.788569 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:20.788695 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:20.791615 20158 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:20.795630 23652 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:20.756990 23697 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:20.770771 20130 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:20.805244 20011 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:20.809463 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:20.812953 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:20.814777 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.760751 23952 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.758231 23885 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.806999 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:20.829599 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.832908 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.823274 20086 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
W20260709 13:01:20.820701 20743 leader_election.cc:336] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.622s (SENT)
I20260709 13:01:20.834821 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.836179 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.836359 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.837168 20744 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:20.837334 20857 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:20.721993 23875 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:20.841104 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.828754 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:20.841953 20091 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:20.847203 20165 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:20.821841 23957 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:20.850442 23957 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.856796 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:20.860286 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.860579 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:20.861047 20853 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:20.805197 23722 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:20.809717 23726 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:20.741528 23929 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:20.868109 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 61187.2us
I20260709 13:01:20.828881 23802 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.836606 23711 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.877027 20089 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:20.838335 23952 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.797669 23934 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:20.868353 20744 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:20.819942 23697 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:20.877221 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:20.854404 23892 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.886126 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:20.888056 20743 leader_election.cc:336] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.786s (SENT)
I20260709 13:01:20.888478 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:20.868021 23696 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:20.873592 23802 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.877333 23711 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.854293 23758 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:01:20.905568 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:20.878465 23869 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:20.891645 23933 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.885396 23965 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:20.797188 23968 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:20.884863 23150 raft_consensus.cc:2764] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:01:20.904337 20743 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.824s (SENT)
I20260709 13:01:20.912434 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.901087 23934 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:20.914703 23720 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:20.915465 23972 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.905889 23711 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.904301 23802 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:20.937448 20743 leader_election.cc:336] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.914s (SENT)
I20260709 13:01:20.958494 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:20.936450 23965 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:20.960160 20743 leader_election.cc:336] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.941s (SENT)
I20260709 13:01:20.960326 23965 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:20.961778 23720 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:20.935412 23940 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:20.963511 23958 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:20.964371 23940 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:20.964579 23731 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:20.965382 23940 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.965859 23958 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:20.967850 23967 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:20.968243 23950 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:20.901345 23696 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:20.954699 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:20.971967 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.892071 23892 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:20.969987 20090 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:20.977084 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:20.910741 23933 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:20.981940 20162 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:20.986016 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 141051us
W20260709 13:01:20.981014 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 124315us
I20260709 13:01:20.953388 23934 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:20.939119 23969 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:20.954941 23739 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:20.994766 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 86560.3us
I20260709 13:01:20.990535 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.979691 23975 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.002202 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:20.980382 23868 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:20.974177 23604 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:20.982995 23659 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 13:01:21.021763 23963 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:01:21.024698 23870 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:21.026450 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:20.987622 23933 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.012051 23739 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.013682 23969 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:21.030591 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.014526 23968 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:21.034329 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:20.996500 23934 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.028163 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:21.029356 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.008806 23973 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:21.037585 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.050434 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.053225 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.054808 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.056037 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.056039 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.056896 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.058423 20086 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:21.058954 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:21.026916 23868 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:21.059834 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.060680 20086 raft_consensus.cc:2376] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10. Re-sending same reply.
I20260709 13:01:21.061997 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.062552 20086 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:21.063165 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.063882 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.064940 20162 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:21.065241 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.066422 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.067173 20056 leader_election.cc:400] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10.
I20260709 13:01:21.069813 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.070452 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:21.073249 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.074301 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.076167 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.038380 23950 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:21.044435 23973 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:21.076987 23982 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:21.143752 23973 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.144671 23972 consensus_queue.cc:1059] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:21.146484 23972 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:21.131147 20854 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:21.056111 23869 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:21.134488 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:21.031548 23904 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:01:21.086071 23958 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:21.087579 20164 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:01:21.137745 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68049us
W20260709 13:01:21.137110 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63161.9us
I20260709 13:01:21.070005 23868 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:21.071831 23962 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:21.135358 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:21.161744 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 77160.4us
W20260709 13:01:21.164625 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 78164.7us
I20260709 13:01:21.144829 23940 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:01:21.088361 23875 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:21.166311 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.169651 23962 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.173381 23962 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:21.174736 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.176119 23873 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:21.137928 23731 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:01:21.177170 23094 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:21.137390 23795 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.089059 23929 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:21.178267 23795 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.178900 23929 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:21.180459 23795 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:21.180925 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:21.171085 23869 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:21.181984 20129 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.029559 23975 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:21.171873 20161 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:21.180821 23929 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:21.137050 23720 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:21.076267 23604 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:01:21.167670 23892 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:21.194205 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:21.214526 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.170827 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:01:21.216632 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 45815.9us
I20260709 13:01:21.139357 23696 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:21.220829 20016 raft_consensus.cc:1240] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:21.221192 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.222098 20017 raft_consensus.cc:2393] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 16: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:21.223189 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.138586 23957 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:01:21.154910 23972 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:21.213428 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.228278 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.215760 20164 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:21.233471 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.227133 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.236248 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:21.237114 20010 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.208838 23908 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:21.227190 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.238639 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.182204 23869 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.241928 23972 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.184576 23968 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:01:21.226871 23958 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:21.235116 23957 consensus_queue.cc:1243] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:21.209174 23904 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:21.262336 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.266206 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.266892 20083 raft_consensus.cc:2410] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:01:21.263099 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:21.265424 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:21.268585 20855 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.270146 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.258069 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.215466 23818 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.264858 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:21.182008 23952 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:21.274672 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.254227 23908 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.237890 23720 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:21.240005 23094 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:21.283195 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 109262us
I20260709 13:01:21.261626 23042 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:21.274194 23460 consensus_meta.cc:287] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.006s
I20260709 13:01:21.305394 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.306628 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.310235 23940 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:21.312736 23731 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:21.313496 23731 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:21.314693 23731 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:21.315626 23739 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:21.318521 23975 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:21.319515 23802 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:21.189858 23969 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.253777 23696 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:01:21.322501 20014 raft_consensus.cc:2368] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12.
W20260709 13:01:21.323832 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 49254.7us
I20260709 13:01:21.324905 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.297011 23908 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:21.386024 23874 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.387215 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:21.387811 20164 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:21.389552 23996 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:01:21.330195 23968 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:01:21.391556 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:21.392438 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 67656.6us
I20260709 13:01:21.392665 20159 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:01:21.393285 20159 raft_consensus.cc:606] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:21.393334 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.328740 20743 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:21.318387 23652 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:21.339116 23933 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.348870 20010 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:21.350589 23958 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:21.354040 23795 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.354350 23952 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:21.354781 23991 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:21.356776 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:21.360257 23976 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:01:21.374392 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:21.378612 23987 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.210s	user 0.002s	sys 0.000s
I20260709 13:01:21.380942 20084 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:21.294983 23818 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.276258 23965 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.323573 23460 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.329077 23972 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:21.398716 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 144589us
W20260709 13:01:21.399348 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 70717.2us
W20260709 13:01:21.402977 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 431030us
I20260709 13:01:21.405047 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.395874 23932 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.371435 23697 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.440922 20008 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:21.438688 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:21.451992 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.432822 20010 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:21.441895 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.442818 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.398068 23992 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.416849 23957 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.459676 20856 raft_consensus.cc:1240] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:21.454758 20013 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.455314 20854 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:21.456965 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.407395 23968 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:21.468189 23813 consensus_meta.cc:287] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.756s	user 0.000s	sys 0.013s
I20260709 13:01:21.468670 23813 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:21.470851 23813 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:21.474310 23868 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.459267 23720 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:21.476610 20156 raft_consensus.cc:2368] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 16. Current term is 17.
I20260709 13:01:21.517257 20853 raft_consensus.cc:1240] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:21.477521 23652 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:21.478223 23094 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:21.478652 23818 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:21.407465 23795 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.480134 23460 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:21.480129 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.480792 23930 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:21.483479 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 44884.1us
I20260709 13:01:21.483592 23611 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:21.484076 23958 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.484905 23968 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.488101 23933 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:21.460112 23932 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.489394 20084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:21.441488 23974 consensus_queue.cc:1048] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:01:21.494796 23802 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:21.421337 23711 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:21.496546 23965 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:21.504148 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:21.509574 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57739.1us
I20260709 13:01:21.569135 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.442625 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.463481 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.458010 23952 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:21.530918 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:21.545943 20858 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:21.547490 20084 raft_consensus.cc:606] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:21.571575 20084 raft_consensus.cc:614] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:21.467814 23979 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:21.577350 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.516594 23982 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:21.574199 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 110857us
W20260709 13:01:21.574927 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 132364us
I20260709 13:01:21.581984 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.582413 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.583634 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.585304 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.547909 23968 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.602283 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.547998 23652 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.626889 23611 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:21.610703 23962 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.895748 20009 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:01:21.895898 23973 consensus_meta.cc:287] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.750s	user 0.014s	sys 0.005s
I20260709 13:01:21.896721 23962 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:21.896929 23611 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:21.898608 23652 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:21.907279 20855 consensus_meta.cc:287] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.635s	user 0.013s	sys 0.001s
I20260709 13:01:21.581183 23930 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:21.907892 20855 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:21.908362 23930 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.908823 23930 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.581821 23952 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:21.911866 23652 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:21.874487 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 431980us
I20260709 13:01:21.876930 23997 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:21.915710 23997 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.918118 23997 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:21.918565 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.530774 23908 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.923470 23908 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:21.877815 23934 consensus_meta.cc:287] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.821s	user 0.011s	sys 0.007s
I20260709 13:01:21.560925 24000 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:01:21.924250 23934 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:21.925112 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 351497us
I20260709 13:01:21.876607 23813 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:21.880178 23094 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.926501 23934 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:21.928789 23952 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:21.881760 23982 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.930325 23982 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:21.876295 23999 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.249s	user 0.003s	sys 0.005s
I20260709 13:01:21.560675 23818 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:01:21.932235 20014 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:21.615664 20091 raft_consensus.cc:2410] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:21.552078 23720 consensus_queue.cc:1059] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:01:21.583803 23918 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:21.891583 23874 consensus_meta.cc:287] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.005s
I20260709 13:01:21.893703 23711 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:21.894212 24002 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:21.896453 23973 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.900688 20165 raft_consensus.cc:2368] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12.
I20260709 13:01:21.901613 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.901798 23795 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:21.945439 24000 raft_consensus.cc:3055] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:21.951656 23818 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:21.952179 24000 raft_consensus.cc:740] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:21.952322 23795 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:21.952704 23818 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:21.952747 23795 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:21.953184 24000 consensus_queue.cc:260] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:21.948573 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.903002 24012 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:21.877130 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 291899us
I20260709 13:01:21.933632 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:21.956848 20009 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:21.956841 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.937242 23042 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.935753 20082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:21.940487 20085 pending_rounds.cc:85] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 6
I20260709 13:01:21.931599 23997 raft_consensus.cc:993] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower 48da76e971b84219a6ba80d1ba83b7eb to start an election
I20260709 13:01:21.969347 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:21.943396 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.943639 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.937834 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:21.970634 20089 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.970870 20010 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:21.946056 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:21.971385 20850 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:21.946373 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:21.946964 20086 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:21.951503 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:21.933900 23094 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:21.886893 20164 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:21.960736 20856 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:21.973424 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:21.966720 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.974709 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.967442 20082 raft_consensus.cc:606] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.975059 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.975497 20082 raft_consensus.cc:614] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:21.975797 20090 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:21.967645 20085 pending_rounds.cc:107] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:21.968091 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.964088 20744 leader_election.cc:400] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 16. Current term is 17.
I20260709 13:01:21.976315 20085 raft_consensus.cc:2889] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:21.976446 20744 leader_election.cc:403] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:21.976624 20017 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:21.970227 20084 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.975546 20015 raft_consensus.cc:2410] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:21.961725 20016 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.976641 20083 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:01:21.963770 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 62250.4us
I20260709 13:01:21.979269 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:21.979599 20010 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.979300 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:21.980640 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:21.953855 23795 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.938190 23726 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:01:21.941131 23908 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:01:21.953217 23818 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:21.986160 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.984406 20091 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:21.936692 23934 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:01:21.987249 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.987203 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:22.001338 20083 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.988925 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.986420 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.990247 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:22.004510 20090 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:22.005704 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:22.006469 20090 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:22.007581 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:21.993878 20129 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:21.994879 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:21.958281 24000 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.945184 23933 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:01:21.963068 23973 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:22.022300 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.025619 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:22.026067 20056 leader_election.cc:400] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12.
I20260709 13:01:22.026448 20056 leader_election.cc:403] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:22.026543 20087 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:21.984606 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:22.010504 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:22.011430 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:21.977936 23874 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:21.978350 23711 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:21.983561 24010 raft_consensus.cc:1081] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Signalling peer 48da76e971b84219a6ba80d1ba83b7eb to start an election
I20260709 13:01:22.018069 20161 raft_consensus.cc:2368] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:22.021344 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:21.982929 24014 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.986413 23993 raft_consensus.cc:3060] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:01:21.998606 23997 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.986145 24009 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:22.031618 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:21.997651 24016 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:22.053206 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 58414us
I20260709 13:01:22.036023 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:21.978569 23042 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.969550 24012 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:21.986903 23094 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:22.017221 23720 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:22.051002 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.028349 23918 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:22.042284 23795 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:22.037834 23869 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.069152 20089 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:22.088820 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:22.089572 20084 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:22.090449 20084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746d924541574d6086a6ae086476a1e7"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:22.064296 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:22.041677 23711 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:22.080220 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:22.084801 20850 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:22.086306 20087 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:22.028278 23833 consensus_queue.cc:1059] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:01:22.081580 23958 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:22.047899 23979 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.047248 24009 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:22.098014 20083 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
W20260709 13:01:22.129966 20011 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.992s	user 0.017s	sys 0.024s
W20260709 13:01:22.130330 20011 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.017s	sys 0.024s
I20260709 13:01:22.130928 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1442) took 1106 ms. Trace:
I20260709 13:01:22.102553 23874 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:22.102612 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:22.133473 23950 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:22.133900 23950 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.134241 23950 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.136747 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.138307 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:22.139312 20083 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:22.139667 19981 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:22.099258 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.066295 23995 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:22.091008 20084 raft_consensus.cc:606] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:22.103736 23802 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.129388 23833 raft_consensus.cc:3055] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:22.046284 24014 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.115438 23697 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:22.098366 23711 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.116320 23982 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:22.086781 23904 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:22.177547 23904 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.093798 23818 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.180253 20014 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:01:22.181685 20014 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:22.182888 20014 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:22.098589 23659 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:22.184190 23982 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:22.184849 23982 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:22.185298 23868 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:22.119029 23932 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:22.185699 23982 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:22.186564 23932 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.187017 23932 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.072360 24016 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.156481 23833 raft_consensus.cc:740] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:22.189163 23932 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:22.091683 24002 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12.
I20260709 13:01:22.122100 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 1193) took 1181 ms. Trace:
I20260709 13:01:22.189486 24016 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:22.149078 23958 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:22.189832 20853 rpcz_store.cc:276] 0709 13:01:20.940738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.940915 (+   177us) service_pool.cc:224] Handling call
0709 13:01:20.941335 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.525248 (+583913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.121851 (+596603us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54000050d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:22.122038 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324018432}
I20260709 13:01:22.191032 23958 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:22.191422 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:22.122022 23930 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.192247 20088 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:22.192490 24014 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:22.193348 23904 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:22.193738 23930 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:22.081641 23720 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:22.127777 20158 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:22.144076 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:22.195982 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 145080us
I20260709 13:01:22.145390 20852 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:22.142625 20850 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:22.146284 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:22.146091 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:22.147634 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:22.131275 20011 rpcz_store.cc:276] 0709 13:01:21.024639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.024809 (+   170us) service_pool.cc:224] Handling call
0709 13:01:21.025143 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.025351 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:21.132751 (+107400us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.130583 (+997832us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.130642 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.130694 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.130758 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.130875 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2948,"prepare.queue_time_us":150684,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":44705,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":1100144,"threads_started":2}
W20260709 13:01:22.148741 23968 consensus_meta.cc:287] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.004s
I20260709 13:01:22.077840 23918 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:22.132247 20009 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 17.
I20260709 13:01:22.149155 20084 raft_consensus.cc:614] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:22.148526 24009 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:22.173542 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:22.174836 20856 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.188793 23833 consensus_queue.cc:260] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:22.190280 24016 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.121045 23957 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:22.145143 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:22.146975 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:22.153025 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:22.215809 20013 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 10: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:22.157680 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:22.254897 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 97314.5us
I20260709 13:01:22.245859 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.261112 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.180291 23818 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:22.196300 23993 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 16. Current term is 17.
I20260709 13:01:22.180253 24012 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:22.216042 23802 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:22.336809 20012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:34530
I20260709 13:01:22.337594 20012 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:22.338306 20012 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:22.269991 23697 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.220016 23968 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:22.341730 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49873.5us
W20260709 13:01:22.347326 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55978.4us
I20260709 13:01:22.290694 23833 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:22.354574 20130 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:22.292495 23934 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:22.292728 23972 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.341544 23802 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:22.392377 20857 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:01:22.400432 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 301271us
I20260709 13:01:22.347504 23150 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:22.394666 20082 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:22.337615 23904 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:01:22.418591 20012 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:22.365583 23974 raft_consensus.cc:2804] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:01:22.489197 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 76867.8us
W20260709 13:01:22.502091 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88634.9us
I20260709 13:01:22.503311 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:22.509013 20012 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:22.391073 23934 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:22.510011 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:22.404564 24012 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:22.365113 23993 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.138s
I20260709 13:01:22.522475 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:22.405774 23758 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:22.405084 24016 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:22.524394 20163 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:22.410820 23957 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:22.519397 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:22.522293 23987 log.cc:927] Time spent T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:01:22.518050 20091 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.518559 23875 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.509868 24014 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:22.527442 23802 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:22.510041 23869 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:01:22.506348 23720 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:22.534452 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 716) took 1470 ms. Trace:
W20260709 13:01:22.537994 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.344620 23918 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:22.510727 23997 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.592466 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:22.601809 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:22.629806 20856 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:22.488441 23868 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
W20260709 13:01:22.610538 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 67774.8us
I20260709 13:01:22.611091 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:22.502625 23150 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:22.516956 23873 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:01:22.621225 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:22.557276 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 298133us
I20260709 13:01:22.623157 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:22.637753 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:22.517535 24028 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:01:22.638650 20852 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:22.619268 20086 rpcz_store.cc:276] 0709 13:01:21.063570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.063669 (+    99us) service_pool.cc:224] Handling call
0709 13:01:22.534252 (+1470583us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54009b8554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:22.534365 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2029073792}
I20260709 13:01:22.639448 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:22.547168 24031 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:22.641861 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.626782 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:22.546066 23974 raft_consensus.cc:697] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:22.640107 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:22.643098 20086 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
W20260709 13:01:22.606046 23795 consensus_meta.cc:287] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 13:01:22.596917 23659 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:01:22.644412 20086 raft_consensus.cc:1240] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:22.553154 23758 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.606961 24033 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.645757 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:22.546391 24012 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:22.640723 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:22.619227 23957 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:22.656857 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.595680 24016 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.615077 23875 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:22.673462 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:22.675807 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:22.676431 20163 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:22.676571 20089 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:22.676769 20163 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:22.653831 23918 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:22.644845 23995 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:01:22.646059 23934 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:22.702180 20163 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:22.705909 20163 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:22.706046 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.718421 20856 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:22.687156 24032 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:22.733561 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 55498.4us
I20260709 13:01:22.734989 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:22.736474 20853 raft_consensus.cc:2468] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 17.
W20260709 13:01:22.741258 20129 leader_election.cc:336] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.625s (SENT)
I20260709 13:01:22.742105 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:22.753186 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 96420.2us
W20260709 13:01:22.756281 20129 leader_election.cc:336] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.673s (SENT)
W20260709 13:01:22.757444 20129 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 1.696s (SENT) [suppressed 32 similar messages]
I20260709 13:01:22.667100 23982 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 13:01:22.725375 24024 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:01:22.765480 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:22.725848 23968 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:22.666265 24033 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:22.754415 24032 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:22.774415 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:22.777387 20164 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:22.761489 23997 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:22.760071 24000 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:22.758630 24034 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:22.773401 23993 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.767014 23930 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:01:22.838570 20091 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
W20260709 13:01:22.853637 24030 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:01:22.857170 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:22.708276 23974 consensus_queue.cc:237] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:22.849836 24024 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:01:22.871357 24038 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:01:22.872438 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 1166) took 1346 ms. Trace:
W20260709 13:01:22.876891 24049 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:01:22.878206 24045 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:01:22.856779 23995 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:22.880478 23150 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:22.881045 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:22.938651 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:22.938984 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:22.939507 20083 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:22.877012 24012 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:22.944844 23758 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:22.883004 24016 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:22.883589 23982 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:22.945952 23982 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:22.947034 23982 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:22.925052 23711 consensus_meta.cc:287] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.744s	user 0.013s	sys 0.001s
I20260709 13:01:22.949317 23711 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:22.876891 24041 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:01:22.831714 23968 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:22.886320 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:22.836237 24000 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:22.899621 24057 consensus_queue.cc:1059] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:01:22.768427 23957 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:22.893539 20859 rpcz_store.cc:276] 0709 13:01:21.526478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.526919 (+   441us) service_pool.cc:224] Handling call
0709 13:01:21.527218 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.527748 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.527780 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.122731 (+594951us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b2c0001afe8. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:01:22.123157 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.293380 (+170223us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.353093 (+ 59713us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.390648 (+ 37555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.390729 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.871982 (+481253us) raft_consensus.cc:1685] finished
0709 13:01:22.872135 (+   153us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.872347 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38503,"prepare.queue_time_us":57317,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":52891,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2201,"spinlock_wait_cycles":17846528,"thread_start_us":169790,"threads_started":1,"wal-append.queue_time_us":483453}
W20260709 13:01:22.937338 24040 log.cc:927] Time spent T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:01:22.952719 20156 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:22.898965 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:23.007004 20164 raft_consensus.cc:2393] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:22.778523 24053 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 13:01:22.982865 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:22.983657 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.007963 20091 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:23.008108 20084 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:22.979070 19980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.006s	sys 0.008s
W20260709 13:01:23.009130 19980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.006s	sys 0.008s
I20260709 13:01:22.993101 20163 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:22.884815 23875 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:22.947782 23833 consensus_meta.cc:287] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.000s
I20260709 13:01:22.835009 24034 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.031832 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:22.981218 23711 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.032661 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.032799 20013 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:23.032970 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.031988 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.034075 24056 raft_consensus.cc:3055] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:23.034492 24056 raft_consensus.cc:740] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:23.034583 20088 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:23.034997 24054 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:23.035748 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.035549 24054 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.037153 20015 raft_consensus.cc:2410] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:01:23.037213 24033 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.037422 20088 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:01:23.037818 20088 pending_rounds.cc:85] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 3
I20260709 13:01:23.038033 20088 pending_rounds.cc:107] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:01:23.035360 24056 consensus_queue.cc:260] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:23.038260 20088 raft_consensus.cc:2889] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.042153 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.038965 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50794 (request call id 1192) took 1064 ms. Trace:
I20260709 13:01:23.038189 24054 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:23.047528 24009 consensus_meta.cc:287] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.787s	user 0.007s	sys 0.004s
I20260709 13:01:22.981822 23968 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.049734 24034 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.052337 20088 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 728) took 1027 ms. Trace:
I20260709 13:01:23.080269 20088 rpcz_store.cc:276] 0709 13:01:22.025339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.036738 (+1011399us) service_pool.cc:224] Handling call
0709 13:01:23.037070 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.051965 (+ 14895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.052259 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:23.050861 20159 rpcz_store.cc:276] 0709 13:01:21.975001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.975456 (+   455us) service_pool.cc:224] Handling call
0709 13:01:21.975750 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.978832 (+  3082us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:22.010184 (+ 31352us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.206082 (+195898us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.220478 (+ 14396us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.254036 (+ 33558us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:22.254370 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.263399 (+  9029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.263503 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.038745 (+775242us) raft_consensus.cc:1685] finished
0709 13:01:23.038767 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.038904 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40737,"prepare.queue_time_us":58760,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3642,"raft.queue_time_us":25671,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":55729,"thread_start_us":158736,"threads_started":1,"wal-append.queue_time_us":269943}
I20260709 13:01:23.081784 20083 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:23.025892 23758 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.091212 20091 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
W20260709 13:01:23.090202 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 332761us
W20260709 13:01:23.079356 23933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.010s	sys 0.000s
W20260709 13:01:23.092779 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 386796us
I20260709 13:01:23.093003 20083 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:23.094704 20088 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:23.097419 20084 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:23.099277 20084 raft_consensus.cc:1240] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:23.081244 24054 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.102099 20084 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:23.106238 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.102027 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:01:23.093850 23933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.010s	sys 0.000s
I20260709 13:01:23.109295 20084 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:23.109362 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 345476us
I20260709 13:01:23.093076 24056 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.113708 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:23.113854 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.114638 20017 raft_consensus.cc:2393] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:23.116788 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:23.117260 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 351963us
I20260709 13:01:23.118057 24044 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:23.085163 24009 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.123006 20161 raft_consensus.cc:1240] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:23.123299 23818 consensus_meta.cc:287] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.836s	user 0.010s	sys 0.002s
I20260709 13:01:23.124444 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:23.127745 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.130411 20017 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:23.130524 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:23.132128 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:23.132287 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.136675 24048 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:23.133227 24067 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:23.134265 24009 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:23.134753 23802 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:23.135349 20744 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:23.135561 20743 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:23.137276 20090 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:23.143777 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:23.089138 23758 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:23.116333 23950 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.978s	user 0.012s	sys 0.003s
W20260709 13:01:23.138132 23979 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.976s	user 0.008s	sys 0.004s
I20260709 13:01:23.139204 20084 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:23.178999 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43733.7us
I20260709 13:01:23.180320 20084 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:23.137708 23982 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:23.181981 20084 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:23.142287 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:23.137235 24048 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:01:23.228271 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86038.6us
I20260709 13:01:23.229413 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:23.144764 23802 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:23.189512 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.113293 24054 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.244096 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:23.176986 23818 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:23.174935 24033 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:23.140872 24053 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 13:01:23.182205 23659 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:23.226444 24057 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.241510 24009 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:23.145329 24067 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:23.244012 23982 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:23.184909 23979 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:23.242472 23802 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.256944 20013 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:01:23.264914 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.246054 23150 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.247068 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:23.274201 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 148821us
I20260709 13:01:23.184293 24034 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:01:23.263094 23659 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:23.257966 23818 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.179024 23950 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.299249 23982 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.302431 23930 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:23.251147 23726 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:23.256089 23722 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:23.364002 23952 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:23.363672 23659 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:23.364518 23952 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:23.305018 23868 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:23.296602 24053 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:01:23.305792 20089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 729) took 1280 ms. Trace:
I20260709 13:01:23.256726 23932 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:23.263482 24044 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:01:23.306409 24067 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.297456 24009 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:23.312395 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 65410.3us
I20260709 13:01:23.251224 24054 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.357960 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.303444 23697 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:23.252169 23968 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:23.310587 24063 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:01:23.362696 24070 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.357842 23951 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:01:23.364250 23722 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:23.373116 23960 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.305217 23758 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.277522 24020 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.371097 20089 rpcz_store.cc:276] 0709 13:01:22.025617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.088507 (+ 62890us) service_pool.cc:224] Handling call
0709 13:01:23.305580 (+1217073us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54005a1754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:23.305725 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1296520448}
I20260709 13:01:23.399761 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.360183 23818 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.377084 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:23.378347 20013 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:23.371450 23930 consensus_queue.cc:1243] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:23.372241 23932 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:23.384347 23697 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:23.363111 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.410346 20091 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:23.359086 23957 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:01:23.380788 24054 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.252898 24048 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:23.389946 23969 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.390945 23950 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:23.397023 20165 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50794 (request call id 1179) took 2355 ms (client timeout 1840 ms). Trace:
I20260709 13:01:23.406144 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.369067 23652 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.413553 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:23.418570 24054 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.420805 23726 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:23.409906 23697 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:23.412200 23758 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:23.459189 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.422729 23932 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.429065 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:23.439672 20165 rpcz_store.cc:269] 0709 13:01:21.037195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.037332 (+   137us) service_pool.cc:224] Handling call
0709 13:01:23.392799 (+2355467us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54000dacd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:23.393019 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252833536}
I20260709 13:01:23.440948 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.465323 20089 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.395998 24070 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.468104 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.468868 20091 raft_consensus.cc:2376] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13. Re-sending same reply.
I20260709 13:01:23.398763 23960 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.436614 23969 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:23.473033 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.480525 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.437463 23982 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.459128 23930 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:23.491067 20089 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:23.497974 20013 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:23.467144 23962 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.471477 23951 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.409266 24020 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.459168 23652 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.478509 24036 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:23.501508 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:23.509995 20089 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:23.496946 23930 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.499497 23932 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.512876 20013 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:23.443542 23722 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:23.526526 24074 log.cc:927] Time spent T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:01:23.528126 23652 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:23.528963 24076 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:01:23.529842 23960 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.531118 24020 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:23.533552 20083 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:23.533730 24054 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.543076 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:23.543983 20165 raft_consensus.cc:2368] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:23.545590 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.521404 24044 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:23.546670 24054 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:23.546805 24044 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.496371 23982 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.509159 23962 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:23.548645 20013 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:23.549535 23982 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:23.550032 23962 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.515686 23659 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:01:23.550678 24057 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.513491 24043 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:23.502554 23951 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.551770 24043 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:23.552829 24043 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.504020 23697 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.554836 23697 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.555199 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.555867 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 837) took 1572 ms. Trace:
I20260709 13:01:23.556881 23659 consensus_queue.cc:1243] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:23.557302 23968 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.557637 23968 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:23.558075 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.558660 23968 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.560134 23968 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.561643 24020 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.563271 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.565116 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.566006 20856 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:23.567981 23720 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:23.569032 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:23.553360 23951 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.556377 20156 rpcz_store.cc:276] 0709 13:01:21.983863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.984008 (+   145us) service_pool.cc:224] Handling call
0709 13:01:22.009990 (+ 25982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.006691 (+996701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.555618 (+548927us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b540080fcd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:23.555804 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2167465856}
I20260709 13:01:23.570766 20083 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 13:01:23.571267 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:23.562299 24020 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.513327 23969 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.577704 23966 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:23.578603 23966 consensus_queue.cc:1243] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:23.579418 24000 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.575892 20858 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:23.573652 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.515588 23930 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.582623 24070 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:23.513916 24036 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.572868 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:23.587206 24044 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:23.585033 24036 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:23.572099 23720 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:23.588330 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:23.589133 20165 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.586184 24020 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:23.603427 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.592572 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:23.596150 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.600754 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.608451 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.609256 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.608702 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.610730 20856 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:23.612536 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.613760 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.614131 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:23.616750 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.614478 20858 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:23.617473 20856 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:23.626165 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.626416 20165 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:23.603688 23932 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:23.630124 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.588239 23818 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.626274 23659 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:23.635556 23932 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.642702 24083 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:01:23.642776 23659 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:23.645236 23659 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.643194 23932 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.651765 24086 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:23.652853 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.654733 20091 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:01:23.657832 23802 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:23.659500 24051 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.652s	user 0.000s	sys 0.003s
I20260709 13:01:23.660347 20010 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:01:23.660559 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 1165) took 2134 ms. Trace:
I20260709 13:01:23.661065 20857 rpcz_store.cc:276] 0709 13:01:21.526152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.526275 (+   123us) service_pool.cc:224] Handling call
0709 13:01:21.526580 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.394757 (+868177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.660266 (+1265509us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54000041d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:23.660471 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990418176}
I20260709 13:01:23.665825 23720 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.667094 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 719) took 1682 ms. Trace:
I20260709 13:01:23.668421 23818 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.667544 20087 rpcz_store.cc:276] 0709 13:01:21.984805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.115499 (+130694us) service_pool.cc:224] Handling call
0709 13:01:22.115859 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:22.406834 (+290975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.406913 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:22.520473 (+113560us) log.cc:844] Serialized 26 byte log entry
0709 13:01:22.691951 (+171478us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.723083 (+ 31132us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.759187 (+ 36104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.759295 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.658867 (+899572us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:23.666753 (+  7886us) raft_consensus.cc:1685] finished
0709 13:01:23.666869 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.667028 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35738,"prepare.queue_time_us":91770,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":4223,"raft.queue_time_us":111976,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":4396,"spinlock_wait_cycles":32735488,"thread_start_us":390287,"threads_started":3,"wal-append.queue_time_us":319707}
I20260709 13:01:23.670809 23802 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:23.670955 23973 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.671516 23973 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:23.672107 23697 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.672524 23973 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.672710 23697 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:23.674893 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.673331 23952 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.675665 23952 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:23.679442 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:23.679172 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86732.7us
I20260709 13:01:23.676491 23697 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.680917 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:23.652487 24086 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:23.686292 23042 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.687958 24086 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.691602 23973 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.693414 23697 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:23.693907 23697 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:23.694280 23697 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.696663 23972 consensus_queue.cc:1059] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:23.699667 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:23.700935 24086 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:23.702430 23908 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.703104 23908 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:23.703459 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:23.704821 24086 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:23.706094 20850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44210 (request call id 1176) took 1350 ms. Trace:
I20260709 13:01:23.706274 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:23.707429 20854 raft_consensus.cc:2410] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:01:23.707587 24086 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:23.708086 24086 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.708405 24086 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.706712 20850 rpcz_store.cc:276] 0709 13:01:22.355489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.357541 (+  2052us) service_pool.cc:224] Handling call
0709 13:01:22.357859 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.387511 (+ 29652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.387539 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.387574 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.387608 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.705860 (+1318252us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b54000096d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:23.705906 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.706035 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67259136}
I20260709 13:01:23.709961 23908 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.711405 23908 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.713191 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:23.714872 20088 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:23.717213 24009 raft_consensus.cc:3055] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.717677 23983 consensus_queue.cc:1059] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:23.717839 23932 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.717679 24009 raft_consensus.cc:740] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:01:23.721405 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 107382us
I20260709 13:01:23.723618 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:23.727480 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:23.733145 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.738716 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:23.738965 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:23.740092 20089 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.734730 23930 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.751837 20089 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:23.757812 23932 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:23.761233 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.719836 24009 consensus_queue.cc:260] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:23.764082 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:23.765100 24009 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.766299 24064 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:23.767608 24064 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:23.774062 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 733) took 1013 ms. Trace:
I20260709 13:01:23.774643 20090 rpcz_store.cc:276] 0709 13:01:22.760839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.938760 (+177921us) service_pool.cc:224] Handling call
0709 13:01:23.773807 (+835047us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400664ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:23.773980 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423497088}
I20260709 13:01:23.768587 24064 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.750188 23951 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:23.777726 23951 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.778260 23951 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.763527 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.779186 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:23.779862 20015 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
W20260709 13:01:23.761139 24018 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.670s	user 0.003s	sys 0.015s
I20260709 13:01:23.786154 20090 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:23.733299 24036 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.786981 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:23.790804 20090 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:23.790683 20087 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:23.792604 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:23.797204 20090 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:23.798780 20855 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:23.801997 24029 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.804203 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1453) took 1865 ms. Trace:
I20260709 13:01:23.805017 24004 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:23.795460 23991 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:23.804693 20085 rpcz_store.cc:276] 0709 13:01:21.939263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.939432 (+   169us) service_pool.cc:224] Handling call
0709 13:01:21.940180 (+   748us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:21.977243 (+ 37063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.977271 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:21.977800 (+   529us) log.cc:844] Serialized 52 byte log entry
0709 13:01:22.068582 (+ 90782us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:22.068640 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.068738 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.760249 (+1691511us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:23.803815 (+ 43566us) raft_consensus.cc:1685] finished
0709 13:01:23.803855 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.804139 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7907,"spinlock_wait_cycles":14216960,"thread_start_us":90447,"threads_started":1,"wal-append.queue_time_us":112475}
I20260709 13:01:23.798722 23992 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:23.807241 23992 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.807611 23992 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:23.806241 23991 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.808439 23991 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.750710 23966 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:01:23.784955 23869 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:23.813726 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.802649 24029 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:23.813747 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:23.824579 24057 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:23.792438 24054 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:01:23.827168 20085 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:23.814687 23802 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.835258 20013 raft_consensus.cc:2410] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:23.825057 24057 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.817655 23869 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.795939 24036 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.825536 23973 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:01:23.850104 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:23.869405 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:23.871110 20085 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:23.873783 20015 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:23.821511 23934 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:01:23.875934 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:23.847980 23869 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.852289 24057 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.837736 23966 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:23.890252 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.834774 23652 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:23.838022 23802 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.837168 24067 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 13:01:23.903340 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.886143 23659 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:23.908310 24029 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:23.915205 23802 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:23.920204 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 727) took 1895 ms. Trace:
I20260709 13:01:23.897354 23869 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.920758 20086 rpcz_store.cc:276] 0709 13:01:22.025107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.649200 (+624093us) service_pool.cc:224] Handling call
0709 13:01:22.649574 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:22.699557 (+ 49983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:22.699597 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:22.700842 (+  1245us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:22.700889 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.919888 (+1218999us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b540053b954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:23.919964 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:23.920137 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114299776}
I20260709 13:01:23.928360 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:23.937095 20744 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:23.941478 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.902906 23962 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.911103 23659 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.944214 23869 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:23.969014 24025 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:23.969551 24025 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:23.970484 24025 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:23.970985 23869 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.972777 24036 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:23.972791 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.976132 23659 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:23.964548 24063 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:23.903402 23966 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:23.990657 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.000816 20088 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:24.003108 20090 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:24.008846 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:24.009172 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.010001 20089 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:24.010247 20856 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:24.019727 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:23.944716 23697 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.026156 20015 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:01:24.006333 23965 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.998389 23930 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:23.945142 24086 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:24.013337 24073 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:23.983361 23962 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:24.025313 24067 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:24.039065 20089 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:24.040477 24067 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.048477 24067 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:24.068828 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 104636us
I20260709 13:01:24.025116 23991 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:24.052062 23802 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:01:24.081061 23869 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:24.036094 23965 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:24.076233 23974 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.128s	user 0.016s	sys 0.028s
I20260709 13:01:24.041836 24073 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:24.090706 23974 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.017s	sys 0.028s
W20260709 13:01:24.091812 24093 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.192s	user 0.000s	sys 0.002s
I20260709 13:01:24.113169 23965 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.114853 23958 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:24.115818 23958 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:24.117280 24086 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:24.117074 24032 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:24.120975 23991 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:24.121929 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.122678 24032 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.119081 23958 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.086489 24057 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:24.105296 23932 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.138780 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:24.139336 23992 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.150126 20085 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:24.152266 20090 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:24.129719 23965 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:24.156409 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.157280 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:24.158478 20085 raft_consensus.cc:2376] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14. Re-sending same reply.
I20260709 13:01:24.160737 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:24.162359 20090 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:01:24.163160 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 84477.3us
I20260709 13:01:24.164255 20744 leader_election.cc:400] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
I20260709 13:01:24.165073 20744 leader_election.cc:403] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:24.139906 23932 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:24.168265 20744 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:24.160943 23992 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:24.169643 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:24.157276 23965 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:24.173489 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:24.172964 20857 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:24.170496 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:24.163633 23966 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.182929 23974 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:24.185025 24055 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:24.186175 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:24.198896 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.200717 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.185636 24055 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:24.208127 20085 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:01:24.210930 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.183902 23966 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11.
W20260709 13:01:24.212159 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.212702 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.193467 24063 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:24.226081 24055 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.193979 23932 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.242594 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:24.223260 23962 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:24.248716 24108 log.cc:927] Time spent T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:01:24.245402 23966 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:01:24.268291 23974 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:24.268743 23966 consensus_queue.cc:1243] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:24.268919 23974 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.247288 23930 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.271296 23974 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.271100 23930 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.271018 24067 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.274139 23930 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:24.274436 24067 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:24.277006 24009 consensus_meta.cc:287] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.511s	user 0.015s	sys 0.000s
I20260709 13:01:24.280435 24057 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:24.244128 24073 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.282598 20008 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:24.286441 20158 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:24.287175 20158 pending_rounds.cc:85] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 6
I20260709 13:01:24.287648 20158 pending_rounds.cc:107] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:24.288064 20158 raft_consensus.cc:2889] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.289311 23930 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.289923 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:24.290985 20088 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 14: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:24.292788 23722 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.298988 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.300822 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:24.306089 20850 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 14: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:24.296708 23951 consensus_meta.cc:287] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.002s
I20260709 13:01:24.307679 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:24.309360 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:24.311713 20010 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:24.316722 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:24.323693 20084 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:01:24.324412 20084 pending_rounds.cc:85] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 5
I20260709 13:01:24.324680 20084 pending_rounds.cc:107] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:01:24.324918 20084 raft_consensus.cc:2889] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.329463 23094 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:24.290005 23930 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.293433 23659 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:24.358002 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.362188 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.396010 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.317005 23951 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:24.423686 20856 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:24.401582 23611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.005s	sys 0.025s
I20260709 13:01:24.467679 23930 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:24.401481 24073 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:24.471282 23611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.006s	sys 0.025s
W20260709 13:01:24.491957 20082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 726) took 2467 ms (client timeout 1598 ms). Trace:
I20260709 13:01:24.490829 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1124) took 2520 ms. Trace:
W20260709 13:01:24.494658 20082 rpcz_store.cc:269] 0709 13:01:22.024831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.035739 (+ 10908us) service_pool.cc:224] Handling call
0709 13:01:24.491733 (+2455994us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b540020e4d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:24.491845 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649999232}
I20260709 13:01:24.492722 24067 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:24.509608 23968 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:24.510226 23968 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:24.502398 20851 rpcz_store.cc:276] 0709 13:01:21.970444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.971819 (+  1375us) service_pool.cc:224] Handling call
0709 13:01:21.972121 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.972322 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:22.060307 (+ 87985us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.061525 (+  1218us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.917326 (+1855801us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b1800255700. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:24.306781 (+389455us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b1800255700. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:01:24.490470 (+183689us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.490521 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:24.490570 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.490651 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.490772 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20573,"prepare.queue_time_us":117617,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":29427,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":4564,"spinlock_wait_cycles":264528128,"thread_start_us":255194,"threads_started":2}
I20260709 13:01:24.498159 20016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 690) took 3269 ms. Trace:
I20260709 13:01:24.521165 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 742) took 1186 ms. Trace:
W20260709 13:01:24.521523 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.471763 24032 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.502676 24067 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:24.521070 20016 rpcz_store.cc:276] 0709 13:01:21.222570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.259862 (+ 37292us) service_pool.cc:224] Handling call
0709 13:01:21.260467 (+   605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.983629 (+723162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.491683 (+2508054us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54002463d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:24.491858 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612181760}
I20260709 13:01:24.477406 24117 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:24.528123 20853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 1170) took 2378 ms (client timeout 2148 ms). Trace:
I20260709 13:01:24.511201 23968 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.521806 20091 rpcz_store.cc:276] 0709 13:01:23.335029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.491613 (+156584us) service_pool.cc:224] Handling call
0709 13:01:23.492044 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.655723 (+163679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.520789 (+865066us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400307354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:24.521087 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373240704}
W20260709 13:01:24.528630 20853 rpcz_store.cc:269] 0709 13:01:22.150222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.191153 (+ 40931us) service_pool.cc:224] Handling call
0709 13:01:24.527893 (+2336740us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400006ed4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:24.528052 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329146624}
I20260709 13:01:24.492333 23962 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:24.490708 24056 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:24.527163 23951 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.531759 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:24.532766 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.540722 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.541531 20853 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:24.541853 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.545226 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.545365 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.545811 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.546169 20853 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:24.546036 20851 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:24.553835 20165 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:24.550369 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.577126 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.577440 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:24.585160 20162 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:24.585374 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:24.543329 20055 connection.cc:256] RPC call timeout handler was delayed by -1.14689114399971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
I20260709 13:01:24.590299 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.587466 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.590571 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.544261 24012 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:24.543745 24086 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:24.492933 24073 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.597213 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.598022 20156 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:24.600360 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.529011 24032 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:24.571571 23968 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.601194 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.610553 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:24.613246 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.614163 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:24.534296 24056 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.534448 23962 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:24.592242 24009 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:24.593791 23930 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.618142 24067 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.618769 23932 consensus_queue.cc:1059] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:01:24.615410 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 112168us
I20260709 13:01:24.600250 24012 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.619958 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.533497 24117 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.616488 20055 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.620661 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.615710 23094 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:24.630352 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.630661 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.634126 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.634112 20163 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:24.636415 23868 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.636951 23868 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.637878 23659 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:24.639035 23868 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.622783 24009 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:24.607940 24086 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.634860 20156 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:24.641335 20163 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:24.628410 24012 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.625763 23930 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.644990 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.647746 20157 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:24.647843 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.648782 20164 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:24.647166 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.652993 20744 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:24.656181 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 711) took 1275 ms. Trace:
I20260709 13:01:24.656894 20014 rpcz_store.cc:276] 0709 13:01:23.381134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.382005 (+   871us) service_pool.cc:224] Handling call
0709 13:01:23.404078 (+ 22073us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:23.410877 (+  6799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.410943 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:23.416168 (+  5225us) log.cc:844] Serialized 52 byte log entry
0709 13:01:23.616143 (+199975us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:23.682941 (+ 66798us) log.cc:844] Serialized 14 byte log entry
0709 13:01:23.754349 (+ 71408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.754496 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:23.754557 (+    61us) raft_consensus.cc:1685] finished
0709 13:01:23.754569 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.655734 (+901165us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b54003f8ed0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:24.656091 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70859,"prepare.queue_time_us":24361,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":47,"raft.queue_time_us":73495,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":4904,"spinlock_wait_cycles":49902464,"thread_start_us":187950,"threads_started":2,"wal-append.queue_time_us":186041}
I20260709 13:01:24.663373 20157 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:24.665280 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.655870 23868 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:24.673709 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57267.6us
W20260709 13:01:24.676294 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.662382 23962 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.677451 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.680541 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:24.681869 20157 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.681773 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.673617 23868 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.687829 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.692348 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.685521 23932 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:24.703846 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.685959 23868 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.696444 24127 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:24.708943 20055 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:24.695784 24126 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:24.712298 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.711535 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:24.712885 24126 raft_consensus.cc:2764] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:24.711138 23930 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.711226 23932 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:24.701063 24073 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:24.715190 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.719643 23965 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.720328 20157 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:24.720217 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.721138 23965 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.721587 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:24.722218 20157 raft_consensus.cc:2376] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13. Re-sending same reply.
I20260709 13:01:24.713115 24009 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:24.728602 24056 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:24.729539 24117 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.730559 24012 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.731031 23094 raft_consensus.cc:3055] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:24.712968 24086 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.731667 23094 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:24.731786 20854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 1134) took 1180 ms. Trace:
I20260709 13:01:24.732343 20854 rpcz_store.cc:276] 0709 13:01:23.551738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.568780 (+ 17042us) service_pool.cc:224] Handling call
0709 13:01:24.731596 (+1162816us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b5400380254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:24.731734 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315760896}
I20260709 13:01:24.732722 23094 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.737314 23094 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:24.737725 20160 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:01:24.738123 20160 pending_rounds.cc:85] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 6
I20260709 13:01:24.738376 20160 pending_rounds.cc:107] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:01:24.738600 20160 raft_consensus.cc:2889] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.715431 24032 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.741214 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.742728 24012 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:24.741513 20161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1311) took 2685 ms. Trace:
I20260709 13:01:24.743583 20161 rpcz_store.cc:276] 0709 13:01:22.055271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.120940 (+ 65669us) service_pool.cc:224] Handling call
0709 13:01:22.121237 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.123436 (+1002199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.740030 (+1616594us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540049be54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:24.740223 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2303338880}
I20260709 13:01:24.751031 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.751116 20161 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:24.751973 20160 raft_consensus.cc:2368] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12.
I20260709 13:01:24.759497 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.759842 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.760401 20165 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:24.760325 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:24.764021 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:24.829118 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.840262 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 131154us
I20260709 13:01:24.777855 23697 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:24.848026 20857 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:24.848263 23739 consensus_queue.cc:1059] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:24.850445 23739 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:24.852979 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.857515 23580 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:24.859251 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1466) took 1536 ms. Trace:
I20260709 13:01:24.842774 20055 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:24.862196 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.870649 23969 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.774080 23868 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:24.870311 23995 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.882645 23995 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.886188 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:24.887740 23868 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:24.889389 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:24.890734 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:24.892355 23731 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:24.766826 24121 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.251s	user 0.005s	sys 0.000s
I20260709 13:01:24.902878 23995 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:24.866514 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.862548 20084 rpcz_store.cc:276] 0709 13:01:23.322270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.324408 (+  2138us) service_pool.cc:224] Handling call
0709 13:01:23.324767 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.325392 (+1000625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.858365 (+532973us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54000cd5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:24.858894 (+   529us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295349120}
I20260709 13:01:24.895140 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:24.867763 23094 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:24.861522 23580 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:24.863763 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:24.897363 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 1114) took 1306 ms. Trace:
I20260709 13:01:24.908833 20163 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.909281 24043 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:24.913549 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 50325us
I20260709 13:01:24.915344 20055 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:24.916097 20090 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:24.918190 20091 raft_consensus.cc:2393] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:24.920465 24130 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.922258 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 746) took 1381 ms. Trace:
I20260709 13:01:24.922340 23731 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:24.922849 20087 rpcz_store.cc:276] 0709 13:01:23.540838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.669705 (+128867us) service_pool.cc:224] Handling call
0709 13:01:23.670081 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.793341 (+123260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.921516 (+1128175us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400a5e954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:24.922040 (+   524us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276565376}
I20260709 13:01:24.924438 23094 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:24.925057 23094 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:24.925585 23094 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:24.897673 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:24.905242 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:24.928741 20088 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.907990 20013 rpcz_store.cc:276] 0709 13:01:23.590263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.590436 (+   173us) service_pool.cc:224] Handling call
0709 13:01:24.896707 (+1306271us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54000f9bd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:24.897065 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514216320}
W20260709 13:01:24.932755 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.934202 23731 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:24.934242 24123 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.295s	user 0.000s	sys 0.003s
I20260709 13:01:24.946771 23995 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:24.947435 23995 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.947885 23995 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.948868 20014 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:24.957544 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:24.958094 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.958551 20013 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:24.961280 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.975172 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.976166 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:24.988376 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.992420 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:24.993563 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:25.002475 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:25.003661 20016 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.005338 24025 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:25.005647 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:25.010594 24124 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.462s	user 0.006s	sys 0.000s
I20260709 13:01:25.010803 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 859) took 1097 ms. Trace:
I20260709 13:01:25.013018 20164 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:25.011314 20158 rpcz_store.cc:276] 0709 13:01:23.912907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.913198 (+   291us) service_pool.cc:224] Handling call
0709 13:01:23.913492 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.285936 (+372444us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b54003c99a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e729dc0 00007fa02e7b41c0 00007fa02e7b5231 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c
0709 13:01:24.289006 (+  3070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.010236 (+721230us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540028d7d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:25.010589 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849907456}
I20260709 13:01:25.037755 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.041240 24129 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.042254 24129 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:25.044622 24100 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:25.049001 24100 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:25.053200 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:25.054143 20014 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:25.057603 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:25.058848 20087 raft_consensus.cc:2393] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:25.060027 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 193901us
I20260709 13:01:25.065412 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.066864 23992 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:25.066804 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.068660 20087 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:25.069401 20850 raft_consensus.cc:2368] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12.
I20260709 13:01:25.023124 19840 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 8 to 11, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 11 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.071805 19840 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 7 to 8, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.073031 20087 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:01:25.073498 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 141756us
I20260709 13:01:25.065735 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:25.068552 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:25.059970 19835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32772 (request call id 152) took 15860 ms (client timeout 15000 ms). Trace:
I20260709 13:01:25.088806 20163 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:25.117643 20088 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:25.103387 20013 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:25.105890 23094 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:25.119638 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46469.9us
I20260709 13:01:25.123605 20850 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:25.125792 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.126025 24010 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.128438 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.124966 23992 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.132910 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.136658 20744 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:25.129412 23580 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.143949 23580 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:25.147433 23580 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:25.148386 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:25.159756 23094 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:25.135859 24129 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:25.162457 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:25.169198 20850 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.143764 23979 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:25.143934 23758 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:25.179159 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.179359 20859 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:25.183561 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.186122 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:25.191820 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.192499 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 124200us
W20260709 13:01:25.194208 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 96776.5us
I20260709 13:01:25.194307 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:25.184969 24129 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:25.196708 23934 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:25.197371 24130 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:25.197458 23934 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:25.198599 20091 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:25.200029 24016 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:25.213021 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 85019.5us
I20260709 13:01:25.213024 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:25.214706 20854 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.217872 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:25.219641 20858 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:25.213304 23758 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.172592 24010 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:25.220885 23758 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.219441 23711 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:01:25.223178 24010 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:25.225044 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.232321 24053 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:25.226349 23917 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:25.233353 24053 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:25.234023 23917 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:25.236579 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.183665 24059 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.238464 24059 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.239645 23934 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:25.241338 24059 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:25.242034 24059 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:25.242408 24059 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:25.245995 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:25.246716 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:25.248502 20855 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:25.249047 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:25.250651 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.247553 20013 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:25.252363 24010 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.253633 20856 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:01:25.254189 20856 pending_rounds.cc:85] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 6
I20260709 13:01:25.254532 20856 pending_rounds.cc:107] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:01:25.254858 20856 raft_consensus.cc:2889] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:25.190100 19840 catalog_manager.cc:5697] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 6 to 10, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 10 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.263816 20859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1132) took 1712 ms. Trace:
I20260709 13:01:25.264394 20859 rpcz_store.cc:276] 0709 13:01:23.551129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.564127 (+ 12998us) service_pool.cc:224] Handling call
0709 13:01:23.564589 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.189092 (+1624503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.263469 (+ 74377us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54006ea854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:25.263728 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3707146496}
I20260709 13:01:25.265555 19840 catalog_manager.cc:5697] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 5 to 8, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.251621 19982 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:25.253201 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.234748 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:25.305953 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 72114.6us
I20260709 13:01:25.243566 24059 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.317189 24059 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:25.339774 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:25.342582 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:25.254156 24010 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.235118 24053 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:25.349975 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 104053us
I20260709 13:01:25.351130 19982 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:25.091419 19835 rpcz_store.cc:269] 0709 13:01:09.199264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.199441 (+   177us) service_pool.cc:224] Handling call
0709 13:01:25.059644 (+15860203us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:18.040309 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:18.060603 (+ 20294us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:18.060670 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:01:18.060688 (+    18us) tablet.cc:662] Decoding operations
0709 13:01:18.062392 (+  1704us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:18.062493 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:18.062522 (+    29us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:01:18.120284 (+ 57762us) tablet.cc:701] Row locks acquired
0709 13:01:18.120335 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:01:18.120497 (+   162us) write_op.cc:270] Start()
0709 13:01:18.120658 (+   161us) write_op.cc:276] Timestamp: P: 1783602078120437 usec, L: 0
0709 13:01:18.120678 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:01:18.121441 (+   763us) log.cc:844] Serialized 9008 byte log entry
0709 13:01:19.003713 (+882272us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b1800416080. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:19.746492 (+742779us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:20.329875 (+583383us) op_driver.cc:464] REPLICATION: finished
0709 13:01:20.490286 (+160411us) write_op.cc:301] APPLY: starting
0709 13:01:20.490412 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:20.491050 (+   638us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:20.491102 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:20.494319 (+  3217us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:20.495324 (+  1005us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 13:01:20.495534 (+   210us) write_op.cc:312] APPLY: finished
0709 13:01:20.527611 (+ 32077us) log.cc:844] Serialized 157 byte log entry
0709 13:01:20.595946 (+ 68335us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:20.607463 (+ 11517us) write_op.cc:454] Released schema lock
0709 13:01:20.624675 (+ 17212us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:20.678506 (+ 53831us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:20.679551 (+  1045us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1555726,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":143362,"apply.run_cpu_time_us":8017,"apply.run_wall_time_us":179864,"mutex_wait_us":32837,"num_ops":23,"prepare.queue_time_us":165604,"prepare.run_cpu_time_us":193357,"prepare.run_wall_time_us":496711,"raft.queue_time_us":577563,"raft.run_cpu_time_us":7799,"raft.run_wall_time_us":109590,"replication_time_us":2206892,"spinlock_wait_cycles":107175680,"thread_start_us":1134162,"threads_started":4,"wal-append.queue_time_us":429377,"wal-append.run_cpu_time_us":78188,"wal-append.run_wall_time_us":2882825}]]}
I20260709 13:01:25.352088 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:25.352916 20159 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:25.352770 19840 catalog_manager.cc:5697] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 4 to 6, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 6 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:25.356482 20056 leader_election.cc:336] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223): Timed out: RequestConsensusVote RPC to 127.12.30.68:46223 timed out after 1.840s (SENT)
I20260709 13:01:25.358085 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.360186 23659 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.361711 20013 raft_consensus.cc:1240] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:25.346483 23995 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:25.290405 23934 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:25.267846 23969 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:01:25.360523 24129 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:25.355216 19840 catalog_manager.cc:5697] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.377658 20162 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:25.383394 20016 raft_consensus.cc:2410] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:25.343994 24130 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:25.337419 23758 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:25.372157 23580 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:25.376574 23934 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:25.418190 20011 raft_consensus.cc:2368] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
W20260709 13:01:25.421977 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 62308.7us
I20260709 13:01:25.429332 23991 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.430017 23991 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:25.430538 24064 raft_consensus.cc:3055] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:25.431025 24064 raft_consensus.cc:740] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:25.432813 19840 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 7 to 8, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.435895 24032 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:25.361181 23659 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:25.437664 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 86704.2us
I20260709 13:01:25.442121 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:25.418686 23934 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.448244 20016 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:25.449669 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 88285.5us
I20260709 13:01:25.403954 24130 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:25.456966 23968 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:25.457687 23968 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:25.453233 19840 catalog_manager.cc:5697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:25.472924 20854 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
W20260709 13:01:25.496079 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.496284 20858 raft_consensus.cc:2468] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:25.489097 20011 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:25.508067 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:25.481122 23968 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:25.418752 20091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:25.510932 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:25.521127 23094 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:25.521312 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:25.522159 20160 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.522009 23094 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.526557 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.524375 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:25.528939 23094 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:25.530237 20853 raft_consensus.cc:2410] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:25.532742 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.536108 24010 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:25.541599 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:25.536108 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:25.539549 23968 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.547838 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:25.550487 20012 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:25.547971 23659 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:25.552667 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 57144.2us
I20260709 13:01:25.553957 20743 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:25.556253 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.544277 23094 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.524042 23995 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:25.565779 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 88287.1us
I20260709 13:01:25.538129 24025 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.584374 20858 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:25.585139 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.579466 20850 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:25.595970 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:25.593853 19840 catalog_manager.cc:5697] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 9 to 11, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 11 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.595642 23968 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:25.614785 23968 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:25.616230 24140 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.095s	user 0.002s	sys 0.000s
I20260709 13:01:25.617179 23968 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:25.617574 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.617880 23995 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:25.618177 23917 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:25.618443 23995 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:01:25.614790 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 61230.4us
I20260709 13:01:25.619740 23995 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:25.620460 20743 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:25.620931 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:25.621629 20164 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:25.578977 23042 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:25.623234 23833 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:25.623317 24032 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:25.580096 24025 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:25.624709 24032 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.626609 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:25.627305 20853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 1218) took 1042 ms. Trace:
I20260709 13:01:25.627746 20853 rpcz_store.cc:276] 0709 13:01:24.585305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.590318 (+  5013us) service_pool.cc:224] Handling call
0709 13:01:25.627020 (+1036702us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540027e2d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:25.627221 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1997154176}
I20260709 13:01:25.623982 23042 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:25.631564 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:25.632318 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:25.632995 20089 raft_consensus.cc:2393] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:25.633756 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:25.634668 24046 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:25.635136 24046 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:25.636561 24130 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:25.637197 24130 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:25.637749 24130 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.640522 20091 raft_consensus.cc:606] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.641036 20091 raft_consensus.cc:614] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:25.634630 20082 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:25.644562 20016 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:25.646341 20088 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:01:25.642481 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 291234us
I20260709 13:01:25.652704 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.664223 23869 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:25.665248 23094 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:25.665995 23094 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:25.666206 23833 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:25.667052 23833 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:25.667377 23833 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:25.668553 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:25.668895 23042 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:25.669471 20084 raft_consensus.cc:2410] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:25.664250 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:25.670035 20082 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.664338 23968 consensus_queue.cc:1059] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:25.639047 24064 consensus_queue.cc:260] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:25.672520 23968 raft_consensus.cc:3055] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:25.672667 24064 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.672991 23968 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:01:25.674546 24138 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:01:25.668340 23833 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:25.668503 23094 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:25.680449 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.682451 20160 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:25.684448 20854 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:01:25.631814 24016 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:25.684963 20854 pending_rounds.cc:85] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 7
I20260709 13:01:25.689162 20854 pending_rounds.cc:107] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:01:25.689527 20854 raft_consensus.cc:2889] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:25.687974 20089 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:25.691726 20011 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:25.694593 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.695483 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.696525 20855 raft_consensus.cc:2410] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:01:25.696447 23580 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:25.697676 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:25.697917 20853 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:25.698316 20161 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:25.691039 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:25.692548 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:25.704936 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:25.706138 20163 raft_consensus.cc:2393] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:25.688874 24016 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.690248 23711 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.713662 23958 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:01:25.717229 19840 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 5 to 6, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 6 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:25.698747 23968 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:25.722597 24130 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:25.723338 23659 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.722666 23711 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.727006 23711 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:25.723598 23968 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:25.727730 23833 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.730576 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.733768 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.736342 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.736495 23094 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:25.711621 24016 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:25.747362 20082 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:25.742295 19840 catalog_manager.cc:5697] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 4 to 7, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 7 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:25.740449 24086 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:25.767127 24142 log.cc:927] Time spent T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 13:01:25.773164 23992 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.774645 23992 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:25.741194 23580 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:01:25.776573 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46143.5us
I20260709 13:01:25.778023 23991 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:25.778191 23580 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:25.779196 20130 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:25.746915 24046 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:25.780793 24046 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:25.790608 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 98141.7us
I20260709 13:01:25.792807 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:25.798977 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 188744us
I20260709 13:01:25.800757 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:25.801671 20854 raft_consensus.cc:2393] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:25.796705 24086 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:25.804968 23094 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:25.809276 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:25.810246 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.810364 20850 raft_consensus.cc:2368] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:25.810923 20855 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:25.812420 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.812673 20130 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:25.813901 24086 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:25.814077 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.815721 23659 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:25.816365 23659 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:25.818694 23659 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:25.821238 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:25.816704 24025 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:25.825021 23580 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:25.825614 23580 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:25.826817 20055 leader_election.cc:400] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:25.830681 20055 leader_election.cc:403] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:01:25.831524 24129 raft_consensus.cc:3060] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:25.828449 23580 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:25.836557 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:25.839942 23094 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:25.841760 23094 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:25.842183 23094 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:25.844590 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53020.3us
I20260709 13:01:25.860586 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:25.871347 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 150477us
W20260709 13:01:25.882045 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 146277us
W20260709 13:01:25.883841 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.836946 24032 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:25.887562 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:25.887305 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 48955.7us
I20260709 13:01:25.896764 20085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
W20260709 13:01:25.897275 24136 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:01:25.897588 20085 raft_consensus.cc:606] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.898020 20085 raft_consensus.cc:614] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:25.903076 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:01:25.904726 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:25.906594 23869 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:25.907059 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:25.908541 24146 log.cc:927] Time spent T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:01:25.899796 24130 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:25.912623 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:25.890275 23995 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:25.920688 23580 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.916149 20129 leader_election.cc:400] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12.
I20260709 13:01:25.924705 20129 leader_election.cc:403] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:25.925738 24148 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12.
I20260709 13:01:25.927171 24032 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:25.928767 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:25.916921 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:25.919557 24129 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:25.932278 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:25.936168 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48665.5us
I20260709 13:01:25.936966 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:25.929571 23992 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:25.990374 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53473.4us
I20260709 13:01:25.939042 24152 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:25.931588 23934 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:25.951056 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:25.935616 23731 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:25.900709 24064 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 13:01:26.019994 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 75131.6us
I20260709 13:01:26.023645 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:01:26.024576 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 73745.1us
I20260709 13:01:26.026659 23868 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:26.027292 23868 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.027760 23868 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:25.959192 23758 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:01:26.033043 23758 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:26.033638 23758 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:26.034179 24152 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:26.034621 23758 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.024863 24032 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.035250 24152 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.036513 24032 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.040956 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.042228 24032 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:26.036474 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.957636 24129 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:26.044556 19981 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.047482 19982 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:26.053431 23758 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.060873 23094 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:26.063499 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 116929us
I20260709 13:01:26.076761 23094 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.080220 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:26.081421 20850 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:26.075635 20854 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:26.075328 24032 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:26.085105 24032 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.085526 24032 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:26.082669 23731 raft_consensus.cc:2804] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:26.086750 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.016882 24016 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:26.091885 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 68314.2us
I20260709 13:01:26.093262 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.091142 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.068296 24129 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:26.114809 23094 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:26.115737 23094 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:26.117250 23094 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.089601 23833 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.113020 23731 raft_consensus.cc:697] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:26.127401 20160 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:26.125945 20056 leader_election.cc:400] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:26.131206 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.133569 20164 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:26.134526 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.136992 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.119354 24130 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:26.143741 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.144969 20164 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:26.147186 20850 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:26.145880 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.148725 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.152866 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:26.155764 24150 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.156452 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.157322 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:26.158398 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.159319 20160 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:26.160743 23934 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.161911 23042 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:26.162776 23042 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.138446 23958 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:01:26.169589 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.174098 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:26.155561 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.170267 20161 raft_consensus.cc:1240] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:26.172314 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.173082 23868 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.179252 23868 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.167922 23958 consensus_queue.cc:1243] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:26.181005 24016 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.181691 24016 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:26.155678 24156 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:26.183305 24150 consensus_queue.cc:1059] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:26.184458 24129 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.185738 24150 raft_consensus.cc:3055] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:26.186168 24150 raft_consensus.cc:740] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:26.187026 24150 consensus_queue.cc:260] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.188746 23995 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.189311 23995 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:26.190013 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.189636 23042 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.190623 23731 consensus_queue.cc:237] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.156694 24160 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.182569 24064 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:26.212339 20162 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:01:26.219376 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:26.220425 20159 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:26.220448 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:26.224586 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 75587.6us
I20260709 13:01:26.212916 20162 pending_rounds.cc:85] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 7
I20260709 13:01:26.225687 20162 pending_rounds.cc:107] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:01:26.225988 20162 raft_consensus.cc:2889] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.182214 24016 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:26.211246 24140 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:01:26.200693 23958 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.141115 23833 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.219655 24150 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:26.183307 24156 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:26.203642 23992 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:01:26.223421 23659 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:01:26.267293 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 118639us
W20260709 13:01:26.271046 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:26.281936 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.284289 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:26.285866 20016 raft_consensus.cc:2410] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:26.243927 24160 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:26.300127 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 74743.2us
I20260709 13:01:26.262260 23958 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.257069 24016 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.312390 20016 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:26.314046 20163 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:26.326817 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.327744 20159 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
W20260709 13:01:26.338649 24165 log.cc:927] Time spent T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
I20260709 13:01:26.318830 23868 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:01:26.350921 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 49192.7us
I20260709 13:01:26.353713 20016 raft_consensus.cc:1240] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:26.354329 24160 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.354823 23711 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:26.355468 23711 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.312544 24156 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.358003 23711 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.360548 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.362618 23995 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.366854 20016 raft_consensus.cc:1275] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:01:26.367839 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.352891 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.381019 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.381245 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.382007 20855 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:26.382233 20857 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 7: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:26.386088 23094 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:26.390592 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.390882 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.315167 24153 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:01:26.400410 20162 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:26.400715 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.398859 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.393899 23995 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:26.395836 23094 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.413656 20014 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:26.415781 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:26.416038 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.416253 23995 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:26.417251 23995 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.418223 23958 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.420310 24152 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.421037 23094 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.427865 23711 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.430786 23711 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.431243 23711 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:26.438354 24163 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:26.426563 23833 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:26.446993 19982 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.444061 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.466987 24130 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:26.467247 24168 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.469224 20164 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:01:26.467742 24130 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.467797 24168 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.473114 20130 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:26.475716 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52081.6us
I20260709 13:01:26.475713 23094 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:26.421275 24152 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:26.485708 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:26.485514 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.475775 23042 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.464596 23580 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:26.486821 23094 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.473870 24032 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.482213 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.493592 20130 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:26.494438 24143 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:26.495471 24086 consensus_queue.cc:1059] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:26.496001 23711 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.496500 24178 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:26.426226 24064 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:01:26.522292 24178 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.522447 23042 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.524914 24178 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.525033 23042 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.525519 24178 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.525727 23042 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:26.526078 24178 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.481580 24010 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.528266 23094 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.530630 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.535295 20163 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:26.526351 23042 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.535732 20857 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:26.545367 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.546361 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:26.547333 20082 raft_consensus.cc:2410] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:26.544651 23094 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.546718 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:26.549868 23094 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.550354 20089 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:26.551795 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.554713 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.539274 23580 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:26.555842 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:26.557183 24178 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.558132 24130 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.558732 23094 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.560482 24186 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.561103 24186 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.563468 24186 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.566571 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:26.537295 24064 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:26.570814 20088 raft_consensus.cc:2410] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:01:26.573745 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80234.9us
I20260709 13:01:26.567409 20084 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 7.
I20260709 13:01:26.575291 20130 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.569800 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:26.549062 23758 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:26.588608 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:26.588532 23094 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.587482 20085 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:26.586795 24130 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:26.597790 24130 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.598342 24130 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:26.539532 24010 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.602227 24010 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:26.602530 23711 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.603528 24086 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:26.609161 23042 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:26.609802 24174 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:01:26.613263 24168 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.585249 24160 consensus_queue.cc:1048] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:01:26.628105 24168 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.628738 24168 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.630884 24168 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.632130 24059 raft_consensus.cc:3055] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:26.632701 24059 raft_consensus.cc:740] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:26.633924 24059 consensus_queue.cc:260] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.656500 24168 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.657330 24168 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.659729 24168 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.663100 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:26.663782 20853 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:26.664446 24168 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.665004 24168 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.667092 24032 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.667299 24168 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:26.669566 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.671069 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.672254 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:26.672673 20853 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:26.672770 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.673975 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.674440 23968 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.674747 20857 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:26.675037 23968 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.675956 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.676769 20855 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:26.677490 23968 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:26.678287 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.686537 20857 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:26.687745 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.688020 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.688458 20859 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:26.688727 20855 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:26.689286 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:26.689834 23968 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.690213 20165 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:26.690583 20055 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.690367 23968 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.691661 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.693176 24178 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.693675 24178 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.694094 24178 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.694574 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.695968 24160 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:26.696538 24160 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:26.696689 24168 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.697263 24168 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.697484 24059 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.697669 24160 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.697748 24168 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:26.696038 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.701900 24032 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:26.706507 23042 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:26.706879 23042 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:26.704319 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.707620 23042 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.724926 23042 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:26.726246 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 240936us
I20260709 13:01:26.730319 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:26.738128 23094 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.738723 23094 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.738698 19982 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:26.725800 23968 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:26.731017 23958 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:26.770001 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.778350 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:26.779428 20011 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 8.
I20260709 13:01:26.738031 24032 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:26.787088 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:26.764230 24186 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.793330 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:26.737246 23992 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:01:26.795856 23094 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:26.800735 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 315117us
W20260709 13:01:26.806002 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 251363us
I20260709 13:01:26.806598 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.806824 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:26.768471 24175 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.763278 23934 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:01:26.789394 24032 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.798074 23711 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:26.797837 24186 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.826838 20859 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:26.828866 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.790565 23958 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.822701 24032 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.827576 23711 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.825891 24016 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:26.844444 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:26.845160 24129 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:26.828485 24186 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:26.823267 24175 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.839195 24153 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.878695 24189 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.850112 23711 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:26.878911 24150 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:01:26.879303 24189 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.876906 24129 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:26.885852 23958 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.846237 24195 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:01:26.850051 24016 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:26.891002 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44941.8us
I20260709 13:01:26.894057 20855 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
W20260709 13:01:26.896327 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 52033.5us
I20260709 13:01:26.900130 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:26.901134 23042 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.901513 23042 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.901901 23042 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:26.901257 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:26.903638 23094 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:26.904174 23094 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:26.904554 23094 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:26.903782 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.890475 23731 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:26.894678 24189 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:26.886032 24188 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:26.887225 24130 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.920285 24129 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:26.907382 23958 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:01:26.923607 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.898523 24016 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.929755 23094 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.930996 23042 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:26.899140 24195 consensus_queue.cc:1243] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:26.937613 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:26.949649 23042 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.933104 23869 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.932775 24188 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.953994 23042 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:26.954643 23042 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.954710 23094 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:26.955308 23042 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:26.958436 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 54748.6us
I20260709 13:01:26.958595 23869 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:26.959478 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.961038 23869 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:26.961164 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:26.966760 23094 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:26.964485 23580 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:26.970211 24195 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.933897 23731 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.975023 23731 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:26.976828 20859 pending_rounds.cc:85] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 6
I20260709 13:01:26.977288 20859 pending_rounds.cc:107] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:26.977531 20859 raft_consensus.cc:2889] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.986686 24059 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:26.989840 24190 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:01:26.995606 24156 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:26.996273 24156 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:26.997174 24156 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:26.998486 24156 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:27.002251 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.002993 20086 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:27.004304 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.004973 20159 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:27.005795 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:27.015697 20857 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:27.039779 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.041986 20850 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:27.005692 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.040593 20164 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:27.048457 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:27.042589 19829 maintenance_manager.cc:419] P d84e5f78c3264c738b7966fc230d4878: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033346
I20260709 13:01:27.051021 20083 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:26.964653 24160 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.052543 24160 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:27.055442 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49841.7us
I20260709 13:01:26.957263 24188 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:26.987381 24059 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.068780 20090 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:27.054088 24192 log.cc:927] Time spent T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.168s	user 0.003s	sys 0.001s
I20260709 13:01:27.089918 20158 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:01:27.091581 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.003716 24168 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.097759 20083 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:27.104380 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.113839 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:27.114523 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 153464us
I20260709 13:01:27.115828 23042 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:27.117882 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.118012 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.118820 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.119072 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.119266 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.119482 20008 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:27.120234 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.120379 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.121852 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.123477 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.124287 20017 raft_consensus.cc:2376] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11. Re-sending same reply.
I20260709 13:01:27.125586 23934 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:27.096999 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.126417 23934 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:27.129899 20090 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:27.137624 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:27.138600 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.071861 23731 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:27.137307 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:27.144960 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72004.4us
I20260709 13:01:27.138092 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:27.146211 20085 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:27.146690 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.147676 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:27.148518 20856 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:27.111545 23968 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:27.156042 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.156855 20090 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:27.167250 23042 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:27.094944 24178 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.177258 23094 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:27.178078 23094 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.141237 24204 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:27.185359 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:27.153643 24130 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:27.205205 20083 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:01:27.205752 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:27.206534 20856 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:27.212376 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.211642 20016 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:27.216006 20129 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.219815 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.221800 20012 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.210954 24186 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.226544 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.154675 23731 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:27.228633 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.230764 20084 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:27.232195 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.232682 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.233377 20082 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 8: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:27.237468 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.238296 20017 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:27.239882 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.243748 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.251744 20016 raft_consensus.cc:2393] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:27.256970 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.257421 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.258038 20084 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:27.259218 24186 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:27.189469 24204 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:27.260330 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.261011 20082 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:27.262681 24204 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:27.263386 24213 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:27.267314 24163 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:27.267493 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.268666 20012 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:27.254405 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.269536 24010 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:27.270639 24010 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:27.271001 24010 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.275332 24168 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:27.270109 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.279253 20085 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:27.277724 20091 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:27.282182 20856 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:27.296218 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.298257 20083 raft_consensus.cc:1240] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:27.300654 24206 log.cc:927] Time spent T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:01:27.302791 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:27.303148 24203 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:27.303782 24203 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.304198 24203 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:27.308471 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.309705 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.310782 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.312345 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.312695 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.313009 20009 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:27.313807 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.314853 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.244982 24053 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:01:27.316316 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.311843 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:27.320013 24201 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:27.321710 20083 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:27.323767 20009 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:27.317600 20012 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:27.325397 20012 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:27.312361 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:27.295125 24059 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:27.318471 24150 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:27.295467 23958 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.319155 24199 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:27.321600 23992 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:27.331184 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:27.335450 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 102851us
I20260709 13:01:27.336195 20091 raft_consensus.cc:2410] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:01:27.336351 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:27.340724 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:27.336640 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:27.349509 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:27.317476 24175 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:27.287025 24168 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.292771 23580 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:27.323230 24016 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.380178 20129 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.381990 20087 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:27.276208 24032 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.383271 24129 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:27.319752 24064 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:01:27.403362 20082 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:27.403807 20091 raft_consensus.cc:1240] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:27.384802 24059 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:27.393378 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:27.337792 23992 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:27.394811 20008 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:27.377226 24130 consensus_queue.cc:1059] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:01:27.369403 23958 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.394309 24204 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:27.384853 24168 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.404501 23934 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:27.448825 24210 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 13:01:27.450028 24212 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:01:27.450098 24032 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:27.450978 23711 consensus_meta.cc:287] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.006s
I20260709 13:01:27.451468 23711 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:27.452114 24218 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:01:27.454383 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:27.455466 20854 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:27.457697 23711 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:27.423969 24064 consensus_queue.cc:1243] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:27.465719 24168 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:27.412393 24129 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:27.466922 24129 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:27.468971 24010 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.476789 23992 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.478789 23992 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.427680 24186 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:27.482456 24204 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:27.484145 24204 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.491597 23731 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:27.495348 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 109641us
I20260709 13:01:27.496604 20743 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:27.413582 24059 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:27.498276 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:27.499384 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:27.501333 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:27.503783 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 277312us
I20260709 13:01:27.504961 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:27.510311 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:27.400694 24150 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:01:27.520545 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:27.522851 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:27.535861 20850 raft_consensus.cc:1275] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:27.498994 24148 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.520116 24189 consensus_meta.cc:287] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
I20260709 13:01:27.518663 24175 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:27.498752 24186 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:27.518867 24059 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:27.521700 24156 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.523s	user 0.015s	sys 0.001s
I20260709 13:01:27.545079 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:27.520246 24010 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:27.520056 24168 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:27.539449 24130 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:27.534729 24195 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:27.555429 20014 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:27.552562 20853 raft_consensus.cc:1217] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[10.7-12.8]   Dedup: 12.8->[]
W20260709 13:01:27.554140 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 143414us
I20260709 13:01:27.581508 20853 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:01:27.547037 24175 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:27.603015 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 265600us
W20260709 13:01:27.609381 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:27.604393 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 94217.2us
W20260709 13:01:27.666461 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 69297.5us
I20260709 13:01:27.667677 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.548647 24189 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.583046 23833 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.580571 24130 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:27.604355 24175 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:27.688529 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 160874us
I20260709 13:01:27.547447 24148 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.600692 24168 raft_consensus.cc:697] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:27.699791 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.700642 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:27.701058 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.702459 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.706162 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.707122 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.709363 20090 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:01:27.712116 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.692099 24130 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.682760 23869 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:27.716243 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.718109 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.719722 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.695063 24216 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:27.722625 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:27.717398 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:27.727347 20165 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.596778 24195 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:27.773229 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:27.737164 20156 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:27.780462 20163 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:27.774143 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 90268.6us
W20260709 13:01:27.776252 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 234554us
I20260709 13:01:27.778353 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:27.784365 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:27.695135 24213 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:27.696895 23833 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:27.782388 20744 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.730329 23869 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:27.815364 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.817773 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:27.818979 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 125606us
I20260709 13:01:27.820591 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:27.792018 24228 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:27.855086 24228 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:27.855752 24228 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:27.855474 23869 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:27.808074 23731 consensus_queue.cc:1059] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:01:27.860587 23731 raft_consensus.cc:3055] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:27.860970 23731 raft_consensus.cc:740] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:01:27.811966 24203 consensus_meta.cc:287] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.012s
I20260709 13:01:27.874995 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:27.878451 20857 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:27.880493 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 60169.3us
I20260709 13:01:27.881902 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:27.883194 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 182640us
I20260709 13:01:27.887017 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.771073 24214 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.340s
I20260709 13:01:27.889335 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:27.721783 24186 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.892839 20743 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:27.879580 24216 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:27.895459 24222 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.898885 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:27.902467 24207 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:27.911695 20163 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:27.887866 24010 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:27.886904 23995 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.902099 24235 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.925297 24141 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.926249 24141 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:27.928952 24141 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:27.812067 24213 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:27.934191 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 266583us
I20260709 13:01:27.938968 20163 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:27.946180 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:27.879426 24203 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.851233 24168 consensus_queue.cc:237] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:27.960212 24231 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.960621 24242 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:27.961207 24231 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:27.921613 24010 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:27.960389 24241 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:27.963106 20158 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:27.969336 24203 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:27.972769 20165 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:27.954337 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:27.982730 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:27.984937 24248 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:27.986027 24248 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:27.988197 24232 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:01:27.893760 23731 consensus_queue.cc:260] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.990628 23731 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.994263 24202 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:27.957158 24204 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:27.953670 24207 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:27.974387 24227 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.393s	user 0.000s	sys 0.002s
I20260709 13:01:27.903550 24222 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:27.956661 23869 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:28.004577 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 111813us
I20260709 13:01:27.976244 24010 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:27.934721 23995 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.009784 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:27.978971 24189 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:28.015650 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.995817 24202 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:01:28.018448 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 228155us
W20260709 13:01:28.012667 23992 consensus_meta.cc:287] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.529s	user 0.015s	sys 0.000s
I20260709 13:01:28.033054 20017 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:01:28.009913 24233 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 13:01:28.025673 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:28.028479 20083 raft_consensus.cc:1240] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:28.031383 20086 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:27.938299 24235 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.010970 24241 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:28.008844 23869 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:28.038987 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.042148 20086 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:28.039903 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 57290.8us
I20260709 13:01:27.973459 24236 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.168s
I20260709 13:01:28.035907 19981 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:28.016995 24148 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.064677 24195 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:28.064909 23934 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:28.065511 23934 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.065939 23934 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:28.066088 24245 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:28.066895 24249 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.010715 24207 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.072026 24203 consensus_queue.cc:1048] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:28.073117 24207 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.075203 23995 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.075867 23995 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:28.076274 23995 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:28.076759 23995 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:28.042711 24188 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.007s
I20260709 13:01:28.047010 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.091225 20086 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
W20260709 13:01:28.091374 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 44491.5us
I20260709 13:01:28.085177 24188 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:28.090112 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52243us
W20260709 13:01:28.092624 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46154.5us
I20260709 13:01:28.090689 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.097797 20083 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:28.099809 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:28.033615 23833 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.038935 24241 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.105005 24214 consensus_queue.cc:1059] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:28.106498 24255 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:28.106900 24255 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:28.108076 24255 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.109776 24255 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:28.046713 23869 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.116379 24213 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:28.117416 24129 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.003s
I20260709 13:01:28.098739 24189 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.085841 24235 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.120355 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1177) took 1048 ms. Trace:
I20260709 13:01:28.120347 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.089298 23995 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.124107 20852 rpcz_store.cc:276] 0709 13:01:27.071872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.454338 (+382466us) service_pool.cc:224] Handling call
0709 13:01:27.454703 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.455142 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.455176 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.455696 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.553705 (+ 98009us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.553765 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.553872 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.120078 (+566206us) raft_consensus.cc:1685] finished
0709 13:01:28.120108 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.120270 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1221,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"thread_start_us":93745,"threads_started":1,"wal-append.queue_time_us":228266}
I20260709 13:01:28.096056 24203 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:01:28.131259 24251 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:01:28.131740 24216 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:28.130694 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:28.112656 24241 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.144032 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.146203 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.147723 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:28.148916 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:28.150465 24195 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.151177 24195 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:28.151443 24243 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:28.151981 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:28.152134 24243 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:28.153203 24243 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:28.154557 20159 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:28.154901 24059 consensus_meta.cc:287] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.003s
I20260709 13:01:28.155205 24239 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:28.134589 24204 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:28.155956 24235 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:28.151819 24195 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:01:28.157711 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.158529 20159 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:28.161556 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.161857 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.162724 20086 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:28.155318 24059 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.117937 24129 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.122246 24175 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.170465 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.158749 24189 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:28.182153 20157 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 7.
I20260709 13:01:28.186139 23833 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:28.226280 24130 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.184398 24010 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:28.235631 24209 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.928s	user 0.003s	sys 0.000s
I20260709 13:01:28.237704 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 737) took 1120 ms. Trace:
W20260709 13:01:28.240216 24205 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.991s	user 0.007s	sys 0.009s
I20260709 13:01:28.252950 20010 rpcz_store.cc:276] 0709 13:01:27.117003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.132682 (+ 15679us) service_pool.cc:224] Handling call
0709 13:01:27.132978 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.251903 (+118925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.252024 (+   121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:27.252086 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.252127 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.237419 (+985292us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540064f5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:28.237474 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.237609 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272957312}
I20260709 13:01:28.262264 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1173) took 1250 ms. Trace:
I20260709 13:01:28.230584 23995 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.259347 24249 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:28.262862 24188 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:28.256244 24129 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.256206 24010 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.262758 24059 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.271371 20851 rpcz_store.cc:276] 0709 13:01:27.012002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.012287 (+   285us) service_pool.cc:224] Handling call
0709 13:01:27.045850 (+ 33563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.046295 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.046328 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.046879 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.137842 (+ 90963us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:27.372541 (+234699us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.372852 (+   311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.372973 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:28.234870 (+861897us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:28.261876 (+ 27006us) raft_consensus.cc:1685] finished
0709 13:01:28.262008 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.262179 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14476,"prepare.queue_time_us":65914,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4995,"raft.queue_time_us":5285,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":49471104,"thread_start_us":263021,"threads_started":2,"wal-append.queue_time_us":260275}
I20260709 13:01:28.256851 24258 consensus_queue.cc:1059] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.116s
I20260709 13:01:28.314759 24258 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:28.315296 24258 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.316428 24258 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.318100 24258 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.328112 20087 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1521) took 1194 ms. Trace:
I20260709 13:01:28.328677 20087 rpcz_store.cc:276] 0709 13:01:27.133812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.133953 (+   141us) service_pool.cc:224] Handling call
0709 13:01:27.150291 (+ 16338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.387791 (+237500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.327731 (+939940us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54000cd5d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:28.328024 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532218240}
I20260709 13:01:28.329319 20012 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.330798 20012 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.286860 24130 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.253263 24262 consensus_queue.cc:1059] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 13:01:28.333392 23992 raft_consensus.cc:3055] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:28.333803 23992 raft_consensus.cc:740] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.334832 23992 consensus_queue.cc:260] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.290407 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:28.337260 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46992.8us
I20260709 13:01:28.261693 23934 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.340516 23934 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.285013 23869 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.289448 20082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1526) took 1013 ms. Trace:
I20260709 13:01:28.343876 24129 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:28.344864 24129 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.344261 20082 rpcz_store.cc:276] 0709 13:01:27.276143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.278005 (+  1862us) service_pool.cc:224] Handling call
0709 13:01:27.279341 (+  1336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.404072 (+124731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.289139 (+885067us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400771d54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:28.289353 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284485504}
I20260709 13:01:28.345569 24129 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.348397 24129 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.262919 24228 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.354990 24228 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.343565 23869 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.360030 24202 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:28.361704 24202 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:28.366179 20012 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:28.367509 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1146) took 1057 ms. Trace:
I20260709 13:01:28.367959 20012 rpcz_store.cc:276] 0709 13:01:27.309820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.317197 (+  7377us) service_pool.cc:224] Handling call
0709 13:01:27.317464 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.328954 (+1011490us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400420a20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e73006f 00007fa02e7a8f60 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62
0709 13:01:28.366972 (+ 38018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.367427 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313604352}
W20260709 13:01:28.374085 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 240362us
I20260709 13:01:28.376878 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.377849 20851 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:28.378702 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.379084 24267 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.379333 20082 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:28.389202 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.379627 24267 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:28.392130 20851 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:28.381593 24188 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:28.390218 20859 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
W20260709 13:01:28.402169 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 204980us
W20260709 13:01:28.403636 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.405797 24267 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:28.406255 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.407331 20082 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:28.407537 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:28.408982 20859 raft_consensus.cc:2368] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
W20260709 13:01:28.412572 20744 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 2.146s (SENT) [suppressed 11 similar messages]
I20260709 13:01:28.413975 24236 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:28.414770 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.416957 20087 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:28.419457 24222 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.423610 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:28.424342 20852 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:28.428929 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.429107 24016 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:28.429389 20163 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:28.414587 24236 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.432415 20008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 745) took 1312 ms. Trace:
I20260709 13:01:28.432937 20008 rpcz_store.cc:276] 0709 13:01:27.119551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.120139 (+   588us) service_pool.cc:224] Handling call
0709 13:01:28.431974 (+1311835us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54002c9554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:28.432345 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629557504}
I20260709 13:01:28.434810 23094 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:28.436370 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:28.437569 20164 raft_consensus.cc:2393] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:28.438194 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.439674 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.440896 20851 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:28.442543 20856 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:28.445102 24242 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:28.445569 24242 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:28.446439 24242 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.400060 24188 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.448750 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.449905 20156 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:28.451042 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:28.461135 20164 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:01:28.461349 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52805.8us
I20260709 13:01:28.456369 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.448413 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:28.462306 20082 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:28.462723 20160 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.449175 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.463819 20163 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:28.464828 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.465327 20163 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:28.415716 20015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 1142) took 1107 ms. Trace:
I20260709 13:01:28.466921 20015 rpcz_store.cc:276] 0709 13:01:27.308671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.312043 (+  3372us) service_pool.cc:224] Handling call
0709 13:01:28.414602 (+1102559us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x7b5800867998. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b68db71 00007fa02b68da22 00007fa0345a6c09 00007fa033700ee0 00007fa0337ca0ab 00007fa0337ca131 00007fa0337780a0 00007fa033777984 00007fa02ccf26d4 00007fa02ccf2662
0709 13:01:28.415640 (+  1038us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13983488}
W20260709 13:01:28.472383 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.474356 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.474623 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.475064 20012 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:28.475327 20010 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:28.476172 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.476239 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.476891 20012 raft_consensus.cc:2368] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13.
I20260709 13:01:28.477087 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.477938 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.479252 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.479730 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.480898 24199 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:28.481551 24199 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.481683 24010 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:28.447887 24242 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:28.486904 23731 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:28.492261 24199 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.438248 24175 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.483945 20129 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:28.496579 24010 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.504652 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.499933 23731 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:28.514644 24211 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 1.215s	user 0.001s	sys 0.004s
I20260709 13:01:28.504142 24269 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:28.516368 24269 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.517048 24269 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.517409 24010 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.517529 24241 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.496259 24259 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:28.520334 24259 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:28.520908 24259 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:28.523627 24201 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:28.508143 24225 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.903s	user 0.003s	sys 0.000s
I20260709 13:01:28.526366 23731 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.528668 24241 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.529362 23042 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:28.529794 23042 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:28.530540 24236 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.531389 24188 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.532829 23992 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.536273 20129 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:28.537173 24059 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:28.537554 24059 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.537871 24059 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:28.526643 24270 log.cc:927] Time spent T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:01:28.543118 20090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1523) took 1353 ms. Trace:
I20260709 13:01:28.537881 24231 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.543566 20090 rpcz_store.cc:276] 0709 13:01:27.190073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.190266 (+   193us) service_pool.cc:224] Handling call
0709 13:01:27.190641 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.715178 (+524537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:28.542837 (+827659us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b5400782654. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:28.543047 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191328128}
I20260709 13:01:28.545743 24150 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:28.530553 23042 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:28.549304 23042 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.545830 19982 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:28.553638 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.555379 24230 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:28.587085 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:28.593029 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.577669 20852 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:28.570560 20160 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:28.597604 20129 leader_election.cc:400] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
I20260709 13:01:28.598582 20129 leader_election.cc:403] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:28.559206 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.566632 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.568606 20163 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:28.604503 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.607645 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:28.609095 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.609907 20012 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:28.617681 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.620322 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.622467 20165 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:28.625244 20164 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:28.627132 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.631398 20851 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:28.613847 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:28.631992 20017 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:28.558310 24046 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:28.557768 24143 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:28.633308 24201 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:28.632686 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:28.634091 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:28.635299 20852 raft_consensus.cc:3060] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.624509 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:28.637096 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.638650 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:28.640075 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.555863 24230 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.556181 24150 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:28.646404 19982 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:28.571350 24086 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:28.654471 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.558504 24231 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:28.661307 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.662199 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.661778 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.663517 24200 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:28.664062 24271 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.664753 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.664687 24221 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:01:28.723012 20852 raft_consensus.cc:2468] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:28.675288 24230 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:28.626823 24129 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:28.696285 24191 raft_consensus.cc:2804] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:28.604147 24188 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:28.674896 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:01:28.699824 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 287261us
W20260709 13:01:28.752228 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 77453.7us
I20260709 13:01:28.555635 24059 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.606901 24016 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:01:28.650384 24241 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:28.727819 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 64910.3us
I20260709 13:01:28.675027 24267 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:01:28.665442 20163 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:28.673744 24175 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:28.657069 24152 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:28.633626 24046 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.633606 24143 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.675980 24064 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:01:28.640714 24201 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.759744 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:28.748207 24232 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:28.775270 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.734771 24231 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.781188 23042 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:28.724720 24141 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:28.792637 23042 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.747263 24129 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:28.791029 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:28.757503 24150 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:28.789629 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:28.801362 20857 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.765620 24241 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.788894 24236 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:28.798307 20850 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:28.803224 20851 raft_consensus.cc:2410] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:28.769908 24191 raft_consensus.cc:697] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.732722 24086 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:28.808822 20014 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57974 (request call id 733) took 1695 ms (client timeout 2146 ms). Trace:
I20260709 13:01:28.770756 24010 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 7. Current term is 8.
W20260709 13:01:28.829054 20014 rpcz_store.cc:269] 0709 13:01:27.113093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.113565 (+   472us) service_pool.cc:224] Handling call
0709 13:01:28.808517 (+1694952us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b540073f0d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:28.808718 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013707136}
I20260709 13:01:28.818229 23042 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.776194 24152 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.726153 24271 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:28.796236 20744 leader_election.cc:400] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13.
I20260709 13:01:28.841341 20744 leader_election.cc:403] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:01:28.841867 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 56239.7us
I20260709 13:01:28.715116 24200 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:28.818446 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:28.826402 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 45603.3us
I20260709 13:01:28.764233 24202 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:01:28.762452 24188 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.817400 20013 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:01:28.833021 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 176792us
I20260709 13:01:28.800920 24129 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:28.819043 24241 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.837792 24046 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.817492 24153 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:28.787012 24160 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.824849 24236 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:28.853544 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.768748 23731 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:01:28.862610 20156 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:28.878703 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.863579 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:28.865715 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:28.850764 24150 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.868005 24086 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.841619 24152 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.871634 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.872383 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.872498 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.805413 24141 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.842379 24178 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.852017 24222 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13.
I20260709 13:01:28.922519 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.932685 24150 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:28.938530 20857 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:28.939778 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:28.940625 23042 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:28.942121 24200 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:28.942298 23042 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:28.910193 23731 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:28.905777 23958 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:28.943360 23731 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.944885 23731 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:28.881783 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.947691 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.950366 20085 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:28.951579 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.952304 20086 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:28.954802 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.886163 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.874902 24153 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.957221 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.957595 24153 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.909430 23580 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:01:28.958639 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.958726 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.959630 20088 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:28.959825 23958 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:28.960240 23958 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:28.960322 24059 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.961148 23958 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.884434 24201 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.899720 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.901706 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:28.963435 20850 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:28.904526 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:28.904773 20858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1172) took 1893 ms. Trace:
I20260709 13:01:28.877339 24236 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:28.877311 24199 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:28.912153 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:28.913357 20016 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:01:28.947939 20082 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:01:28.955577 20089 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:28.948161 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.958952 24153 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:28.968746 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:28.893232 24191 consensus_queue.cc:237] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:28.965426 20014 raft_consensus.cc:1240] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:28.942762 23042 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:28.861238 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.940498 24188 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:28.979741 20083 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:28.980989 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.969619 20850 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:28.970685 20082 pending_rounds.cc:85] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 6
I20260709 13:01:28.982774 20082 pending_rounds.cc:107] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:01:28.946962 23731 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:28.964681 20858 rpcz_store.cc:276] 0709 13:01:27.011690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.012216 (+   526us) service_pool.cc:224] Handling call
0709 13:01:27.500267 (+488051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:27.500498 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.500522 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:27.500606 (+    84us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:27.500663 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.500732 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:28.904574 (+1403842us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400008cd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:28.904706 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065695104}
I20260709 13:01:28.974993 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:28.978120 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.974455 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:28.983093 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 121953us
I20260709 13:01:28.983139 20082 raft_consensus.cc:2889] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:28.959837 24231 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:28.990469 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:28.992455 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:28.993479 24207 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:29.001174 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:28.999236 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:29.002760 23042 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:28.989391 24010 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:29.008144 24010 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.008646 24010 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:29.011212 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:29.012307 24143 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:29.012694 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:29.013384 20009 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:29.007304 24228 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:29.013741 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:29.014180 24228 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.017040 24228 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:29.020021 24129 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:29.021744 20085 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:29.023494 24129 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.024152 23833 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:29.025535 24230 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:29.027611 20011 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:28.996579 24016 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:29.030790 24230 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.033253 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:29.036356 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:29.038175 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:29.033370 24271 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.042649 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:29.043682 20163 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:29.043273 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:29.051677 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:29.052364 20082 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:29.053401 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:28.971944 24199 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:29.059485 20009 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:29.060674 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:29.070068 20160 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:29.070822 20014 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:29.072258 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:29.075270 20008 raft_consensus.cc:2410] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:29.078047 20165 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:01:29.082558 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:29.096113 20082 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:29.084421 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:29.108263 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:29.004105 24178 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:01:29.113680 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:29.116895 24242 consensus_meta.cc:287] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.616s	user 0.003s	sys 0.009s
I20260709 13:01:29.125849 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.134963 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:29.138906 20851 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:01:29.139500 20851 pending_rounds.cc:85] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 7
I20260709 13:01:29.139782 20851 pending_rounds.cc:107] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:01:29.140126 20851 raft_consensus.cc:2889] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:29.152303 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:29.040508 24259 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.158252 20129 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:29.174193 24156 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:29.174918 24156 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:29.179987 24207 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.180759 24207 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:29.186173 24274 log.cc:927] Time spent T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.173s	user 0.002s	sys 0.000s
I20260709 13:01:29.110515 24231 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:29.122674 24269 consensus_meta.cc:287] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.003s
I20260709 13:01:29.189148 20013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 746) took 1158 ms. Trace:
I20260709 13:01:29.192569 20013 rpcz_store.cc:276] 0709 13:01:28.030233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.030367 (+   134us) service_pool.cc:224] Handling call
0709 13:01:28.812013 (+781646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:28.851182 (+ 39169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.188924 (+337742us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b540073f0d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:29.189076 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796751616}
W20260709 13:01:29.156435 24227 log.cc:927] Time spent T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:01:29.196527 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 317341us
I20260709 13:01:29.188257 23711 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.105904 24188 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:01:29.125911 24280 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:29.198309 20162 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1355) took 2197 ms. Trace:
I20260709 13:01:29.191268 24221 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:29.164927 24160 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:29.233273 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 73369.2us
I20260709 13:01:29.191327 24153 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.192188 24231 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:29.195966 23917 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:29.188948 24016 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:01:29.233456 24262 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.256582 20162 rpcz_store.cc:276] 0709 13:01:27.000523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.009283 (+  8760us) service_pool.cc:224] Handling call
0709 13:01:27.009746 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.010280 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:27.073497 (+ 63217us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.409785 (+336288us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.423390 (+ 13605us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.423632 (+   242us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:27.428231 (+  4599us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.044911 (+1616680us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b180010cc00. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:01:29.193822 (+148911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:29.193903 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:29.196003 (+  2100us) raft_consensus.cc:1685] finished
0709 13:01:29.196027 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:29.198222 (+  2195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86703,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":81,"raft.queue_time_us":49403,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":45781,"spinlock_wait_cycles":5984000,"thread_start_us":407308,"threads_started":2,"wal-append.queue_time_us":367637,"wal-append.run_cpu_time_us":3341,"wal-append.run_wall_time_us":1015251}
I20260709 13:01:29.309299 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.256333 24280 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:29.193974 24269 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.255749 24148 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:01:29.294212 24231 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:29.388247 20161 raft_consensus.cc:1240] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:29.318771 24280 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:29.303716 24153 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:29.392375 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 162559us
I20260709 13:01:29.359652 23711 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:29.396585 24257 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:29.400064 19981 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.316488 24262 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:29.437587 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.447435 23042 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:29.440631 24237 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.401031 23934 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:01:29.474499 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.482199 24242 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:29.484086 24148 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:29.484851 24148 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:29.487064 24152 consensus_meta.cc:287] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.000s
I20260709 13:01:29.488101 24283 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:29.488597 24269 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:29.489389 24259 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:29.491861 24141 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:29.499238 24053 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:29.500039 24053 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:29.501024 24053 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:29.502205 24288 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:29.504323 24153 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.449370 24199 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:29.509688 20130 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:29.519891 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 394130us
I20260709 13:01:29.525692 23042 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:29.528168 23042 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:29.530483 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:29.533955 23042 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:29.487574 24148 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:29.613029 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 131078us
I20260709 13:01:29.444196 24257 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.621853 23042 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:29.478579 23731 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:29.533421 20056 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:29.632536 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 99247.5us
I20260709 13:01:29.633332 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:29.634774 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.635856 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.637579 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:29.642748 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:29.645452 20010 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:29.530925 24141 raft_consensus.cc:2804] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:29.649876 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:29.650731 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:29.652982 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:29.661769 20856 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:29.662753 20084 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:29.654131 20163 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:01:29.657599 20083 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:29.612231 24130 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:29.612643 24199 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.666687 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:29.669909 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:29.672328 20089 raft_consensus.cc:3060] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:01:29.673803 24241 consensus_meta.cc:287] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.797s	user 0.009s	sys 0.002s
I20260709 13:01:29.674280 24241 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:29.676265 24141 raft_consensus.cc:697] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:29.677547 24141 consensus_queue.cc:237] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.668329 24130 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:29.679988 24130 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:29.668891 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:29.681252 24207 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:29.681284 24130 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:29.684856 24258 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:29.685333 24258 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:29.686295 24258 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:29.692544 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:29.693142 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:29.698276 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:29.693190 20089 raft_consensus.cc:2468] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:29.698292 20853 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:29.487596 24152 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:29.657720 24202 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.645188 24148 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:29.673803 24010 consensus_meta.cc:287] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.000s
I20260709 13:01:29.770843 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:29.667346 24279 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:29.767938 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 388424us
W20260709 13:01:29.773087 24285 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.260s	user 0.000s	sys 0.000s
I20260709 13:01:29.666951 23731 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:29.773497 24259 consensus_queue.cc:1059] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:01:29.774156 24202 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:29.766793 24207 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:29.777264 24152 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:29.778314 20017 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:29.780853 24241 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:29.781085 24148 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:29.780865 24262 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.705161 24199 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:29.784389 24294 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:29.787406 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:29.788648 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 72842.8us
I20260709 13:01:29.789108 20157 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:29.784440 24296 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:29.796409 24236 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.820s	user 0.011s	sys 0.001s
I20260709 13:01:29.771660 24299 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.767593 24163 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:01:29.795097 24286 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:01:29.816025 20165 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 9.
I20260709 13:01:29.817840 24241 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:29.815996 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:29.840261 24287 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:01:29.820221 24236 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.788268 24010 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.836838 24299 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:29.838701 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:29.802943 24207 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:29.797480 23731 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:29.892215 24295 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:29.899309 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:29.935873 20854 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:29.936264 20158 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:29.918329 23711 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:29.819012 24303 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:29.926039 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 110181us
I20260709 13:01:29.927441 20157 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:29.871771 24293 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.849043 24163 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:29.887447 24302 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.930912 24290 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:29.939715 20857 raft_consensus.cc:2368] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10.
I20260709 13:01:29.926656 24294 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.900784 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:29.934070 24130 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.945235 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:29.929981 24301 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:29.931682 24304 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:29.934110 24305 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:29.919248 24280 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:29.942137 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 301246us
W20260709 13:01:29.963636 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:29.928510 24284 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:01:29.959594 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:29.952746 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 52089.3us
I20260709 13:01:29.944366 24207 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:29.976046 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:29.977516 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:29.980182 20082 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:29.971767 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:29.981712 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:29.979982 20010 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:01:29.983011 20010 pending_rounds.cc:85] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 5
I20260709 13:01:29.983424 20010 pending_rounds.cc:107] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:29.983686 20010 raft_consensus.cc:2889] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:29.961225 24237 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:29.962049 24188 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.936972 24299 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:29.988447 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:29.961685 24241 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:29.962405 24152 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.991392 20156 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:29.946923 23711 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:29.990444 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:29.970571 24262 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.952569 24293 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.012315 24262 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.013950 24293 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.961848 24130 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.015097 24262 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.017112 24130 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.018806 24299 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.021523 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.000866 24188 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.025955 24188 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:29.952680 24163 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.028625 24163 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:29.957008 24294 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.031003 24258 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.031416 24294 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:29.965853 24303 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:29.999652 20085 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:01:30.008279 24148 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.008049 24152 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:29.982858 24257 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.009003 24241 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:29.952921 24302 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.009375 23731 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.009208 24201 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:29.994769 24202 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:29.983027 24207 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.011612 23711 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:29.966621 24301 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.033253 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.096891 20858 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:30.033466 24280 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.034062 24010 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.034801 24236 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:30.038229 24231 consensus_meta.cc:287] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.001s
I20260709 13:01:30.040681 24199 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.942644 24295 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.134295 24295 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:29.969663 24304 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.136435 24304 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:29.972225 24300 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:29.971171 24230 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.137569 24230 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.137643 24156 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.138657 24156 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:30.139463 20853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 1252) took 1365 ms (client timeout 1547 ms). Trace:
W20260709 13:01:30.140012 20853 rpcz_store.cc:269] 0709 13:01:28.774287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.774904 (+   617us) service_pool.cc:224] Handling call
0709 13:01:30.139218 (+1364314us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54006eaad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:30.139360 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2082064768}
W20260709 13:01:30.056378 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 74265.5us
I20260709 13:01:30.145520 20129 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:30.148348 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:30.059751 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 88082.4us
I20260709 13:01:30.072489 20854 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:29.969419 24305 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.015775 24262 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:30.153923 24294 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.162721 24236 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.163326 24236 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.165344 24152 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.165438 24236 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.166152 24236 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.166639 24236 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.167062 24207 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.107456 24279 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:30.139664 24230 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.168064 24228 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:30.168217 24279 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.168622 24228 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.168754 24279 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.169472 24301 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.170861 24228 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.078282 23833 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.083009 24241 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.171641 23833 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.173638 24241 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.174007 23833 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.110018 24257 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.075410 24311 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:01:30.108970 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.174819 24257 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:30.100898 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 131983us
I20260709 13:01:30.128299 24053 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:30.129110 23958 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.130535 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.131726 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.132123 24251 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:30.038257 20158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 943) took 1001 ms. Trace:
I20260709 13:01:30.147840 23042 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.152865 20056 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.088939 24296 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:30.156390 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.157199 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.158370 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.159718 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.159791 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.160569 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.160569 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.171065 24279 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.077997 24303 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:30.175156 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 66303us
I20260709 13:01:30.124673 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 1260) took 1208 ms. Trace:
W20260709 13:01:30.177269 20017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 1152) took 1306 ms (client timeout 1547 ms). Trace:
I20260709 13:01:30.100875 24237 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:30.234879 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:30.244989 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 92195.1us
I20260709 13:01:30.255105 20160 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:30.255784 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.256168 20852 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:30.186728 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.257401 20164 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:30.258674 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:30.125653 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 154667us
I20260709 13:01:30.184018 23042 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.183681 20158 rpcz_store.cc:276] 0709 13:01:29.036568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.036677 (+   109us) service_pool.cc:224] Handling call
0709 13:01:29.036981 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:29.938857 (+901876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.037974 (+ 99117us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54000bf754. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:30.038193 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067645312}
W20260709 13:01:30.259853 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.260658 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56335.9us
I20260709 13:01:30.261826 20851 rpcz_store.cc:276] 0709 13:01:28.916552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.918639 (+  2087us) service_pool.cc:224] Handling call
0709 13:01:28.918962 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:29.159503 (+240541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.124425 (+964922us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54002413d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:30.124604 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504769792}
W20260709 13:01:30.264093 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 86970.1us
I20260709 13:01:30.144423 24010 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:30.254209 20017 rpcz_store.cc:269] 0709 13:01:28.871287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.871403 (+   116us) service_pool.cc:224] Handling call
0709 13:01:30.176987 (+1305584us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54000fafd4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:30.177170 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2017744128}
I20260709 13:01:30.261475 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.265617 19981 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.269627 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.177114 20743 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.270574 20164 raft_consensus.cc:2393] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 9: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:30.270527 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.160697 24310 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.271507 20158 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
W20260709 13:01:30.273209 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 91797.9us
I20260709 13:01:30.269881 20010 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:30.274421 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.167062 24302 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.233156 24296 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.303337 24228 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:30.284389 24310 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.267196 24294 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:01:30.305511 24284 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.306129 24310 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.306041 24284 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.306761 24310 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:30.306993 23731 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.307297 24310 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:30.240657 24318 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:01:30.270187 24301 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:01:30.307629 23731 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.284588 24300 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:01:30.308735 24284 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.309031 24300 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:30.309535 24300 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:30.309824 23731 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.264570 24016 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:30.310789 24296 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.181655 24178 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:01:30.311442 24310 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.311952 19981 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:30.314008 24152 raft_consensus.cc:2804] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:01:30.298969 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 121912us
W20260709 13:01:30.314989 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.315330 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.315246 20014 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:30.318043 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.318666 20017 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:30.274888 20016 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:30.276297 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.319557 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.270723 20015 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:30.297503 20160 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:30.245538 23917 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:30.288595 23042 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.301128 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.294338 24141 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:01:30.275929 24305 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.281037 24237 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.335142 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:30.336540 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.337152 20012 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:30.337278 24141 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:30.337739 24141 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:30.338527 24141 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.328904 24241 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.304136 24262 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.304278 24228 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.340715 20854 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:01:30.340970 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:30.341710 20008 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:30.186673 24291 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.342130 24228 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.156239 24231 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.270794 24207 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:01:30.310379 24300 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.310684 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:30.287180 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.273581 24230 consensus_queue.cc:1059] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:01:30.342458 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.350298 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.183432 24251 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:30.322537 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:30.373641 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.323413 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.374642 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.375283 20089 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:30.368319 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.376325 20084 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:30.321861 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:30.180001 24153 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:30.267168 24130 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:30.326200 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.322734 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.384387 20010 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:30.384532 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.331689 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:30.332226 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.332585 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.175694 24311 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:30.343385 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.371882 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.417694 20009 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:30.282130 24257 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.329807 19982 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:30.315183 23833 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:30.351049 20017 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:30.373989 20013 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:30.322667 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:30.440189 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 110459us
I20260709 13:01:30.441296 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:30.440372 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 182757us
I20260709 13:01:30.368278 20090 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:30.377197 20016 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.339690 24141 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.374666 20859 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:30.378422 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:30.378715 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.379261 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.379983 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.381855 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:30.448686 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.449378 20082 raft_consensus.cc:2410] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:30.383631 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.449864 20086 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:30.449901 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.419585 20164 raft_consensus.cc:2393] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:30.382443 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:30.347817 24308 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:30.417098 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.450852 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.434006 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.321041 24152 raft_consensus.cc:697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:30.435065 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.453619 20856 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:30.342439 24291 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.455336 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:30.456528 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.457638 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.457911 24231 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.458680 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.434233 24228 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:30.459245 20012 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:30.459579 24305 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.442108 24143 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:30.434149 24311 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.460495 24311 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.460383 24143 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.444440 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:30.451340 20085 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:30.445186 20161 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:30.451586 20853 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:30.445782 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:30.440963 23731 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:30.450875 20087 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.452571 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.453454 20855 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:30.432946 20850 raft_consensus.cc:3060] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.460188 24305 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.368806 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.461527 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.293480 24010 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.461565 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.462548 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.462733 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.464306 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:30.471225 20016 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:30.465159 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.472427 20012 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:30.478600 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:30.480260 20086 raft_consensus.cc:2393] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:30.485087 24230 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:30.485605 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.486449 24237 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.487547 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:30.489658 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 228240us
I20260709 13:01:30.490170 19981 leader_election.cc:400] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10.
I20260709 13:01:30.443413 23580 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.495222 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.496034 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.496372 20085 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:30.498251 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.499349 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.499431 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:30.491426 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.506043 20159 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:30.502842 20013 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:30.434434 24153 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.462452 23833 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:30.535493 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.482206 24280 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:30.482849 24207 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:30.477587 23731 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.548746 20850 raft_consensus.cc:2468] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:30.478482 24150 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:30.508829 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.465976 24129 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:30.546706 24280 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:30.495230 24327 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.509024 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.492883 24228 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:01:30.553292 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 73638.1us
I20260709 13:01:30.555231 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.531602 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.530628 24237 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.532334 24262 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:30.559190 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 66885.7us
I20260709 13:01:30.560940 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.548542 24236 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.586712 23042 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:30.508383 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.556943 24336 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:30.589035 20130 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:30.589847 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 81524.7us
I20260709 13:01:30.551618 23731 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.501000 23580 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.590518 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:30.580147 24258 consensus_meta.cc:287] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.006s
I20260709 13:01:30.461709 24308 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.601832 24308 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.558745 24150 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:30.604738 24143 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.604841 24228 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.547065 24207 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.606302 24207 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.568400 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:30.608501 24325 log.cc:927] Time spent T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:01:30.608964 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.609774 24230 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:30.610227 24230 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.610582 24230 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:30.611219 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.557235 24152 consensus_queue.cc:237] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.612643 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.615571 24303 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.617128 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:30.618379 24305 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.619618 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.619885 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:30.620383 20083 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:30.621315 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.621521 24143 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:30.622092 24143 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:30.622642 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.623478 20083 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:30.592861 20851 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:30.595995 23833 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.621340 24279 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:30.625571 24279 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:30.626058 24279 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:30.587304 24237 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.597150 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.628808 24237 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.597764 24324 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:30.629688 24324 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.630069 24324 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:30.632498 24331 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:01:30.633059 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:30.585119 24262 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.633826 20011 raft_consensus.cc:2393] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:30.634117 24262 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.634706 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.596521 24291 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:30.599414 20008 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:30.599578 24336 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.604555 24150 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.571133 24129 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.622928 24143 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.625385 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.691208 20082 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:30.691329 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.618408 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.692412 20088 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:30.577240 24280 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:30.598964 23580 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.627660 24283 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:30.668022 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 79330.5us
I20260709 13:01:30.667452 20013 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:30.660158 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.670280 20857 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:30.677575 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.679435 20087 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:30.682370 20157 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:30.700639 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.579412 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.688138 23731 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.586993 24010 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.481964 24251 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.717972 20159 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:30.719092 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.726104 20852 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:30.718562 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 139242us
W20260709 13:01:30.720481 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 253326us
I20260709 13:01:30.722246 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.703510 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 793) took 1729 ms. Trace:
I20260709 13:01:30.727022 20129 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.724797 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.729729 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:30.694638 24337 log.cc:927] Time spent T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.198s	user 0.003s	sys 0.006s
I20260709 13:01:30.731822 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.732221 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:30.732709 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 140123us
I20260709 13:01:30.734953 23042 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:30.734627 20851 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:30.730631 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:30.736091 20156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50786 (request call id 1378) took 1096 ms. Trace:
I20260709 13:01:30.737850 19981 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.737170 20156 rpcz_store.cc:276] 0709 13:01:29.643369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:29.713074 (+ 69705us) service_pool.cc:224] Handling call
0709 13:01:30.735892 (+1022818us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540049be54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:30.736032 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461547904}
I20260709 13:01:30.741220 20858 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:30.727293 20091 rpcz_store.cc:276] 0709 13:01:28.974327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:28.982053 (+  7726us) service_pool.cc:224] Handling call
0709 13:01:29.001285 (+ 19232us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:29.029324 (+ 28039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:29.029361 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:29.029993 (+   632us) log.cc:844] Serialized 52 byte log entry
0709 13:01:29.388915 (+358922us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:29.441190 (+ 52275us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.494790 (+ 53600us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b18003d75c0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:29.787637 (+292847us) log.cc:844] Serialized 14 byte log entry
0709 13:01:29.787870 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.703050 (+915180us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b54004fea54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:30.703131 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.703202 (+    71us) raft_consensus.cc:1685] finished
0709 13:01:30.703223 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.703427 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100,"prepare.queue_time_us":64830,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":15248,"raft.queue_time_us":99542,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":7879,"spinlock_wait_cycles":70438656,"thread_start_us":556019,"threads_started":2,"wal-append.queue_time_us":424315}
I20260709 13:01:30.743081 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.583099 24327 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.746202 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:30.735858 24335 log.cc:927] Time spent T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:01:30.726332 24305 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.728825 24303 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:30.731838 24150 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.681409 24130 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:01:30.752848 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.721201 24339 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.758558 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.767573 20129 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.759864 24129 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.766286 20850 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:30.759886 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.774901 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.778154 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:30.745147 24269 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:30.781104 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.781344 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 43571.7us
W20260709 13:01:30.790311 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.796407 20082 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:30.756453 24244 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.814431 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.797739 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.768658 24152 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:30.801744 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.803723 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:30.804315 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.789805 24129 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:30.806562 20091 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:30.739893 23934 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:01:30.806655 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:30.808198 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:30.765805 24327 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:30.776697 24148 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:30.772571 24303 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:30.812168 24153 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.813159 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:30.813447 24279 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:30.777063 24339 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.817430 19982 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:30.823871 20088 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:30.749210 24207 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.797739 24305 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:30.798583 23917 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:30.842937 20853 raft_consensus.cc:1275] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:30.852115 24291 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:30.819026 24152 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.866412 20160 raft_consensus.cc:1240] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:30.821617 24129 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.850234 20855 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
W20260709 13:01:30.851187 24156 consensus_meta.cc:287] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.712s	user 0.014s	sys 0.000s
I20260709 13:01:30.871492 24129 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.846912 24153 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.816898 24311 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:30.877874 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 55868.9us
I20260709 13:01:30.882248 19981 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.872725 24152 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.824693 24280 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:30.917829 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 241976us
I20260709 13:01:30.881031 24237 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:30.920835 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:30.920436 20090 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:30.923556 20159 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:30.921672 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.925484 20744 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.914980 24259 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:30.940675 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.943189 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.911782 24156 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:01:30.910751 24153 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:29.646196 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:30.961356 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 79170.2us
I20260709 13:01:30.957677 20744 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.971381 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:30.969162 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 201659us
I20260709 13:01:30.972421 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:30.976006 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:30.937147 24259 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:30.973415 24236 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:30.984524 24236 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:30.979423 24336 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:30.985010 24236 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:30.939237 24237 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:30.984027 24259 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:30.938302 23833 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:30.985270 24350 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:30.944949 24228 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:30.949416 24323 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:30.917387 24262 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.988446 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:30.983754 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:30.989403 23958 consensus_meta.cc:287] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.003s
I20260709 13:01:30.949751 24327 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:30.993896 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:30.991930 23934 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.973264 23731 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:30.985781 24222 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:30.945339 24230 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:30.992534 24053 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:30.988531 24336 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:31.013213 20156 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:01:30.994855 24244 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.996642 24311 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:31.072145 20009 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
W20260709 13:01:31.060936 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 67145.5us
I20260709 13:01:31.107517 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:31.095419 24053 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.053310 24327 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
W20260709 13:01:31.100286 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 80984us
I20260709 13:01:31.056921 24202 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.113336 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.066251 24343 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:31.065999 24349 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:31.106318 23958 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:31.143863 24153 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.145004 24153 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:31.145682 24153 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:31.146674 24153 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.115314 24356 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:31.147464 20014 raft_consensus.cc:1275] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:31.148733 24355 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:31.150805 24280 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.151939 24279 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:31.136581 24207 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:31.114660 24053 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:31.170682 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.115967 24327 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.103534 24324 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.105981 24336 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:31.110112 24244 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.147730 24356 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:31.117971 24202 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.184818 24290 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.183770 24343 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:31.211009 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 96696.4us
I20260709 13:01:31.217555 20015 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:31.208629 24356 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:31.234297 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 101123us
I20260709 13:01:31.214424 24343 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:31.213276 24290 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.242866 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.239131 23934 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.247537 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.248238 20012 raft_consensus.cc:2376] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10. Re-sending same reply.
I20260709 13:01:31.249166 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.249588 24244 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.250308 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.250664 20012 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:31.250938 20014 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
W20260709 13:01:31.252398 24357 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:01:31.252475 24230 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:31.254670 24362 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:01:31.256114 24365 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.256883 24365 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.260663 24324 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.251750 20012 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:31.264106 20012 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:31.265398 20012 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:31.267839 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.270464 20156 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:31.276285 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.259720 24365 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.282459 24237 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:31.288964 24291 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:31.289534 24291 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.290567 24129 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.291975 24236 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.274524 24294 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:31.296021 24294 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:31.297326 24236 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.298219 24294 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:31.298960 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:31.302168 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:31.299228 24336 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:31.303022 24302 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.280242 24130 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:31.279194 24207 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:31.303642 24304 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:31.299505 24356 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:31.309582 24150 consensus_meta.cc:287] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:01:31.310530 24356 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:31.310969 24199 raft_consensus.cc:3055] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:31.311422 24199 raft_consensus.cc:740] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:31.312273 24199 consensus_queue.cc:260] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.313563 24199 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:31.356940 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 54858.1us
W20260709 13:01:31.336390 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 54285.9us
I20260709 13:01:31.324455 20055 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.363574 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:31.364279 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.365108 20013 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:31.364343 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.366992 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.367515 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.368031 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.368515 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.369122 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.369446 20085 raft_consensus.cc:2393] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 15: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:31.369778 20089 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:31.371295 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.371474 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.372092 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.372790 20086 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
W20260709 13:01:31.373376 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 49074.4us
I20260709 13:01:31.372128 20089 raft_consensus.cc:2410] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:31.374635 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:31.375139 20017 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:31.376199 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:31.371778 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.380669 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.382052 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:31.382753 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.385495 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.389751 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:31.391390 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.320901 24339 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.321048 24290 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.394140 20164 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:01:31.394363 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.378706 24310 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:31.352586 24129 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:31.396719 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.409081 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:31.413250 20852 raft_consensus.cc:3060] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.414455 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:31.415359 20162 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:31.350502 24262 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.385272 24228 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:31.346798 24302 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:31.378162 24294 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:31.409621 20164 pending_rounds.cc:85] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 6
I20260709 13:01:31.353066 24152 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.321295 24343 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:31.410267 20851 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.421746 20164 pending_rounds.cc:107] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:01:31.429409 20164 raft_consensus.cc:2889] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:31.390472 24222 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:31.350708 24053 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.337054 24207 consensus_queue.cc:1059] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:01:31.375844 24300 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.438905 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:31.439342 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53061.6us
I20260709 13:01:31.442116 20852 raft_consensus.cc:2468] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:31.443297 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.436939 24222 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.420135 24259 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.452725 20055 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:31.468122 20011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 768) took 1127 ms. Trace:
I20260709 13:01:31.468605 20011 rpcz_store.cc:276] 0709 13:01:30.340638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.635607 (+294969us) service_pool.cc:224] Handling call
0709 13:01:30.667164 (+ 31557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.667663 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:30.667692 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:30.691024 (+ 23332us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.153974 (+462950us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b180023a9a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e3520d1 00007fa02e76da0f
0709 13:01:31.235697 (+ 81723us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:31.236017 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.339077 (+103060us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b180023a9a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5557 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:31.467500 (+128423us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b180023a9a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:01:31.467727 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.467825 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.467875 (+    50us) raft_consensus.cc:1685] finished
0709 13:01:31.467891 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.468058 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5194,"prepare.queue_time_us":173,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":134,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":16589,"spinlock_wait_cycles":23577600,"thread_start_us":81513,"threads_started":1,"wal-append.queue_time_us":620962}
I20260709 13:01:31.471611 24191 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:01:31.472661 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 214240us
W20260709 13:01:31.474793 24368 log.cc:927] Time spent T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:01:31.475109 19982 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:31.459021 24222 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:31.472327 24327 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:31.424657 24228 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.477893 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:31.479624 20164 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
W20260709 13:01:31.480803 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 71885.5us
I20260709 13:01:31.481877 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:31.483525 20160 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:01:31.485599 20090 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:31.490600 20163 raft_consensus.cc:1240] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:31.478394 24237 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:31.497581 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 254751us
I20260709 13:01:31.493986 24376 raft_consensus.cc:2804] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:31.500586 24053 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.503422 24259 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:31.503871 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1177) took 1006 ms. Trace:
I20260709 13:01:31.504057 24377 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:31.504637 24377 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:31.504303 20010 rpcz_store.cc:276] 0709 13:01:30.497951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.848322 (+350371us) service_pool.cc:224] Handling call
0709 13:01:30.848757 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.263860 (+415103us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b1800139900. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7aabfb 00007fa02e7b564b
0709 13:01:31.264035 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:31.264060 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.264563 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.441606 (+177043us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:31.441642 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.441706 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.503578 (+ 61872us) raft_consensus.cc:1685] finished
0709 13:01:31.503600 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.503810 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105596,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7874,"spinlock_wait_cycles":20580736,"thread_start_us":287445,"threads_started":2,"wal-append.queue_time_us":212900}
I20260709 13:01:31.488375 24294 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:31.485831 24336 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.506680 24294 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:31.515301 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72192.9us
I20260709 13:01:31.517061 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:31.519063 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:31.519587 20159 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.521087 20857 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:31.489104 24291 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.529867 20165 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:31.511860 23934 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:31.504361 24327 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:31.526206 24152 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:31.526842 24228 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.528069 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:31.534032 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.535259 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.529047 24237 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:31.517143 24376 raft_consensus.cc:697] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:31.536618 24237 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.536774 24376 consensus_queue.cc:237] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.536664 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.537987 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.538592 20089 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:31.529587 23731 consensus_queue.cc:1048] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:31.539391 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.521250 24354 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:31.540171 20852 raft_consensus.cc:2393] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:31.540428 24354 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.540798 24354 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.541096 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.541764 23731 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:31.542184 23731 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:31.542285 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.542582 23731 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:31.536019 20090 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:31.513222 24294 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.526005 24377 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:31.544667 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.545095 23934 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:31.530946 24291 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.545812 24262 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:31.487498 24323 consensus_queue.cc:1059] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:01:31.548432 24323 raft_consensus.cc:3055] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:31.548787 24323 raft_consensus.cc:740] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:31.549613 24323 consensus_queue.cc:260] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:31.550702 24323 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:31.551203 20851 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:31.551440 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.552196 20090 raft_consensus.cc:2393] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:31.516264 24336 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.552800 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.553531 20857 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:31.554155 24336 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:31.557364 24222 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:31.561266 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:31.564256 20163 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.569166 20744 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:31.574046 20163 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:31.571470 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:31.580755 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.581676 20090 raft_consensus.cc:3060] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:31.585901 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.587601 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.576906 19982 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:31.590780 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.592315 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.593618 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.594274 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.616232 20086 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.588800 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:31.602741 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.639372 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:31.644977 20017 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:31.644917 20159 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:31.647152 20090 raft_consensus.cc:2468] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:31.644116 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.649398 20085 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.653021 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.647421 19982 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:31.580267 24304 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:31.593535 24053 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.642980 23833 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:31.643239 24262 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:31.648775 24152 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:31.555393 23731 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:31.660490 24362 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:31.579967 24290 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:01:31.705828 20085 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:31.652686 24231 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.659128 24175 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:31.660805 24304 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:31.643385 24343 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:01:31.692448 24262 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:31.658303 23934 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:31.681564 20086 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:31.715721 24231 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:31.729014 24251 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.730132 23934 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:31.731168 23934 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.736757 24202 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.724524 24262 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.739812 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.740437 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:31.741300 20015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 1171) took 1248 ms (client timeout 1659 ms). Trace:
W20260709 13:01:31.741734 20015 rpcz_store.cc:269] 0709 13:01:30.492546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.499116 (+  6570us) service_pool.cc:224] Handling call
0709 13:01:31.741149 (+1242033us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b540073e1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:31.741249 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649657728}
I20260709 13:01:31.720865 24294 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.743559 20085 raft_consensus.cc:1275] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:01:31.744622 20163 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:31.718660 24304 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:31.745700 24354 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:31.740531 20090 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:31.752566 24303 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:31.753149 24327 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:31.753690 23731 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.754277 24327 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:31.754671 24327 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:31.756147 24327 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:31.757400 24327 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:31.762131 24294 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:31.729065 24231 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:31.764962 24222 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:31.765461 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:31.681463 24053 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.766022 20163 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.767419 24053 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.768194 20851 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:31.770038 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.772143 20856 raft_consensus.cc:2410] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:01:31.695201 24336 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:01:31.773633 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44210 (request call id 1263) took 1335 ms. Trace:
I20260709 13:01:31.773958 20856 rpcz_store.cc:276] 0709 13:01:30.438024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.499230 (+ 61206us) service_pool.cc:224] Handling call
0709 13:01:31.773433 (+1274203us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400240254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:31.773561 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2923881856}
I20260709 13:01:31.776512 19839 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 8 to 11, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 11 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.669166 23833 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:31.784920 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34542 (request call id 1172) took 1292 ms. Trace:
I20260709 13:01:31.778939 19839 catalog_manager.cc:5697] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 7 to 9, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 9 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:31.790936 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.802210 20009 rpcz_store.cc:276] 0709 13:01:30.492848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.608667 (+115819us) service_pool.cc:224] Handling call
0709 13:01:31.784674 (+1176007us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54000f96d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:31.784845 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062974208}
I20260709 13:01:31.791671 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.793504 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.796248 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.806819 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.811049 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.812232 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.816834 20856 raft_consensus.cc:2368] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16.
I20260709 13:01:31.817071 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:31.817683 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 301648us
I20260709 13:01:31.817016 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.818989 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.819742 20088 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.831101 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:31.831542 20013 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:31.832257 20162 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:01:31.834501 20163 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:31.837167 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.839200 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.840093 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.840404 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.840706 20853 raft_consensus.cc:2376] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13. Re-sending same reply.
I20260709 13:01:31.841504 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.842423 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:31.842499 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:31.843329 20856 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:31.844105 20056 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.844429 20854 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:01:31.844944 20854 pending_rounds.cc:85] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 8
I20260709 13:01:31.845275 20854 pending_rounds.cc:107] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:01:31.845551 20854 raft_consensus.cc:2889] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:31.848493 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:31.849380 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:31.805184 23833 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.806495 24354 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:31.861470 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.862211 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.867491 20014 raft_consensus.cc:2410] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:31.869277 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:31.869930 20156 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:31.864040 19981 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.871094 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.858819 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:31.872747 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:31.873054 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.874329 20165 raft_consensus.cc:1240] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:31.861034 24143 raft_consensus.cc:2764] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:01:31.847265 24304 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:31.877918 24304 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:31.817080 24308 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:31.853646 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:31.888744 20088 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:31.847062 24300 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.851600 23958 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:31.901791 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.905237 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.910082 20090 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 16.
I20260709 13:01:31.862538 24129 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:31.906312 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52765.3us
I20260709 13:01:31.909170 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:31.917814 20085 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:31.862349 24259 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:31.912626 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:31.924556 24356 consensus_queue.cc:1048] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:31.928457 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:31.891402 24141 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:01:31.936081 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 412278us
I20260709 13:01:31.912125 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:31.914178 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:31.915244 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:31.916091 24324 consensus_queue.cc:1048] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:31.918190 20086 raft_consensus.cc:2393] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 12: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:31.866706 24293 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:31.866328 24202 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:31.921060 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:31.909027 24152 raft_consensus.cc:2804] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:32.018013 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 79027.5us
I20260709 13:01:31.910112 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:31.913487 24053 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:31.923184 24129 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:31.927657 24148 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.009634 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:31.937628 24141 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:31.911887 24300 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:31.954730 24356 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:31.933322 24259 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:32.093338 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 74477.2us
W20260709 13:01:32.084779 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 174779us
W20260709 13:01:32.094295 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 173319us
I20260709 13:01:32.095088 20743 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:32.104377 20852 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:32.114157 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:32.115028 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 135235us
I20260709 13:01:32.084499 24129 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.120394 20056 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.094164 24141 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:32.126157 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:32.096839 24228 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.099419 24291 raft_consensus.cc:2764] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:01:32.134796 20086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:32.134869 20055 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.102634 24302 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.102058 24236 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.136694 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.142513 20086 raft_consensus.cc:606] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.150251 20086 raft_consensus.cc:614] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:32.118165 24010 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.153605 24010 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.156134 24010 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:32.148723 24207 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:32.157205 24376 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:32.156988 24207 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.157671 24376 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:32.158850 24376 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.159739 24207 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:32.148627 24302 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:32.123590 24387 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.097364 24148 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.131723 24175 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:32.163414 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 228690us
I20260709 13:01:32.199543 20017 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:32.200834 20162 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 16.
I20260709 13:01:32.142048 24311 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.145376 24130 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:32.202541 20744 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:32.209544 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 74794.6us
I20260709 13:01:32.166312 24302 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.184413 24376 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:32.143577 24228 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:32.212050 24385 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:01:32.226804 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.204412 24396 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.163784 24236 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.215626 24394 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:01:32.271751 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.227654 24393 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:01:32.218456 24207 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:32.229399 24323 consensus_meta.cc:287] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.674s	user 0.013s	sys 0.000s
I20260709 13:01:32.273839 20852 raft_consensus.cc:2393] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:32.198287 24387 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:32.281000 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 160720us
I20260709 13:01:32.234001 24294 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:32.283005 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:32.214411 24130 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.288252 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.281996 20162 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:32.282609 24394 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:32.298023 24394 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:32.298218 20086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:32.299224 20086 raft_consensus.cc:606] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.299682 20086 raft_consensus.cc:614] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:32.285030 24403 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:32.304675 24403 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:32.290647 24400 log.cc:927] Time spent T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:01:32.288655 24207 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:32.288784 24271 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:32.316990 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.322126 24392 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:01:32.322383 20086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:32.347546 20086 raft_consensus.cc:606] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.347891 20086 raft_consensus.cc:614] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:32.325079 24399 log.cc:927] Time spent T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:01:32.325587 24407 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:01:32.326447 24398 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:01:32.349704 24271 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.327952 24387 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:32.328656 24391 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:01:32.284093 24310 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:01:32.329314 24302 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:32.333616 24327 consensus_meta.cc:287] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.000s
I20260709 13:01:32.334956 24228 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:32.341238 24148 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:32.342820 24236 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:32.344789 24130 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:32.276639 24396 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.386055 24396 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:32.349027 24242 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:32.315550 24259 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:32.391336 20855 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
W20260709 13:01:32.371491 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 275694us
I20260709 13:01:32.369895 24294 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:32.374353 24408 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.376984 20083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:32.364594 24377 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.372578 20858 raft_consensus.cc:2468] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:01:32.398089 24294 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.404659 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.405303 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.411592 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:32.412528 20129 leader_election.cc:400] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16.
I20260709 13:01:32.412845 20129 leader_election.cc:403] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:32.413781 24010 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.414175 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.416105 20857 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:32.394260 24403 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:32.418593 24010 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16.
I20260709 13:01:32.408133 24408 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.424305 24408 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:32.424674 24129 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.428014 24129 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:32.430619 24141 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.431530 20850 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:32.432981 24302 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:32.433090 24141 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:32.440029 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:32.441545 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 148513us
I20260709 13:01:32.443027 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.442054 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.448256 24010 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:32.451884 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:32.453251 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.455252 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:32.451097 24383 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:32.456506 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:32.459055 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:32.459836 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.450454 24376 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.456300 24403 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.462008 24143 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:32.456427 24408 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.449885 24388 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:01:32.465513 24388 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:32.465867 24388 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:32.466753 24388 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.467960 24388 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:32.476601 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.456664 24375 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.480863 24375 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.481323 24375 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.481009 24291 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:32.479880 20743 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.482467 24291 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:32.482610 24389 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.482851 24291 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:32.483722 24291 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.478762 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:32.485164 24291 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.485307 20091 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:32.488874 24305 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:32.490914 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:32.491271 20850 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:32.493170 20743 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.496753 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.498015 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.498648 20850 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:32.498672 20743 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.489382 24305 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:32.502390 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:32.504585 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.505306 20850 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:32.503666 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.502434 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:32.505645 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.505221 20164 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:32.509155 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.478338 24339 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:32.476894 24143 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:32.511955 24339 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.514585 24339 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:32.514829 24175 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:32.519021 23833 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.003s
I20260709 13:01:32.519505 23833 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:32.526007 24397 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.314s	user 0.004s	sys 0.001s
I20260709 13:01:32.529290 20012 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.529569 24284 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:32.529966 24284 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.530315 24284 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:32.529376 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 614205us
I20260709 13:01:32.533154 23934 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.533254 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.533751 23934 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.538959 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.504683 24010 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:32.539811 20014 raft_consensus.cc:2410] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:32.539846 24010 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:32.532051 24152 raft_consensus.cc:697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:32.540840 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.540973 24010 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:32.541615 24152 consensus_queue.cc:237] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.512101 24228 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.534058 24323 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:01:32.534981 24305 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.535526 20013 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:32.553092 20009 raft_consensus.cc:1275] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:32.535847 19981 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:32.536906 24191 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.537297 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.477840 24302 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:32.548696 20129 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.544086 24304 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:32.504606 24053 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:32.554574 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.570089 20009 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:32.558858 20012 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:01:32.570557 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:32.570957 20012 pending_rounds.cc:85] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 6
I20260709 13:01:32.571226 20085 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:32.571861 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:32.571240 20012 pending_rounds.cc:107] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:01:32.572525 20089 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:32.572955 20012 raft_consensus.cc:2889] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:32.573647 20012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 782) took 1331 ms. Trace:
I20260709 13:01:32.506117 24413 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.574142 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:32.574159 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.575170 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:32.582867 20090 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:32.573946 20012 rpcz_store.cc:276] 0709 13:01:31.242473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.251375 (+  8902us) service_pool.cc:224] Handling call
0709 13:01:31.251636 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.783525 (+531889us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b180013fbc0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7ed646 00007fa02e72758e 00007fa02e7a8f9e 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:01:32.422521 (+638996us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b58001844d0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:01:32.529128 (+106607us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b580044cfd0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e70e0aa 00007fa02e790447 00007fa02e7a8fe7 00007fa02e7aa7de 00007fa02e7b4b49 00007fa02e7b5416 00007fa02e7b1d8f 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4
0709 13:01:32.573438 (+ 44310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.573598 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528375168}
I20260709 13:01:32.576932 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.579828 20853 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:32.575105 20129 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:32.584268 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.585661 20009 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
W20260709 13:01:32.586956 24417 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:01:32.588119 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:32.592118 24413 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.513873 24376 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:01:32.594306 24413 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:32.540843 23934 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:32.598644 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.600704 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:32.559551 24262 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:32.563273 24207 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:32.606786 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:32.607738 20091 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:32.607951 20087 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:32.600534 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:32.609467 20088 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:32.612461 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
W20260709 13:01:32.615649 20055 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.572106 24148 raft_consensus.cc:2804] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:32.615624 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.569270 24361 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:32.569747 24053 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.598515 24305 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:32.655175 24284 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.567656 24191 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.657459 24284 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:32.658080 24191 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:32.650681 24053 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:01:32.668762 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 90151.6us
I20260709 13:01:32.669618 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:32.670814 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:32.670969 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 134290us
I20260709 13:01:32.673128 20743 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:32.673722 20853 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:32.676039 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 51568.4us
I20260709 13:01:32.601119 23934 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.613582 24376 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:32.606619 24141 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.681263 24323 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:32.681643 24175 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.569852 24302 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:32.617170 24383 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.618609 24207 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.683821 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:32.688305 20009 raft_consensus.cc:2368] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13.
I20260709 13:01:32.623518 24129 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.692762 24129 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.693271 24129 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.698673 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.694876 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:32.717487 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.718526 20012 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:32.718834 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.719609 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:32.721545 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.722193 20017 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:32.722347 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:32.730819 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 50540.9us
I20260709 13:01:32.732053 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.732998 20083 raft_consensus.cc:606] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.733734 20156 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:32.680429 24361 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:32.740423 20162 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:32.738509 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:32.744417 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.745270 20163 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:32.733762 24228 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:32.696220 24207 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:32.657423 24339 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:32.694860 24175 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.685611 24323 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.686789 24141 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.747534 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:32.747565 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.748153 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:32.749262 20165 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:32.683234 24311 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:01:32.768167 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:32.767614 20853 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:32.761552 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:32.772619 20016 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:32.757089 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:32.735131 24242 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:32.769676 20014 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:32.773605 20017 raft_consensus.cc:2393] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:32.747534 24294 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.775048 24242 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.775231 23580 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:32.775609 24242 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:32.775902 23580 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.769536 24413 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:32.769778 24323 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.782545 24302 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:32.772918 24304 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:01:32.785321 24300 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:32.688141 23934 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.786859 20056 leader_election.cc:400] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13.
I20260709 13:01:32.787222 20056 leader_election.cc:403] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:01:32.740371 24349 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:32.788242 24349 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.750291 24388 consensus_queue.cc:1048] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:01:32.792234 24387 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.775673 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:32.794512 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:32.794963 20851 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:32.739183 24376 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:32.795517 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 48079.8us
I20260709 13:01:32.797777 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.723474 24383 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.807392 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:32.808799 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 92822.2us
I20260709 13:01:32.809679 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:32.810675 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:32.772017 24175 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.769909 24228 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:32.814131 20164 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:01:32.815501 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:32.801388 24143 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:32.844959 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.835561 24361 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.800186 24141 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.770735 19982 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.849296 20744 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:32.852598 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 82009.1us
I20260709 13:01:32.856130 23731 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:32.808427 24375 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:32.821147 24231 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:32.854445 19982 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:32.859270 24152 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:32.901242 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:32.902555 24148 raft_consensus.cc:697] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:32.866701 24389 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.907557 24300 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:32.916709 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:32.859563 24323 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.966182 20013 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.966784 20852 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:32.908830 24228 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:32.891615 24375 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:32.784842 24242 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.904456 24388 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:32.972807 20091 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 16.
I20260709 13:01:32.979038 20013 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:32.979772 24148 consensus_queue.cc:237] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:32.980661 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 62715.9us
I20260709 13:01:32.987545 24207 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:32.998824 20850 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:33.005609 24354 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:33.007270 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:33.038616 20163 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:33.039121 20853 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:33.008476 24271 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:33.008631 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:33.009012 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:33.009387 24339 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:32.872642 24376 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:33.014042 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:33.013083 24341 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.026515 20084 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:33.025141 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:33.064855 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:33.066233 20859 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:33.068967 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:33.031450 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:33.037359 20161 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:33.037890 20159 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:33.046103 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:33.049763 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:33.081219 20165 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.055699 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:33.057193 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.015653 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.083868 20012 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:33.058503 20158 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
W20260709 13:01:33.089246 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 64195us
I20260709 13:01:33.101159 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.115202 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.116223 20012 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:33.115258 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.110642 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:33.061689 24408 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:33.130445 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.132557 20013 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:33.136272 20088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:33.137272 20088 raft_consensus.cc:606] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:33.137735 20088 raft_consensus.cc:614] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:33.088841 24311 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:33.035439 24152 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:01:33.047142 24395 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:01:33.141855 24311 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.060984 24284 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:01:33.157110 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:33.115972 24305 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:33.060985 24129 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.115332 24435 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:33.146040 24311 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.147119 24300 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:33.092163 24341 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:33.217041 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 156526us
W20260709 13:01:33.201426 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 387856us
I20260709 13:01:33.218731 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:33.221273 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:33.202258 20165 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:33.223688 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:33.155898 24383 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:33.196751 24424 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.170s	user 0.001s	sys 0.001s
I20260709 13:01:33.211468 24435 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:33.225638 24053 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.005s
I20260709 13:01:33.245931 20011 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:33.255435 24341 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:33.231256 24440 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:01:33.217415 24242 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:33.249970 24435 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.283027 24305 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:33.287703 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 255635us
W20260709 13:01:33.289884 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 238071us
I20260709 13:01:33.268378 24175 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:33.301687 24428 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:33.304098 24438 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:01:33.276043 24141 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.304554 24175 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:33.305205 24432 log.cc:927] Time spent T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:01:33.306126 24429 log.cc:927] Time spent T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:01:33.309316 24305 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:33.312453 24436 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:01:33.312328 24451 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:33.315397 24141 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:33.316726 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:33.318536 24259 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:33.319006 24259 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.319113 24451 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:33.319527 24259 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.268991 24441 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:33.326143 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:33.340969 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:33.341818 20088 raft_consensus.cc:2393] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:33.268909 24053 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:33.300747 24440 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.349813 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:33.354276 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:33.354521 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:33.372393 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 146248us
W20260709 13:01:33.372576 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 148980us
I20260709 13:01:33.374658 20851 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:33.346731 24387 consensus_meta.cc:287] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.003s
I20260709 13:01:33.383031 24228 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.385710 24387 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13.
I20260709 13:01:33.386852 24228 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.387794 24387 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:33.340003 24262 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.345885 24242 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.393362 20857 raft_consensus.cc:2368] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 16. Current term is 17.
I20260709 13:01:33.406503 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.347187 24388 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:33.407454 20013 raft_consensus.cc:3060] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.355845 24440 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:33.389729 24387 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.372570 24413 consensus_queue.cc:1059] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:01:33.438704 24010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.000s	sys 0.023s
W20260709 13:01:33.439127 24010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.000s	sys 0.023s
I20260709 13:01:33.423235 24388 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:33.444622 24388 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:33.429140 19828 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.516s	user 0.015s	sys 0.013s
I20260709 13:01:33.423035 24440 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.417586 24242 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.409960 24262 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.440874 24129 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:33.472513 20013 raft_consensus.cc:2468] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:33.456059 24388 raft_consensus.cc:3055] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:33.484472 24010 consensus_queue.cc:1059] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:33.486364 24455 raft_consensus.cc:3055] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:33.486908 24455 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:33.488003 24455 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:33.489924 20130 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:33.492522 24456 raft_consensus.cc:2804] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.493234 24456 raft_consensus.cc:697] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:33.494318 24456 consensus_queue.cc:237] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.544351 24387 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:33.553576 24242 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:33.556030 24262 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:33.557451 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:33.557737 24053 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:33.462895 24311 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:33.561197 24311 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:33.562981 24383 raft_consensus.cc:993] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:33.563477 24383 raft_consensus.cc:1081] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:33.479207 24388 raft_consensus.cc:740] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:33.564633 24388 consensus_queue.cc:260] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:33.565927 24388 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:33.580219 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:33.595566 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 485018us
I20260709 13:01:33.598635 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:33.599756 24327 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:33.600347 24327 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.600752 24327 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.603593 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.604507 20013 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.604882 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.605537 20016 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.612912 24327 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.617396 24327 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:33.618076 20011 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:33.615520 20016 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:33.621909 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.622069 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.622522 20011 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:01:33.629186 24242 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.629932 24242 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.631390 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.632827 24242 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:33.635097 20744 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:33.582026 24354 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:33.639681 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.641269 20016 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
I20260709 13:01:33.642414 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:33.636482 24271 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.645628 20013 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:33.644819 20011 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 17.
W20260709 13:01:33.646129 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 156311us
W20260709 13:01:33.653048 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 156689us
I20260709 13:01:33.636037 24242 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:01:33.660120 20744 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:33.677567 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:33.637006 24259 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.658355 24271 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:33.638280 24435 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.652701 24465 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:33.645444 24354 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.656068 24327 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:33.706146 20854 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:33.763290 24259 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:33.731562 24148 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:33.732194 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.682909 24293 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:33.728448 24462 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:01:33.706240 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:33.805292 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 124970us
W20260709 13:01:33.756469 24461 log.cc:927] Time spent T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.056s	user 0.001s	sys 0.001s
I20260709 13:01:33.688678 24305 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.709074 24228 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:33.789965 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:33.756850 24448 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:33.758091 24435 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:33.699295 24300 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:33.793802 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:33.792698 24327 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.808655 20850 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.816318 24327 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.807482 24293 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:33.818346 24293 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.816507 20854 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:33.809024 24474 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.805989 24456 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.786190 24465 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.823889 20850 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:33.824198 20851 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:33.801110 24148 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.814779 24305 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.828117 24474 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.827811 24456 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:33.836812 24354 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:33.837144 24465 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:33.837674 24439 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.564s	user 0.003s	sys 0.000s
I20260709 13:01:33.807071 24271 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:33.825380 24300 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:33.838418 24456 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:33.839190 24300 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:33.839601 20010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 791) took 1085 ms. Trace:
I20260709 13:01:33.841089 20010 rpcz_store.cc:276] 0709 13:01:32.754204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.755587 (+  1383us) service_pool.cc:224] Handling call
0709 13:01:32.756517 (+   930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.803427 (+ 46910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.803458 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:33.389526 (+586068us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:01:33.389967 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:01:33.594861 (+204894us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:33.594920 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.839289 (+244369us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b54002c9554. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:33.839347 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:33.839395 (+    48us) raft_consensus.cc:1685] finished
0709 13:01:33.839407 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.839544 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":277,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":9201,"spinlock_wait_cycles":117333120,"thread_start_us":204597,"threads_started":1,"wal-append.queue_time_us":334857}
I20260709 13:01:33.842937 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:33.843837 20014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1199) took 1041 ms. Trace:
I20260709 13:01:33.844295 20014 rpcz_store.cc:276] 0709 13:01:32.802262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.802379 (+   117us) service_pool.cc:224] Handling call
0709 13:01:32.816963 (+ 14584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.819550 (+  2587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.819587 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:32.832272 (+ 12685us) log.cc:844] Serialized 26 byte log entry
0709 13:01:33.096837 (+264565us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:33.096897 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:33.097095 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:33.837100 (+740005us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:33.840757 (+  3657us) raft_consensus.cc:1685] finished
0709 13:01:33.840783 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:33.843758 (+  2975us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2102,"raft.queue_time_us":2569,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3627,"spinlock_wait_cycles":8624512,"thread_start_us":225443,"threads_started":1,"wal-append.queue_time_us":355623}
W20260709 13:01:33.846392 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 492218us
I20260709 13:01:33.853842 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.855404 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.856040 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:33.855688 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:33.857378 20010 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 12: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:33.857858 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:33.858039 20156 raft_consensus.cc:1240] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:01:33.862468 20163 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:33.863812 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.864425 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.865788 20851 raft_consensus.cc:2393] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 16: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:33.866765 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:33.866757 20090 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.867126 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:33.869212 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:33.869650 20156 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:33.812495 24448 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:33.835811 24305 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:33.872262 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:33.873385 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:33.875800 20089 raft_consensus.cc:2393] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:33.877626 20013 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:33.835811 24474 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:33.879006 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:33.885128 20083 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:01:33.886076 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.887850 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:33.890264 20088 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:33.891093 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:33.893432 20088 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:33.894896 20088 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:33.904719 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:33.905311 24148 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:33.815198 19828 maintenance_manager.cc:643] P d84e5f78c3264c738b7966fc230d4878: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 6.764s	user 0.183s	sys 0.045s Metrics: {"bytes_written":29218,"cfile_init":1,"dirs.queue_time_us":70975,"dirs.run_cpu_time_us":348,"dirs.run_wall_time_us":5895,"drs_written":1,"lbm_read_time_us":326,"lbm_reads_lt_1ms":4,"lbm_write_time_us":167319,"lbm_writes_10-100_ms":5,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":63,"thread_start_us":252663,"threads_started":2,"wal-append.queue_time_us":390605}
I20260709 13:01:33.865446 24141 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:33.907747 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:33.935281 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:33.961230 20090 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:33.968387 20089 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:33.970486 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:33.971616 19829 maintenance_manager.cc:419] P d84e5f78c3264c738b7966fc230d4878: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8090 bytes on disk
I20260709 13:01:33.969957 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:33.903600 24327 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.965114 24141 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.965893 24388 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.977445 20088 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:33.898672 24440 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:33.987237 20088 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:33.980757 19828 maintenance_manager.cc:643] P d84e5f78c3264c738b7966fc230d4878: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.008s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":416,"lbm_reads_lt_1ms":4}
W20260709 13:01:33.982035 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 109143us
I20260709 13:01:33.930872 24354 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:33.987876 24440 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:34.005362 20084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44890 (request call id 841) took 1199 ms. Trace:
I20260709 13:01:34.097067 20084 rpcz_store.cc:276] 0709 13:01:32.807193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.807310 (+   117us) service_pool.cc:224] Handling call
0709 13:01:33.026115 (+218805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.063180 (+ 37065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.005083 (+941903us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400664ed0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b18ad 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:34.005280 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502522240}
I20260709 13:01:34.010119 24323 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.016805 23934 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.021343 20852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 1227) took 1261 ms. Trace:
I20260709 13:01:34.024034 20086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1588) took 1457 ms. Trace:
I20260709 13:01:34.100659 20086 rpcz_store.cc:276] 0709 13:01:32.566630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.566814 (+   184us) service_pool.cc:224] Handling call
0709 13:01:32.567215 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.596446 (+ 29231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.596474 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:32.601381 (+  4907us) log.cc:844] Serialized 26 byte log entry
0709 13:01:32.903219 (+301838us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:32.903282 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.023006 (+1119724us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54009b9454. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:34.023250 (+   244us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.023336 (+    86us) raft_consensus.cc:1685] finished
0709 13:01:34.023356 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.023935 (+   579us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3076,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14543,"spinlock_wait_cycles":66387840,"thread_start_us":223238,"threads_started":1,"wal-append.queue_time_us":423511}
I20260709 13:01:34.095620 23833 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.099668 20852 rpcz_store.cc:276] 0709 13:01:32.759971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.761273 (+  1302us) service_pool.cc:224] Handling call
0709 13:01:34.021054 (+1259781us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54001a0254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:34.021260 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448067328}
I20260709 13:01:34.030542 24455 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:34.056241 23580 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.064312 24336 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:34.108615 24336 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:34.109704 24336 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.071029 24207 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:34.075325 24349 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.085379 24408 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:34.114450 24408 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.117184 24408 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:34.034197 24421 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 1.255s	user 0.004s	sys 0.000s
I20260709 13:01:34.119560 20855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44216 (request call id 1296) took 1656 ms. Trace:
I20260709 13:01:34.120036 20855 rpcz_store.cc:276] 0709 13:01:32.463620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.472514 (+  8894us) service_pool.cc:224] Handling call
0709 13:01:32.492491 (+ 19977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.492845 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.492871 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:32.501130 (+  8259us) log.cc:844] Serialized 26 byte log entry
0709 13:01:32.714453 (+213323us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:32.714506 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:32.714600 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.033410 (+1318810us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b4400022b4c. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02b54ddb1 00007fa02b5ebc85 00007fa02b5ec6a2 00007fa02e35e1db 00007fa02e359aaf 00007fa02e34d86e 00007fa02e34cbb6 00007fa02e34c7d8 00007fa02e356221 00007fa02e3561d9 00007fa02e356169
0709 13:01:34.119177 (+ 85767us) raft_consensus.cc:1685] finished
0709 13:01:34.119206 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.119485 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34649,"raft.queue_time_us":590,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7156,"spinlock_wait_cycles":15924096,"thread_start_us":180561,"threads_started":1,"wal-append.queue_time_us":266782}
I20260709 13:01:34.089514 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 1230) took 1081 ms. Trace:
I20260709 13:01:34.123801 20857 rpcz_store.cc:276] 0709 13:01:33.008258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.008382 (+   124us) service_pool.cc:224] Handling call
0709 13:01:34.089196 (+1080814us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54005d1954. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:34.089422 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812484352}
I20260709 13:01:34.138145 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:34.139194 24207 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:34.140012 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:34.139750 24207 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.141464 19839 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 8 to 10, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 10 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.146063 24207 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.146961 24207 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:34.148293 19839 catalog_manager.cc:5697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 5 to 7, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 7 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:34.154999 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:34.156083 24477 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:34.156495 24477 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.156831 24477 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.174242 24327 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:34.176550 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 165664us
W20260709 13:01:34.177054 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 165757us
I20260709 13:01:33.970451 24395 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.958683 24435 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:01:34.180868 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:34.178563 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 313975us
I20260709 13:01:33.931174 24305 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:33.982738 24262 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:33.966153 24259 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.213s
I20260709 13:01:33.930578 24360 consensus_queue.cc:1059] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.250s
I20260709 13:01:34.198465 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.199517 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:34.214860 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:34.218829 20853 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:34.188154 24394 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:34.227722 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:34.228713 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:34.249775 20162 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:34.253957 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.227325 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:34.257997 24394 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.259096 24448 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.214358 24479 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:34.260381 24479 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.263345 24479 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:34.268710 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 88043.3us
I20260709 13:01:34.269052 24230 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:34.269904 24230 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.270666 19981 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:34.272287 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.213624 24222 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:34.244632 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106286us
I20260709 13:01:34.321192 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:34.179834 24388 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:34.326824 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 52659.5us
I20260709 13:01:34.277310 20853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44216
I20260709 13:01:34.329468 20853 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:34.329972 20853 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:34.306924 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 101871us
I20260709 13:01:34.307264 20856 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:34.308841 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 69249.2us
I20260709 13:01:34.337219 20744 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:34.176054 24141 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.328699 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:34.351804 20055 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:34.354490 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.194630 24305 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.327111 24349 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.369104 20853 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.373766 20089 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:34.189075 24395 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.147367 24207 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.395648 24456 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.377106 24228 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:34.408175 24456 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.332247 24222 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.359786 24448 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:01:34.400876 23833 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:34.373210 24143 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:01:34.368008 24259 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:34.374629 24408 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.402158 24230 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.379171 24291 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.406584 23731 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:34.382320 24403 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:01:34.414381 24466 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:34.416836 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34530 (request call id 1554) took 1702 ms. Trace:
I20260709 13:01:34.364286 20159 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:34.414439 24448 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:34.441642 19839 catalog_manager.cc:5697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 2 to 5, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 5 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:34.409312 24284 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.217s	user 0.015s	sys 0.020s
I20260709 13:01:34.418416 24291 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.421021 24388 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.420413 24259 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:01:34.419945 24485 log.cc:927] Time spent T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:01:34.492525 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.493485 20160 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:34.415936 24395 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.415704 24222 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.411844 24456 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.423249 20853 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.437551 24327 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:34.419788 24477 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:34.428572 20083 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.000s
I20260709 13:01:34.432675 24207 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:34.413115 24141 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.457469 24479 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.471225 20162 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
W20260709 13:01:34.517282 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 163843us
I20260709 13:01:34.495316 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:34.518118 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:34.518894 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:34.514865 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 244301us
I20260709 13:01:34.520825 19981 leader_election.cc:400] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 16. Current term is 17.
I20260709 13:01:34.520908 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:34.479003 24284 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.298s	user 0.016s	sys 0.021s
I20260709 13:01:34.480485 24448 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
W20260709 13:01:34.489589 24271 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.081s	sys 0.173s
W20260709 13:01:34.512722 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 256604us
I20260709 13:01:34.512890 20083 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 17.
I20260709 13:01:34.521152 19981 leader_election.cc:403] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:34.449927 20009 rpcz_store.cc:276] 0709 13:01:32.714864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:32.720511 (+  5647us) service_pool.cc:224] Handling call
0709 13:01:32.740363 (+ 19852us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:32.740764 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:32.740796 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:33.986581 (+1245785us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b2c0001ef28. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:01:33.986961 (+   380us) log.cc:844] Serialized 26 byte log entry
0709 13:01:34.406329 (+419368us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:34.406379 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.416317 (+  9938us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:34.416393 (+    76us) raft_consensus.cc:1685] finished
0709 13:01:34.416425 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:34.416747 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4584,"raft.queue_time_us":6241,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5228,"spinlock_wait_cycles":14320384,"thread_start_us":418930,"threads_started":1,"wal-append.queue_time_us":419252}
W20260709 13:01:34.526448 24271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.082s	sys 0.174s
I20260709 13:01:34.534435 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:34.546330 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:34.529980 24259 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.542162 24336 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.539323 24291 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:34.513509 24327 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:34.548352 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 196655us
I20260709 13:01:34.530292 24230 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 16. Current term is 17.
I20260709 13:01:34.553221 20162 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:34.524166 24262 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:34.523114 24479 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.559638 24451 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:34.549659 24435 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:34.549486 24349 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:34.565953 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:34.566679 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:34.566869 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:34.571520 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:34.572937 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:34.575428 24327 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.576443 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:34.568360 24349 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.581050 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:34.583472 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:34.585265 23833 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.582440 24175 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:34.586156 24175 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.586345 24394 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.586514 24175 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.588357 24428 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:34.588858 24428 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:34.589694 24428 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.591415 24327 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.584249 24323 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.591995 24327 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.596586 24356 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.596408 24207 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.597280 24305 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.597227 24356 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.597941 24349 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.598600 24207 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.599475 24356 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.564849 24498 raft_consensus.cc:3060] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:34.565097 24433 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:01:34.604439 24327 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:34.605984 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44241.6us
I20260709 13:01:34.608245 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:34.608747 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:34.609097 20090 raft_consensus.cc:2410] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:01:34.609488 20083 raft_consensus.cc:2368] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 16. Current term is 17.
I20260709 13:01:34.626752 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.631302 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:34.565193 24435 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.643254 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:34.644174 20015 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:34.636742 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:34.645821 20087 raft_consensus.cc:2368] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15.
I20260709 13:01:34.637581 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.637856 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.639083 20130 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:34.640864 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:34.647625 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:34.624891 24230 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.650153 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.650736 24230 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.636189 24262 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.652348 24448 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.652928 24230 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.655072 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.666842 20156 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 14: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:34.671841 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.673425 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.671363 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c36c61154404ebebcf309833a8bf5ea" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:34.687250 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.643100 24222 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.740847 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 72018us
I20260709 13:01:34.719066 20013 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:34.720731 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:34.748715 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.751706 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.669554 24291 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.664816 23580 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:34.625417 24390 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:01:34.764189 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:34.777072 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 56418.3us
I20260709 13:01:34.740056 24230 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.781643 24230 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.784605 24230 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.746968 24481 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.753562 24354 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:34.753304 24477 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:34.758155 24308 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.786172 24259 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:34.813090 24323 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:34.754015 24222 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.833428 24477 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:34.785218 23580 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.822041 24354 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:34.845187 24435 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:34.846015 24479 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.848946 24441 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:34.855497 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 74987.7us
I20260709 13:01:34.837978 24323 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:34.867192 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:34.839107 24259 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:34.885387 20157 raft_consensus.cc:1240] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:34.886333 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:34.887148 20010 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:34.882287 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:34.897801 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:34.817612 24481 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:34.832317 24308 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.900277 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.862450 24349 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.904219 20156 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:34.905203 20015 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:01:34.863642 24477 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.898123 24479 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:34.931833 20016 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:34.904074 24435 consensus_queue.cc:1059] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:01:34.926206 24141 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.875685 24441 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.926687 24479 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:34.909899 24349 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.952169 24361 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:34.952721 24361 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:34.953763 24361 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:34.958230 24308 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.961128 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:34.960767 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:34.965775 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.966123 19981 leader_election.cc:400] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 16. Current term is 17.
I20260709 13:01:34.969105 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.948520 24479 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:34.925000 24323 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.974870 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:34.971354 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:34.961678 24343 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:34.945904 24175 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.966344 24481 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:34.996690 24323 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.945456 24259 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:34.977893 24308 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.991468 24498 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:35.003881 20091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1598) took 1115 ms. Trace:
I20260709 13:01:35.009159 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:35.010699 24311 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:35.011336 24222 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:35.012821 24356 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:35.013358 24356 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.013185 24175 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:35.013984 24349 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.014745 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:35.015704 24514 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:01:35.013810 24356 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:35.017800 24477 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.011855 20091 rpcz_store.cc:276] 0709 13:01:33.888811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.934452 (+ 45641us) service_pool.cc:224] Handling call
0709 13:01:33.973934 (+ 39482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:33.974189 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:34.178581 (+204392us) log.cc:844] Serialized 14 byte log entry
0709 13:01:34.484730 (+306149us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.003214 (+518484us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b18003648a0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:01:35.003462 (+   248us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:35.003526 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:35.003585 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.003670 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.003810 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4127,"prepare.queue_time_us":19862,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":92,"raft.queue_time_us":521544,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":7219,"spinlock_wait_cycles":37432320,"thread_start_us":290522,"threads_started":1,"wal-append.queue_time_us":364668}
I20260709 13:01:35.009902 24395 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:35.010084 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 492013us
W20260709 13:01:35.010993 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 459682us
I20260709 13:01:35.026096 20009 raft_consensus.cc:1275] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:35.027969 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:35.031514 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:35.032158 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:35.033342 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:35.034368 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:35.034789 20159 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:35.010716 24515 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.009938 23580 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.042079 20159 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:35.039973 24515 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:35.042742 24354 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.043450 24441 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.000104 24343 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.061254 24515 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:35.065413 24343 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:35.056315 24336 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.003s
I20260709 13:01:35.082640 24356 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:35.084784 24293 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:35.086107 24390 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:35.089375 24390 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.100486 24390 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.101773 24516 raft_consensus.cc:3055] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:35.104903 24516 raft_consensus.cc:740] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:35.106174 24516 consensus_queue.cc:260] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:35.110617 24356 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.102407 24341 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:35.082734 24479 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:35.116983 24341 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:35.117871 24341 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.118741 24479 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:35.118924 24341 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.112819 24516 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.122721 24448 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:35.126365 24428 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.130508 24361 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:35.179267 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:35.000756 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:35.202793 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 201964us
I20260709 13:01:35.214814 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:35.215312 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:35.217020 20008 raft_consensus.cc:2393] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:35.220928 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:35.224602 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:35.226115 20013 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:35.229185 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:35.230454 20009 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:35.238234 20017 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.248641 20010 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:35.249154 20009 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
W20260709 13:01:35.261622 19840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32774 (request call id 153) took 29099 ms (client timeout 15000 ms). Trace:
W20260709 13:01:35.053500 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.276618 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 250713us
I20260709 13:01:35.283553 23731 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:35.289149 20163 raft_consensus.cc:2393] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 16: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:35.290835 20856 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.292867 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.316872 23731 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:35.323781 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:35.334486 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.338544 24408 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.338733 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:35.339957 20159 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 10.
I20260709 13:01:35.339748 24408 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.346300 24408 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:35.349215 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.352761 24516 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.353753 24516 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:35.354189 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.354589 20859 raft_consensus.cc:1275] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:35.357945 24516 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.362309 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.363168 20091 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
W20260709 13:01:35.262532 19840 rpcz_store.cc:269] 0709 13:01:06.161701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.905269 (+1743568us) service_pool.cc:224] Handling call
0709 13:01:35.261167 (+27355898us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:26.032303 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:26.033011 (+   708us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:26.033070 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:01:26.033089 (+    19us) tablet.cc:662] Decoding operations
0709 13:01:26.034295 (+  1206us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:26.034413 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:26.034441 (+    28us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:01:26.035817 (+  1376us) tablet.cc:701] Row locks acquired
0709 13:01:26.035874 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:01:26.064762 (+ 28888us) write_op.cc:270] Start()
0709 13:01:26.084877 (+ 20115us) write_op.cc:276] Timestamp: P: 1783602086049425 usec, L: 0
0709 13:01:26.084909 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:01:26.085762 (+   853us) log.cc:844] Serialized 4384 byte log entry
0709 13:01:26.604636 (+518874us) op_driver.cc:464] REPLICATION: finished
0709 13:01:26.845817 (+241181us) write_op.cc:301] APPLY: starting
0709 13:01:26.845921 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:26.852985 (+  7064us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:26.853178 (+   193us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:26.884992 (+ 31814us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:26.885378 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:01:26.885424 (+    46us) write_op.cc:312] APPLY: finished
0709 13:01:26.885797 (+   373us) log.cc:844] Serialized 82 byte log entry
0709 13:01:26.898343 (+ 12546us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:26.910379 (+ 12036us) write_op.cc:454] Released schema lock
0709 13:01:26.910465 (+    86us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:26.910976 (+   511us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:26.911012 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3080177,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":240856,"apply.run_cpu_time_us":4664,"apply.run_wall_time_us":65084,"num_ops":11,"prepare.queue_time_us":129069,"prepare.run_cpu_time_us":33864,"prepare.run_wall_time_us":173460,"raft.queue_time_us":199241,"raft.run_cpu_time_us":16783,"raft.run_wall_time_us":253065,"replication_time_us":483367,"thread_start_us":418488,"threads_started":4,"wal-append.queue_time_us":141128,"wal-append.run_cpu_time_us":66118,"wal-append.run_wall_time_us":1680926}]]}
I20260709 13:01:35.365075 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.383008 24466 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.375315 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.388672 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.342907 23731 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.396353 20091 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:35.410872 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.411906 24408 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.412361 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:35.412648 24408 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.413314 20164 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 10.
W20260709 13:01:35.413460 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.417076 24437 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:35.418120 24437 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.407656 20850 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 14: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:35.420435 24465 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.428167 24465 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.432943 24437 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.407615 24466 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.415802 24408 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.434638 20084 raft_consensus.cc:1240] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:01:35.435281 24466 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.431559 24465 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.407740 24010 consensus_queue.cc:1048] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:01:35.448899 24336 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:01:35.450451 24336 consensus_queue.cc:1243] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:35.450933 24501 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:35.452018 24501 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:35.460332 24395 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.460886 24504 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.461223 24395 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.461741 24504 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:35.463755 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.466161 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.466849 20158 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:35.467049 20084 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:35.470584 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.471058 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:35.472129 20091 raft_consensus.cc:2393] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:35.459772 24506 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.472810 20088 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:35.473526 24506 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.475121 24242 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.456807 24437 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.476027 24242 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.477139 24437 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:35.477622 20090 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:35.479384 24242 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.480232 24053 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:35.481079 24053 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.481339 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.480304 24395 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.483443 24395 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:35.455072 24336 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:35.483556 24053 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.485680 20084 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:35.487247 20850 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:35.489488 20852 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:35.469799 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 48649.1us
I20260709 13:01:35.492391 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.492791 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.490548 24336 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:35.493767 20082 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:35.501189 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:35.493793 24242 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:35.503252 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.503598 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.503330 24242 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.505299 24501 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:35.506395 24506 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:35.508745 24053 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.509421 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:35.510725 20082 raft_consensus.cc:3055] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:35.510208 24336 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.511055 20082 raft_consensus.cc:740] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:35.501026 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.484364 24437 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:35.507731 24531 log.cc:927] Time spent T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:01:35.494065 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:35.518716 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.528401 20162 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:35.544557 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.539283 20852 raft_consensus.cc:1240] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:35.556558 20163 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:35.545342 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.536324 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.533587 20084 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:01:35.557789 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 53961us
I20260709 13:01:35.518004 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.549471 20857 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:35.546705 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.523011 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.536547 19981 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:35.527151 23731 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.574766 20854 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:35.594785 20851 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:35.540321 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.512604 20082 consensus_queue.cc:260] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.557662 24053 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:01:35.611434 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:35.611765 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.612773 20084 raft_consensus.cc:2393] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:35.613041 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.613646 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.545238 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.524632 24452 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:35.615942 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.616707 20084 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:35.616485 24452 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.617694 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.618271 20084 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:01:35.619164 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.522325 24506 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.611313 20082 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:35.511868 24336 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:35.578253 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.522149 24383 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.619043 24452 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.521800 24501 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:35.636471 24383 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.637428 24501 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.539381 24504 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.536356 24242 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:35.637943 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.636610 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.634756 20084 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:35.535430 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.524317 24294 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:35.654582 20085 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 9.
I20260709 13:01:35.647500 20087 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:35.666374 24501 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:35.668886 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 200321us
I20260709 13:01:35.670627 24259 consensus_queue.cc:1048] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:35.644858 23958 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.673448 24053 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.672720 24506 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.674098 24053 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.678110 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.647289 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:35.681044 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 60548.1us
I20260709 13:01:35.681229 24259 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:35.682237 24259 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.682729 24259 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.656191 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:35.646893 24383 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.655462 24395 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:35.730701 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.705865 24437 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.740057 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:35.745316 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:35.753851 23731 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:35.780993 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.785115 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.785115 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:35.748991 24053 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:35.776508 24395 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:35.731809 24294 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.790107 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.745976 24437 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:35.794104 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:35.799877 20017 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:01:35.800583 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:35.803303 20008 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:35.803812 20013 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:35.797346 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.809890 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.795786 24506 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:35.816021 20131 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.816500 24262 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:35.817696 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 100785us
I20260709 13:01:35.828130 20082 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:35.840657 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:35.844029 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:35.872449 24437 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:35.876999 20091 raft_consensus.cc:1275] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:35.886314 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:35.888588 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:35.894335 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45552.9us
I20260709 13:01:35.902211 20085 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:35.912037 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:35.916687 19981 leader_election.cc:400] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15.
I20260709 13:01:35.917330 19981 leader_election.cc:403] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:35.900354 24501 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:35.935452 24262 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15.
I20260709 13:01:35.946579 20010 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:35.948427 24431 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:35.976712 20054 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.981880 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.998833 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58343.2us
I20260709 13:01:36.001677 20085 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:01:36.030503 24437 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:36.034875 24259 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.050676 24294 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:36.041594 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 180018us
W20260709 13:01:36.036792 24543 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
I20260709 13:01:36.040869 20056 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.056831 20017 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:01:36.070449 24294 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:36.068575 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 466717us
I20260709 13:01:36.095696 20163 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:36.098443 20159 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:36.110320 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 84137us
I20260709 13:01:36.110231 23731 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:01:36.149289 23731 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:36.162856 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.165707 23731 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.164860 24053 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:01:36.167382 23958 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:36.171868 20089 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:36.173295 20089 pending_rounds.cc:85] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 6
I20260709 13:01:36.174156 23958 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.178355 20056 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.178567 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.183416 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.173557 20089 pending_rounds.cc:107] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:36.185979 20089 raft_consensus.cc:2889] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:36.187058 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.187703 20163 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:36.188158 20159 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:36.198761 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.200268 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:36.208415 24294 raft_consensus.cc:2764] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:01:36.186719 24360 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:36.215044 23731 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.184577 24259 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:36.189749 24506 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:36.199942 24191 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:36.220933 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.186869 23958 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.223980 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:36.224344 20083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1613) took 1575 ms. Trace:
I20260709 13:01:36.224748 20083 rpcz_store.cc:276] 0709 13:01:34.649183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.741033 (+ 91850us) service_pool.cc:224] Handling call
0709 13:01:34.773250 (+ 32217us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:35.024722 (+251472us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:35.307185 (+282463us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b180049afe0. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6a7e07 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c2455 00007fa0321aa3c7 00007fa02e7bc9a7 00007fa02e7cc0bf 00007fa02e7cc038
0709 13:01:35.454332 (+147147us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.755863 (+301531us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:35.838088 (+ 82225us) log.cc:844] Serialized 52 byte log entry
0709 13:01:35.838458 (+   370us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:35.839717 (+  1259us) log.cc:844] Serialized 14 byte log entry
0709 13:01:35.839998 (+   281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.845228 (+  5230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.223936 (+378708us) raft_consensus.cc:1685] finished
0709 13:01:36.223968 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.224161 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":262284,"prepare.queue_time_us":282766,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":2000,"raft.queue_time_us":287534,"raft.run_cpu_time_us":470,"raft.run_wall_time_us":18942,"spinlock_wait_cycles":4178816,"thread_start_us":305083,"threads_started":2,"wal-append.queue_time_us":513166}
W20260709 13:01:36.227520 20743 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.67:39831 timed out after 2.001s (SENT) [suppressed 5 similar messages]
I20260709 13:01:36.224750 20160 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:36.217461 24360 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:36.228693 24262 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:36.239398 20013 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:36.233325 24516 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:36.245163 20745 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.043s	sys 0.060s
W20260709 13:01:36.245609 20745 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.044s	sys 0.060s
I20260709 13:01:36.237269 23958 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:36.240769 24262 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:36.251816 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:36.244508 24516 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:36.251680 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:36.249183 23958 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:36.256559 24516 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:36.252576 24262 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:36.267580 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 228149us
W20260709 13:01:36.296624 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 45017.1us
I20260709 13:01:36.259562 23958 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:36.340063 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.267264 24544 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:36.339418 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.327354 24501 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:36.354148 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.354130 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.358052 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.326757 24506 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:36.327204 24336 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:01:36.368093 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 222578us
I20260709 13:01:36.369338 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.355554 24548 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:36.398672 20857 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:36.368492 24336 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.375519 24506 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:36.422788 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 195903us
I20260709 13:01:36.401616 24336 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:36.375598 24544 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:36.434216 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 207052us
I20260709 13:01:36.388372 24360 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.442310 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.444480 20859 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:36.458820 20013 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:36.462138 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:36.465857 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:36.466961 20010 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:36.468282 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:36.476540 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 298078us
I20260709 13:01:36.477180 24506 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.430953 24553 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:36.480232 24553 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.480624 24553 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:36.415952 24548 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.487452 24548 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:36.488517 24360 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:36.504150 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 63392.8us
I20260709 13:01:36.505726 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.507031 24431 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:36.507159 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.533398 24550 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.533856 24550 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:36.534255 24550 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.508137 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.536196 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:36.537652 24501 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:36.538060 24501 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:36.538537 24501 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.509987 24053 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.499637 20744 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:36.544390 24053 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:36.544909 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 100252us
I20260709 13:01:36.546002 20744 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:36.512635 24259 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:36.516963 24294 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:36.558158 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 162884us
I20260709 13:01:36.492946 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:36.499619 24544 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:36.528784 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.562183 20857 raft_consensus.cc:2410] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:01:36.524506 24336 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:36.557046 20160 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:36.546701 24053 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:36.564949 20163 raft_consensus.cc:1240] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:36.566391 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.573155 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.583604 20159 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:36.581337 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.588716 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:36.587535 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.597005 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:36.597738 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.602869 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 95790.5us
I20260709 13:01:36.603147 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.581712 24558 raft_consensus.cc:2764] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:01:36.561525 24259 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:36.617981 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.616858 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:01:36.609843 24552 log.cc:927] Time spent T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.174s	user 0.003s	sys 0.000s
W20260709 13:01:36.623016 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 130421us
I20260709 13:01:36.604095 24437 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:36.609676 24544 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:36.591601 24548 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:01:36.594573 24294 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.602322 24501 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:36.637326 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.638312 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.635285 24566 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.644006 24563 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:36.644486 24566 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.647114 24566 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:36.648160 24395 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:36.648823 24395 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.649504 20743 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:36.650454 20743 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:36.650902 24395 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:36.653119 24336 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.654050 24548 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:36.655211 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:36.644790 24563 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:36.655838 20851 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:01:36.653148 24259 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:36.656635 24563 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:36.656857 24550 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:36.659310 24550 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.625123 24567 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:01:36.660141 24570 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:36.666414 24501 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.667318 24437 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:36.668247 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.668399 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.669247 24566 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:36.669899 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.670614 20161 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:36.669780 24550 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:36.669507 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:36.668497 24437 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.671507 20857 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.668634 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.672669 19982 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.673151 24558 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:36.673556 24558 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:36.673923 24558 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.673983 24259 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:36.674971 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.676597 24259 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:36.655850 24548 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:36.677963 24259 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.680240 24548 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:36.678860 24566 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:36.685917 24566 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.678555 24294 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:36.681269 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:36.681833 24516 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.683192 24553 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:36.667568 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58856.2us
I20260709 13:01:36.669999 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.698148 20857 raft_consensus.cc:2468] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:36.688930 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:36.730513 24561 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:01:36.684890 24548 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:36.731443 24262 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:36.732048 23958 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:36.737713 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 106853us
I20260709 13:01:36.752058 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:36.737131 23731 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.753338 20851 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
I20260709 13:01:36.744926 20055 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:36.755424 24294 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.758052 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.762706 20055 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:36.762953 20013 raft_consensus.cc:2393] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 16: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:36.761759 23731 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:36.767107 23731 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:36.767949 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 98024.2us
W20260709 13:01:36.766849 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 78022.3us
I20260709 13:01:36.770920 20009 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:36.756172 24501 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:36.773872 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.775614 24202 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:36.760546 24550 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:36.776050 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 125673us
I20260709 13:01:36.771647 24506 raft_consensus.cc:2804] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:36.788403 24553 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:36.793110 23958 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.779867 24336 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:36.797147 24516 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.797699 24583 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:36.781312 24259 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:36.797719 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:36.798400 24583 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.801054 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:36.803690 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:36.776103 24202 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.804021 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:36.819365 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46009.4us
I20260709 13:01:36.821401 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:36.822180 20013 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:36.781546 24501 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.826056 24501 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:36.832054 24574 raft_consensus.cc:2804] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:36.832796 24574 raft_consensus.cc:697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:36.818701 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:36.821439 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50076.5us
W20260709 13:01:36.836680 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 290743us
I20260709 13:01:36.839120 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:36.852527 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.852972 24583 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:36.853895 24336 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.854038 24554 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:36.854298 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:36.854921 24550 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:36.854527 24336 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:36.855346 24550 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.855736 24550 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.856763 24395 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:36.857348 24395 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.857635 24336 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:36.859444 24395 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:36.859901 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.778784 24558 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:36.860122 20056 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.862437 24558 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:36.864619 24262 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:36.867136 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 48562.2us
I20260709 13:01:36.820647 24202 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:36.805085 24506 raft_consensus.cc:697] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:36.867841 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:36.873754 20161 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:36.820340 24563 raft_consensus.cc:2764] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:01:36.883666 19982 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.884245 20162 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:36.888298 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.885747 24336 consensus_queue.cc:1059] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:01:36.833920 24574 consensus_queue.cc:237] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:36.865700 24566 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:36.901021 20744 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:36.874653 24395 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:36.907162 20013 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:01:36.909353 24576 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:36.909940 24576 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.910288 24576 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:36.912273 24548 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:36.916409 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:36.916858 24506 consensus_queue.cc:237] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:36.917870 24544 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:36.918063 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.918591 24568 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:36.918668 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.919505 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.854643 24554 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.913090 24437 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:36.897048 24561 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:36.964362 24567 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:36.966519 24567 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:36.941409 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:36.941917 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:36.942821 20858 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:36.942950 24548 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:36.965442 24437 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.944077 24294 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:36.948634 24577 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:36.928200 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.920258 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:36.899044 24259 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:01:36.954697 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 177620us
I20260709 13:01:36.962491 20850 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:36.965235 20856 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:36.975272 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:36.965837 24584 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:36.985581 24584 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:36.986374 24437 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:36.986951 24294 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:36.988736 24563 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.989287 24563 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:36.991544 24563 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:36.977084 24577 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.989204 24567 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:37.012827 24191 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.013103 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:37.013389 24191 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:37.014055 20013 raft_consensus.cc:2410] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:37.014046 19838 catalog_manager.cc:5697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 11, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 11 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:36.989131 24294 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:37.000326 24259 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:36.968215 24568 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:37.016167 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.016866 20009 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:37.016601 24259 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.018309 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.018669 24586 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:37.018944 20089 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:37.019346 24586 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:37.015575 24191 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:01:37.021274 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.981832 24561 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:36.987737 24584 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:37.028699 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 163421us
W20260709 13:01:37.029112 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 53914.7us
I20260709 13:01:37.030558 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:36.998016 24566 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.026620 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:37.030602 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.033088 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.001332 24395 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:37.034001 20009 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:37.034332 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.033800 19838 catalog_manager.cc:5697] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 11, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 11 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:37.029502 20056 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.034940 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.018077 24568 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:37.033900 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.042901 20854 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:37.033816 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:37.041435 20086 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 17.
I20260709 13:01:37.041446 20129 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:37.044106 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:37.045254 20090 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:37.009869 23958 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.039207 20850 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:37.020368 24577 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:37.049372 20856 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:37.051384 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.052383 20013 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:37.052155 19838 catalog_manager.cc:5697] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 16, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
W20260709 13:01:37.055989 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 143053us
I20260709 13:01:37.056665 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.037129 24437 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.060412 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.055312 24553 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:37.066020 20056 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.040934 24395 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:37.049005 24584 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:37.068300 23731 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:37.073240 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.057819 24428 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:37.082398 19838 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 12, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.083941 24590 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.085407 24566 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.088203 24395 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:37.052803 24550 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.089777 24595 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:37.092130 24561 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.093255 24336 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:37.094602 20013 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:37.103416 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.066880 24294 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.104571 20087 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:37.070040 24589 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:37.057662 23958 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:37.070650 24584 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:01:37.156692 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44564.7us
I20260709 13:01:37.064324 24558 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:01:37.161969 20160 raft_consensus.cc:2368] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
I20260709 13:01:37.068497 24437 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:37.170904 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 67542us
I20260709 13:01:37.172000 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
W20260709 13:01:37.175282 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 107381us
W20260709 13:01:37.200659 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 280746us
I20260709 13:01:37.167104 24584 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.164362 24574 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:37.105553 24428 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.163821 24431 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:37.175814 24395 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.157550 24595 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:37.281643 24563 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:37.283514 24437 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:37.285382 24595 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:37.244796 24448 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:37.286728 24550 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:37.271615 24588 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:37.256549 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 84698.5us
I20260709 13:01:37.168970 24566 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:01:37.296130 20090 raft_consensus.cc:2468] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:37.294234 19981 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:37.165553 24589 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:37.309235 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.313951 24603 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:37.314533 24603 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:37.314920 24603 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.320341 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.321179 20013 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:37.322106 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:37.323884 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.324314 24550 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:37.325320 24431 consensus_queue.cc:1048] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:37.326853 24584 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.320422 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.330693 20085 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:37.338117 20744 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:37.169209 24586 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:37.362320 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.362895 19838 catalog_manager.cc:5697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 10, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 10 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.365772 19838 catalog_manager.cc:5697] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 9, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:37.376070 19982 leader_election.cc:400] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
I20260709 13:01:37.333179 24554 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:37.377146 19838 catalog_manager.cc:5697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 10, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 10 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.381662 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.386612 24360 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:37.387077 24360 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:37.386850 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.387619 24360 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.388113 23731 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:37.385080 20086 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:01:37.388489 23731 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.388784 20086 pending_rounds.cc:85] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 7
I20260709 13:01:37.388975 23731 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:37.388545 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:37.389158 20086 pending_rounds.cc:107] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:01:37.390049 24262 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:37.390328 20086 raft_consensus.cc:2889] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:37.396494 23958 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:37.396863 24605 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:01:37.397415 23958 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:37.398002 23958 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.400427 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.401153 20854 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
W20260709 13:01:37.401504 24202 consensus_meta.cc:287] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
W20260709 13:01:37.402207 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 96574.4us
I20260709 13:01:37.401971 24202 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.405216 24589 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:37.405756 24593 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:37.406424 24593 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:37.408808 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.409045 24605 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:37.409528 24605 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:37.409742 20160 raft_consensus.cc:2393] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:37.410502 24605 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.411000 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.412186 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.407523 24593 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.414867 24584 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:37.416599 24428 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:37.303342 24574 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:37.417408 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.412276 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.426591 23731 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:37.428772 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.429483 20162 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:37.431603 19838 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 12, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 12 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:37.433912 19838 catalog_manager.cc:5697] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 12, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 12 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:37.437029 24587 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:37.380121 24583 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:37.425279 19981 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.442492 24587 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:37.411900 24605 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.425038 24548 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.393545 24567 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:37.449347 23731 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:37.459380 20017 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:37.468936 20744 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:37.453361 19838 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 8, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 8 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:37.444150 24544 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:37.466601 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.483770 20856 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:01:37.459290 24570 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:37.476989 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.462462 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:37.464169 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 44655.9us
I20260709 13:01:37.447523 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:37.476727 19839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35376 (request call id 98) took 27292 ms (client timeout 15000 ms). Trace:
I20260709 13:01:37.472491 20156 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:01:37.473678 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 407741us
I20260709 13:01:37.483958 20157 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:37.491694 20160 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:37.464680 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.466342 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.531338 20056 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.530094 20158 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:37.529817 19838 catalog_manager.cc:5697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 12, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.529233 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.603104 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:37.534346 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 45652.5us
I20260709 13:01:37.609114 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.610236 20858 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:37.603797 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 112372us
W20260709 13:01:37.466707 24598 log.cc:927] Time spent T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:01:37.611861 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 149475us
I20260709 13:01:37.612805 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:37.621130 23731 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:37.621697 23731 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.622139 23731 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:37.476166 24202 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:37.489832 24262 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:37.492264 24563 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:37.534880 24191 raft_consensus.cc:2804] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:37.622565 19838 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 14, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 14 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.468452 24587 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:37.631183 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
W20260709 13:01:37.636168 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 104878us
I20260709 13:01:37.643419 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.529498 23958 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:37.646080 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.470345 24603 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.647768 24501 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.648432 24501 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:37.649458 24603 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:37.650055 24594 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.650763 24501 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:37.651757 24395 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:37.652058 20008 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.652331 24589 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.652282 24395 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:37.653101 24395 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:37.653724 24606 log.cc:927] Time spent T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:01:37.443732 24360 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:37.658787 24360 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:37.655896 24589 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:37.650578 24594 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.663412 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.660845 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:37.666067 19838 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 12, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.478470 24548 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.473773 24567 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.669598 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:37.677872 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.678496 20090 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:37.539376 24570 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:37.679584 19838 catalog_manager.cc:5697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 10, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 10 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.675024 20858 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:37.605937 24544 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.639555 24563 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:37.679601 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:37.685010 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.685344 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:37.685883 20085 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:37.473655 24577 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:37.687428 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:37.688083 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:37.642684 24561 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:01:37.682036 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.682279 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.693799 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.694413 20157 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
W20260709 13:01:37.668579 24576 consensus_meta.cc:287] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.758s	user 0.013s	sys 0.001s
I20260709 13:01:37.691488 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.688283 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.692833 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.698140 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:37.698287 23731 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:37.664273 24501 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:37.706067 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.696049 20084 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:37.706907 20010 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 18: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:37.704725 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.705235 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:37.707754 20082 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:37.706254 20858 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:37.710361 19838 catalog_manager.cc:5697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 12, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 12 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:37.704990 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:37.717448 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.718796 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.718933 24614 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:37.720988 24574 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:37.721704 24607 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.725193 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.726020 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.726496 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:37.727078 20163 raft_consensus.cc:3060] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:37.730003 20087 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:01:37.732195 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.732983 20161 raft_consensus.cc:2393] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:37.733307 24544 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:37.733944 24544 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:37.734313 24544 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.734659 24544 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:37.735329 24577 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:37.736230 20857 raft_consensus.cc:2368] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15.
W20260709 13:01:37.527275 19839 rpcz_store.cc:269] 0709 13:01:10.184420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.187464 (+  3044us) service_pool.cc:224] Handling call
0709 13:01:37.476541 (+27289077us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:35.357042 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:35.357986 (+   944us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:35.358064 (+    78us) write_op.cc:435] Acquired schema lock
0709 13:01:35.358097 (+    33us) tablet.cc:662] Decoding operations
0709 13:01:35.359257 (+  1160us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:35.359411 (+   154us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:35.359462 (+    51us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:01:35.361126 (+  1664us) tablet.cc:701] Row locks acquired
0709 13:01:35.361192 (+    66us) write_op.cc:260] PREPARE: finished
0709 13:01:35.361458 (+   266us) write_op.cc:270] Start()
0709 13:01:35.361665 (+   207us) write_op.cc:276] Timestamp: P: 1783602095361358 usec, L: 0
0709 13:01:35.370024 (+  8359us) op_driver.cc:348] REPLICATION: starting
0709 13:01:35.380950 (+ 10926us) log.cc:844] Serialized 2078 byte log entry
0709 13:01:36.197953 (+817003us) op_driver.cc:464] REPLICATION: finished
0709 13:01:36.526917 (+328964us) write_op.cc:301] APPLY: starting
0709 13:01:36.527019 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:36.642223 (+115204us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:36.642370 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:36.756099 (+113729us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:36.775596 (+ 19497us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:01:36.775646 (+    50us) write_op.cc:312] APPLY: finished
0709 13:01:36.776027 (+   381us) log.cc:844] Serialized 56 byte log entry
0709 13:01:36.776232 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:36.778823 (+  2591us) write_op.cc:454] Released schema lock
0709 13:01:36.799766 (+ 20943us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:36.826909 (+ 27143us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:36.826947 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3214665,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":328596,"apply.run_cpu_time_us":13635,"apply.run_wall_time_us":299959,"cfile_cache_hit":18,"cfile_cache_hit_bytes":18792,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6215,"cfile_init":1,"lbm_read_time_us":5568,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":7,"num_ops":5,"prepare.queue_time_us":157985,"prepare.run_cpu_time_us":37764,"prepare.run_wall_time_us":100387,"raft.queue_time_us":529287,"raft.run_cpu_time_us":34683,"raft.run_wall_time_us":195095,"replication_time_us":826556,"spinlock_wait_cycles":1028352,"thread_start_us":748535,"threads_started":4,"wal-append.queue_time_us":220145}]]}
I20260709 13:01:37.737491 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:37.646057 24584 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:01:37.683136 24603 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:37.682535 24593 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:37.752152 20008 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:37.685969 24191 raft_consensus.cc:697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:37.768386 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.768766 20858 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:37.768915 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.689656 24563 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:37.772962 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:37.786481 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:37.783640 20084 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:37.787182 20013 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:01:37.772683 19838 catalog_manager.cc:5697] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:37.773697 23731 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:37.757946 20085 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 18.
I20260709 13:01:37.779145 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.779194 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.772416 24603 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.780866 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.782702 20090 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:37.789750 20089 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:01:37.760289 24614 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.794679 20056 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.747668 20163 raft_consensus.cc:2468] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
I20260709 13:01:37.797051 20010 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:01:37.798156 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 167100us
W20260709 13:01:37.799495 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 129993us
I20260709 13:01:37.797248 19838 catalog_manager.cc:5697] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 15, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 15 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:37.771942 24553 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:37.803553 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.803570 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.807003 20056 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.774241 24593 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:37.813962 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:37.813894 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:37.815532 20010 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.815173 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.816337 19838 catalog_manager.cc:5697] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 2 to 15, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 15 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.818337 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.796269 24603 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:37.707065 24576 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:37.824559 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.824846 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.825292 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:37.825898 24595 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:37.826439 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:37.828536 20852 raft_consensus.cc:2368] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
I20260709 13:01:37.829195 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:37.799783 24614 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:37.801009 24619 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:37.833319 24574 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:37.834038 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.832665 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.815182 24607 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:37.838420 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.849032 20165 raft_consensus.cc:1275] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:01:37.815989 24593 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.794880 24501 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:01:37.873620 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.873709 20017 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:01:37.802759 24448 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:01:37.829118 24553 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:37.856912 19838 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 13, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.881803 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:37.878159 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 91762.9us
I20260709 13:01:37.786525 24262 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:37.856635 24607 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:37.880887 24589 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:37.889181 20082 raft_consensus.cc:2468] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:37.906224 20163 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:37.889472 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.923774 19838 catalog_manager.cc:5697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 6 to 11, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 11 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.901896 20010 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:37.891978 20017 raft_consensus.cc:1240] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:01:37.905781 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.963073 20088 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:01:37.965514 19838 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 12, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:37.969290 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:37.969730 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.856388 24619 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:37.973188 24619 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:37.881699 24191 consensus_queue.cc:237] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:37.890643 24574 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:37.915035 24607 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:37.986685 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 101303us
I20260709 13:01:37.990113 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:37.991015 20013 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:37.995448 19838 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 12, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:38.003777 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:37.986382 24589 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.016372 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:38.016757 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:38.001535 24605 consensus_meta.cc:287] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.002s
I20260709 13:01:37.987195 24567 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:37.984396 24548 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:38.018906 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:38.021770 20012 raft_consensus.cc:1240] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:38.022635 19838 catalog_manager.cc:5697] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 13, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:38.027004 20854 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:38.027897 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:38.031324 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:38.038663 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.036686 19838 catalog_manager.cc:5697] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 12, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:38.039335 20013 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:38.025938 24506 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:38.010811 24395 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:38.047860 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:38.021577 24574 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:01:38.021251 20085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:38.059561 20085 raft_consensus.cc:606] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.060019 20085 raft_consensus.cc:614] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:38.048668 20084 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:38.055356 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:38.025425 24553 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:38.022419 24589 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:38.074819 20082 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:38.075949 24589 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:38.064786 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:38.072053 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:38.073552 24570 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:38.081745 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 1269) took 1051 ms. Trace:
I20260709 13:01:38.082181 20857 rpcz_store.cc:276] 0709 13:01:37.030033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.030181 (+   148us) service_pool.cc:224] Handling call
0709 13:01:38.081541 (+1051360us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400380254. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:38.081661 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620566656}
I20260709 13:01:38.083069 20088 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:38.084311 24576 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:38.085057 20088 raft_consensus.cc:1240] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:38.066660 24624 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:01:38.085103 24191 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:38.073966 24395 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:38.086651 24191 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:38.087107 24191 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.087661 20009 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.065035 19838 catalog_manager.cc:5697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:38.091574 19838 catalog_manager.cc:5697] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:38.104777 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:38.105742 20857 raft_consensus.cc:2368] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11.
I20260709 13:01:38.110100 20743 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:38.111788 20088 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:38.112124 20743 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.112449 20088 raft_consensus.cc:606] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.113281 20088 raft_consensus.cc:614] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:38.114084 19838 catalog_manager.cc:5697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 13, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 13 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:38.114001 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.123885 20130 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.126673 20164 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:38.126953 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.128486 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:38.132458 20014 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 16: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:38.134886 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.135228 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.140709 24428 raft_consensus.cc:2749] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:38.141768 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.142887 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.142951 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.143218 24448 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.144166 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.144510 20129 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.144865 24586 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:38.144550 20163 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.145637 24586 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.146294 24586 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:38.122018 24570 raft_consensus.cc:2749] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.148721 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.149556 24437 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:38.131326 24431 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:38.150223 20161 raft_consensus.cc:3055] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:38.150765 20161 raft_consensus.cc:740] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:38.150795 24568 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:38.151436 24568 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:38.151620 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.152035 24594 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:38.146365 24566 consensus_queue.cc:1059] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:38.145242 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.117487 24617 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.164966 20055 leader_election.cc:400] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11.
I20260709 13:01:38.167750 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:38.172057 20055 leader_election.cc:403] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:38.172377 20082 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.151818 24568 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.190754 20089 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:38.143900 24448 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.118316 24202 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:01:38.192286 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:38.198730 20082 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:38.168728 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.164230 24626 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:38.215615 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 103554us
I20260709 13:01:38.217774 20743 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.204104 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.223101 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:38.224442 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.204964 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.207012 24501 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:38.208706 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:38.209553 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.210246 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.182046 24566 consensus_queue.cc:1243] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer 41af8b41f8a348c5b60ec12d2e323698 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:38.228725 19981 leader_election.cc:400] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
I20260709 13:01:38.243712 20017 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:38.200265 24506 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.194096 24448 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:38.229408 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.243848 19981 leader_election.cc:403] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:38.244419 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.241441 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.150225 24431 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:38.249243 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.262605 19981 leader_election.cc:400] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15.
I20260709 13:01:38.265544 20157 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
I20260709 13:01:38.260040 20161 consensus_queue.cc:260] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:38.264204 24437 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:38.267344 20085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:38.267979 20085 raft_consensus.cc:606] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.261859 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.268232 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:38.269101 20089 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.269070 24437 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:38.271469 24053 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:38.284672 24053 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:01:38.285090 24053 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:38.285956 24053 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:38.273360 20853 raft_consensus.cc:1240] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:38.273554 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:38.266994 20161 raft_consensus.cc:3060] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.276295 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:38.263741 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:38.278288 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 49456.3us
I20260709 13:01:38.270874 20744 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.261407 24448 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.233548 24501 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
W20260709 13:01:38.260919 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 112278us
I20260709 13:01:38.290248 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.177493 24617 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:38.315344 20744 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.250139 24242 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.233709 24570 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:01:38.264389 24360 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
I20260709 13:01:38.246285 24603 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:38.353273 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.344475 20160 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:38.302645 24625 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.297398 24583 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.291250 24574 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:38.356482 24583 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:38.261354 24561 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 13:01:38.366880 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 45184us
I20260709 13:01:38.317272 24501 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.325960 24577 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.330245 24053 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:38.378437 20089 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:38.233182 24626 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:38.343761 24567 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:01:38.386495 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.392976 20011 raft_consensus.cc:2368] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:38.396906 20089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:38.397746 20089 raft_consensus.cc:606] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.398206 20089 raft_consensus.cc:614] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:38.325410 24605 consensus_queue.cc:1059] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:01:38.359176 24584 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.419279 20161 raft_consensus.cc:1275] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:38.373704 24574 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:38.388101 24544 consensus_meta.cc:287] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.000s
W20260709 13:01:38.427028 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 103355us
W20260709 13:01:38.425452 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 125076us
I20260709 13:01:38.378376 24577 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:38.437491 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.382334 24633 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:38.447126 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.375053 24595 consensus_queue.cc:1059] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:01:38.454632 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:38.441344 24550 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:01:38.459074 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.459681 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:38.460877 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 83515.9us
W20260709 13:01:38.457350 24593 consensus_meta.cc:287] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
I20260709 13:01:38.447173 24577 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.470854 19981 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.463164 24262 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:38.471038 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 103195us
I20260709 13:01:38.456951 24558 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:38.433843 24584 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:38.449061 24544 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:38.473804 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.477407 20017 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:38.468624 24595 consensus_queue.cc:1243] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer 48da76e971b84219a6ba80d1ba83b7eb log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:38.464193 24583 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:38.494696 20158 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:38.498294 24560 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:01:38.499344 24646 log.cc:927] Time spent T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:01:38.478699 24262 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:38.461419 24294 consensus_queue.cc:1048] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:01:38.486994 24587 raft_consensus.cc:2804] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:38.484026 24605 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.506500 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.488507 24567 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:01:38.481166 24630 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.209s
I20260709 13:01:38.476933 24593 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:38.511354 24262 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:38.566247 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 59869.2us
I20260709 13:01:38.565341 19981 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:38.570708 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.524497 24560 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:01:38.571337 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 254923us
I20260709 13:01:38.576574 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.577059 20852 raft_consensus.cc:1240] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.585062 24544 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:38.587432 24506 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11.
I20260709 13:01:38.572285 24605 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.588977 24655 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:01:38.560148 24587 raft_consensus.cc:697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:38.541538 24586 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:01:38.602074 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 125270us
W20260709 13:01:38.613519 24644 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:01:38.614290 24578 raft_consensus.cc:3055] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:38.614826 24578 raft_consensus.cc:740] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:38.616873 24657 raft_consensus.cc:3055] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:38.617339 24657 raft_consensus.cc:740] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:38.618376 24574 consensus_queue.cc:1048] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:38.619879 24656 log.cc:927] Time spent T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:01:38.620834 24560 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:38.622119 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 57075.5us
I20260709 13:01:38.573645 24617 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:38.629478 24655 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.629911 24655 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:38.630793 24655 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:38.625190 24570 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.628172 24202 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:38.632570 24202 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
W20260709 13:01:38.632077 24650 log.cc:927] Time spent T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:01:38.627554 20159 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:38.644233 20088 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:38.654320 20852 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:38.572100 24576 consensus_queue.cc:1059] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:01:38.667191 20854 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:38.657050 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.658895 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:38.668381 20089 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.660758 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:38.646881 24202 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:38.635119 24506 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:38.677251 20090 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 11.
W20260709 13:01:38.674050 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 103380us
I20260709 13:01:38.680765 23731 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.681046 20011 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:01:38.681645 20011 pending_rounds.cc:85] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 7
I20260709 13:01:38.681951 20011 pending_rounds.cc:107] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:01:38.682255 20011 raft_consensus.cc:2889] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:38.680264 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:38.684193 24663 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:01:38.684568 24544 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:38.684986 24544 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.688609 20852 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:38.685496 24544 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.690330 20859 raft_consensus.cc:3055] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:38.692644 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:38.692915 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153ccb361669440fa940c70fa6ae8d76" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:38.697333 20164 raft_consensus.cc:3060] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.690704 20859 raft_consensus.cc:740] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:38.648330 24431 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:38.667212 24655 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.708611 20744 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.712338 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:38.713837 20744 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.716378 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:38.717121 20857 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:38.728878 20089 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:38.666669 24626 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:38.738848 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 160624us
I20260709 13:01:38.715602 20859 consensus_queue.cc:260] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:38.748562 20859 raft_consensus.cc:3060] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:38.764382 20056 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.690634 24360 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:01:38.772260 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 51306.2us
I20260709 13:01:38.724413 24630 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:38.767992 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.776384 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:38.777956 20056 leader_election.cc:400] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:38.778258 20056 leader_election.cc:403] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:38.768075 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 69445us
I20260709 13:01:38.615780 24578 consensus_queue.cc:260] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:38.773984 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 84086.4us
I20260709 13:01:38.780573 19981 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:38.723377 24567 consensus_queue.cc:1059] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:38.765549 24584 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:38.767273 24506 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:38.774554 24614 consensus_meta.cc:287] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.900s	user 0.006s	sys 0.007s
W20260709 13:01:38.805969 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 140998us
W20260709 13:01:38.826947 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46740us
I20260709 13:01:38.829348 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:38.829463 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 46644.8us
W20260709 13:01:38.805702 24191 consensus_meta.cc:287] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.717s	user 0.011s	sys 0.003s
I20260709 13:01:38.769300 24626 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:01:38.837754 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:38.836230 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:38.839476 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:38.841986 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:38.840945 20088 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:01:38.848959 24672 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:01:38.632102 24242 consensus_queue.cc:1175] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8b5b4e2a92b046d096f2661c54321d41" responder_term: 12 status { last_received { term: 11 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:38.851382 24664 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15.
I20260709 13:01:38.810135 24593 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:38.742715 24587 consensus_queue.cc:237] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:38.852559 24634 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:38.808986 24548 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:01:38.808082 24616 consensus_queue.cc:1059] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.175s
I20260709 13:01:38.850098 24674 raft_consensus.cc:2749] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:38.839288 24670 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.870026 20164 raft_consensus.cc:2468] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:38.902961 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:38.908468 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 70782.9us
W20260709 13:01:38.913851 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 140872us
I20260709 13:01:38.896795 24578 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:38.929858 20157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:38.905277 24626 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:38.839664 24191 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:38.969700 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 45984us
I20260709 13:01:38.971375 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:38.839841 24614 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.026923 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:39.027652 20011 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
W20260709 13:01:39.032454 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 99262.4us
I20260709 13:01:39.033179 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:39.005168 24607 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.994s	user 0.012s	sys 0.000s
I20260709 13:01:39.035897 20162 raft_consensus.cc:1240] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 15. Current term is 16. Ops: []
W20260709 13:01:39.044649 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 75393.6us
W20260709 13:01:39.035034 24563 consensus_meta.cc:287] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 1.239s	user 0.015s	sys 0.000s
W20260709 13:01:39.034811 24053 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.000s
I20260709 13:01:39.055313 19982 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:39.071944 20859 raft_consensus.cc:1275] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:39.040341 24630 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:01:39.049090 24593 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:01:39.091287 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 58169.3us
W20260709 13:01:39.091825 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:39.092813 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 47256.8us
W20260709 13:01:39.079578 24594 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:01:39.042479 24607 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:39.059744 24563 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:39.125165 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:39.121981 24262 consensus_meta.cc:287] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.002s
I20260709 13:01:39.128084 20855 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:39.135145 24679 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:39.135753 24679 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
W20260709 13:01:39.128090 24501 consensus_meta.cc:287] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.005s
I20260709 13:01:39.136497 24501 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.136873 24679 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:39.137605 24675 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.285s	user 0.001s	sys 0.001s
I20260709 13:01:39.138667 24501 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.139420 20085 raft_consensus.cc:1240] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:39.139953 24687 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:01:39.140686 24692 raft_consensus.cc:2804] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:39.141218 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:39.141395 24694 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:39.147264 20164 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:39.127332 24262 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:39.131901 24673 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
W20260709 13:01:39.125321 24577 consensus_meta.cc:287] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.643s	user 0.006s	sys 0.006s
W20260709 13:01:39.122917 24568 consensus_meta.cc:287] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.925s	user 0.011s	sys 0.002s
W20260709 13:01:39.131615 24448 consensus_meta.cc:287] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.809s	user 0.011s	sys 0.002s
I20260709 13:01:39.190580 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:39.132278 24191 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.166134 24692 raft_consensus.cc:697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:39.142762 24548 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:39.135766 24607 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:39.202317 20011 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:39.188961 24501 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:39.221302 20859 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:01:39.225665 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 240043us
I20260709 13:01:39.225711 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:39.186826 24694 consensus_queue.cc:1059] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:01:39.199726 24568 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:39.200062 24577 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.201136 24448 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.226403 24614 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:39.232044 24589 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:39.271353 24697 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:39.278127 24697 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.280768 24577 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.281102 24262 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.281646 20856 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:39.282078 24668 log.cc:927] Time spent T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.623s	user 0.002s	sys 0.000s
I20260709 13:01:39.258155 24607 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:39.284314 24607 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.286785 24607 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.277469 20156 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:39.281982 24563 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:39.260967 24674 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:39.288424 24674 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.288516 24657 consensus_queue.cc:260] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:39.272787 24696 log.cc:927] Time spent T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:01:39.290153 24634 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:39.290782 24674 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.291287 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:39.292284 20165 raft_consensus.cc:1240] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:39.273933 24692 consensus_queue.cc:237] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.224575 24501 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:39.297868 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 196959us
I20260709 13:01:39.307137 24501 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:39.308423 24568 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:39.323781 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 101711us
I20260709 13:01:39.332454 20087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:39.333171 20087 raft_consensus.cc:606] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.334236 20009 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:39.340921 24704 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:01:39.341845 20855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44218 (request call id 1271) took 1543 ms (client timeout 1664 ms). Trace:
I20260709 13:01:39.342197 24704 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:39.342657 24704 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:39.344247 20085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36124 (request call id 1660) took 1063 ms. Trace:
I20260709 13:01:39.344708 20085 rpcz_store.cc:276] 0709 13:01:38.280528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.281148 (+   620us) service_pool.cc:224] Handling call
0709 13:01:38.281578 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.139887 (+858309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.343963 (+204076us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54009b8f54. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:39.344175 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1972336896}
W20260709 13:01:39.342339 20855 rpcz_store.cc:269] 0709 13:01:37.798936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.799357 (+   421us) service_pool.cc:224] Handling call
0709 13:01:39.341683 (+1542326us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54006eaad4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:39.341788 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3015913344}
I20260709 13:01:39.348307 20085 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:39.349643 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:39.349876 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:39.323693 24501 consensus_queue.cc:1059] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:01:39.351680 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:39.353085 20017 raft_consensus.cc:1240] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:01:39.364876 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:39.366256 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:01:39.373343 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.371906 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:39.377974 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.389081 24704 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:39.390463 24704 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:39.390686 24544 consensus_meta.cc:287] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.002s
I20260709 13:01:39.392210 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:39.393404 20850 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:39.392328 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.396603 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.396549 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:39.391384 24544 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:39.416152 20854 raft_consensus.cc:1240] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:39.421693 24544 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:39.408314 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:39.377444 24697 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.405723 24605 raft_consensus.cc:3055] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:39.439851 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.452562 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.453226 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:39.455200 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.458086 24633 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:39.438519 24605 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:39.450354 24544 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.464007 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.465152 20854 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
W20260709 13:01:39.469070 24695 log.cc:927] Time spent T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.252s	user 0.002s	sys 0.000s
I20260709 13:01:39.474872 24544 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:39.545152 24595 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:39.588531 24710 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:01:39.592373 24662 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:01:39.593120 24707 raft_consensus.cc:493] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:39.593746 24707 raft_consensus.cc:515] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:39.582974 24583 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:39.597994 24583 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.604661 24583 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.604687 24603 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.605363 24605 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.606877 24605 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:39.607623 24603 raft_consensus.cc:493] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:39.608250 24603 raft_consensus.cc:515] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.610199 24603 leader_election.cc:290] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.617957 24704 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.624400 24704 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:39.627044 24704 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:39.633535 24544 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:39.610663 24595 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.637897 24595 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.648618 24713 raft_consensus.cc:493] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.649374 24713 raft_consensus.cc:515] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.652501 24437 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:39.653122 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.653687 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.655452 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.658089 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.660368 24706 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:39.661521 24706 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:01:39.663081 24722 log.cc:927] Time spent T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:01:39.663968 24712 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:39.664072 24574 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.664491 24712 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:39.664757 24574 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:39.665591 24712 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.667305 24712 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:39.668318 24574 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:39.684067 20850 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:39.693885 24574 raft_consensus.cc:493] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:39.694612 24574 raft_consensus.cc:515] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.696781 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33788 (request call id 1046) took 1554 ms. Trace:
I20260709 13:01:39.697217 20165 rpcz_store.cc:276] 0709 13:01:38.142624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.144646 (+  2022us) service_pool.cc:224] Handling call
0709 13:01:38.144933 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.292718 (+1147785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.696485 (+403767us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540009b854. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:39.696678 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638201728}
I20260709 13:01:39.697849 24574 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.702013 24633 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:39.702131 24395 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:39.702564 24633 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.703640 20056 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:39.705209 24589 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:39.706409 20084 raft_consensus.cc:1240] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:39.706540 24448 leader_election.cc:290] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.702651 24395 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:39.710098 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:39.710836 20085 raft_consensus.cc:2368] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15.
I20260709 13:01:39.711073 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:39.711725 20088 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 18.
I20260709 13:01:39.712888 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:39.713948 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:39.714541 20087 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:39.715442 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:39.717159 24544 consensus_queue.cc:1059] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:39.717799 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:39.718274 24544 raft_consensus.cc:3055] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:39.653144 24437 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.718838 24544 raft_consensus.cc:740] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:39.718506 20011 raft_consensus.cc:2393] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:39.720216 20161 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 8.
I20260709 13:01:39.720547 24437 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.719784 24544 consensus_queue.cc:260] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.721570 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.720118 24395 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:39.722098 24544 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:39.722590 24707 leader_election.cc:290] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.664373 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:39.723909 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 64065.4us
I20260709 13:01:39.724010 20087 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:39.725455 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:39.736404 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.727953 24713 leader_election.cc:290] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.732074 24633 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.736325 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.726184 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.742909 20087 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:39.740554 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:39.746842 20082 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:39.754540 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.758268 20743 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:39.759362 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.762212 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:39.742370 24553 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:39.764078 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:39.763958 24664 consensus_queue.cc:1059] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:39.764005 20743 leader_election.cc:400] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15.
I20260709 13:01:39.766471 20743 leader_election.cc:403] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:01:39.767390 24053 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:39.769658 24654 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:39.769737 24622 consensus_queue.cc:1059] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:01:39.770308 24654 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:39.771102 24625 raft_consensus.cc:3055] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:39.771468 24625 raft_consensus.cc:740] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:39.772442 24625 consensus_queue.cc:260] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:39.773664 24625 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:39.780913 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ccd50eb99764ffc91bd06202fd0a5e7" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.778100 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:39.785737 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1a8f437ee9497c987ac7a6206d2338" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.788055 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:39.790539 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:39.791282 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.791893 20164 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:39.790936 20056 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.796095 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:39.798154 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.799659 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.788403 24563 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:01:39.823007 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.818965 24563 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.792868 24395 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:39.817349 24664 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:39.828002 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:39.819301 24576 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:39.854686 23731 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:39.828826 24563 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:01:39.853127 24664 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.856752 24576 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:39.858995 20159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1499) took 1167 ms. Trace:
I20260709 13:01:39.817162 24574 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:39.815156 24506 raft_consensus.cc:3055] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:01:39.860674 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 104467us
I20260709 13:01:39.874795 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.874217 20159 rpcz_store.cc:276] 0709 13:01:38.691418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.691571 (+   153us) service_pool.cc:224] Handling call
0709 13:01:38.691874 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.767624 (+ 75750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.767662 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.425012 (+657350us) spinlock_profiling.cc:243] Waited 9.2 ms on lock 0x7b2c001e7048. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:01:39.450181 (+ 25169us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.545082 (+ 94901us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:39.545124 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.545190 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.858667 (+313477us) raft_consensus.cc:1685] finished
0709 13:01:39.858688 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.858931 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30846,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":7906,"spinlock_wait_cycles":21153536,"thread_start_us":142340,"threads_started":2,"wal-append.queue_time_us":351939}
I20260709 13:01:39.888484 20164 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:39.890278 23731 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:39.890774 23731 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:39.829075 24191 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:39.896044 24191 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.862284 24578 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.900372 24583 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:01:39.904464 24721 log.cc:927] Time spent T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:01:39.900756 24578 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:39.771307 24654 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:39.826087 24560 raft_consensus.cc:493] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:39.899606 24191 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.912547 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.859104 24603 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:01:39.861048 24674 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:39.893595 24506 raft_consensus.cc:740] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:39.871572 24707 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:01:39.892355 24671 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:01:39.871065 24692 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:01:39.923564 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.920725 24053 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15.
I20260709 13:01:39.924392 24431 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:39.946678 23731 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:39.913961 24560 raft_consensus.cc:515] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.951823 20857 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:39.962327 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:39.949847 24568 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:39.949246 24574 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:39.967938 24574 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:39.969540 24560 leader_election.cc:290] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:39.948617 24617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.099s	user 0.013s	sys 0.010s
I20260709 13:01:39.970966 23731 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:39.970078 24574 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:39.980635 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:39.930974 24674 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:39.971722 24617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.013s	sys 0.010s
I20260709 13:01:39.934154 24576 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:39.992273 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 100930us
I20260709 13:01:39.994812 23731 consensus_queue.cc:1059] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:39.996336 19981 leader_election.cc:304] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.004025 23731 raft_consensus.cc:2749] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:40.005613 19981 leader_election.cc:304] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:39.991062 24603 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.008203 23731 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:40.005039 24707 raft_consensus.cc:3055] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:40.013621 24191 consensus_queue.cc:1048] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:40.023051 20744 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:39.994722 24506 consensus_queue.cc:260] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:40.008423 24680 raft_consensus.cc:2749] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:40.046010 24202 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:40.047737 24202 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:40.048713 24202 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.050146 24202 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:40.063458 24664 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:40.074617 20859 raft_consensus.cc:2468] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
W20260709 13:01:40.078855 20090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44890 (request call id 899) took 1908 ms (client timeout 2304 ms). Trace:
I20260709 13:01:40.092259 20855 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:01:40.079403 20090 rpcz_store.cc:269] 0709 13:01:38.167882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.168052 (+   170us) service_pool.cc:224] Handling call
0709 13:01:40.076305 (+1908253us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b5400258354. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:40.076401 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919180032}
I20260709 13:01:40.041375 24707 raft_consensus.cc:740] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:40.117923 24516 raft_consensus.cc:3055] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:40.118185 24674 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:40.118678 24516 raft_consensus.cc:740] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:40.132318 24516 consensus_queue.cc:260] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.036609 24603 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:40.141390 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 113694us
I20260709 13:01:40.143545 19981 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:40.138769 24657 raft_consensus.cc:3060] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.169492 19981 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:40.173105 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.174045 20016 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:40.180712 20090 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:40.185007 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.188306 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:40.167416 24653 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:40.190318 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.192664 20089 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:01:40.198414 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:40.199504 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.166276 24626 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:01:40.213654 24698 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:40.179617 24616 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:40.165402 24680 consensus_queue.cc:1059] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:01:40.238685 20087 raft_consensus.cc:1275] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:40.141002 24707 consensus_queue.cc:260] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.200894 24561 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.267330 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:40.281632 24664 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:40.286144 24664 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:40.286630 24664 raft_consensus.cc:3060] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:40.287186 24563 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.290139 24563 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 19 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:40.260675 24437 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.295729 20854 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 18. Current term is 19. Ops: []
I20260709 13:01:40.297968 24707 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.311347 20130 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.319051 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.309670 24603 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:40.282271 24561 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.310195 24568 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:40.306304 24437 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.323276 24448 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:40.337831 24448 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:40.331630 24671 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:01:40.349385 24448 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:40.364830 24677 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:40.365267 24677 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:40.366199 24506 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:40.366124 24677 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:40.367378 24677 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.334357 24568 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.350744 24437 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.353474 24578 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:40.378328 20858 raft_consensus.cc:1240] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:40.380064 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:40.358582 20851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1289) took 1717 ms. Trace:
W20260709 13:01:40.378144 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 355167us
W20260709 13:01:40.353963 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.385023 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:40.382400 20851 rpcz_store.cc:276] 0709 13:01:38.640841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.641037 (+   196us) service_pool.cc:224] Handling call
0709 13:01:38.641378 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.641636 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:39.753127 (+1111491us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.993101 (+239974us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.207080 (+213979us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b1800219e20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c5522 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:40.358012 (+150932us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b1800219e20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b63fc0a 00007fa034d7f219 00007fa02b6c7a40 00007fa02b6c24e4 00007fa02b6c2455 00007fa02e7aa060 00007fa02e7bc716 00007fa02e7cd287 00007fa02e7cd208
0709 13:01:40.358187 (+   175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.358246 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:40.358303 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.358374 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.358491 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1150250,"prepare.queue_time_us":902046,"prepare.run_cpu_time_us":31432,"prepare.run_wall_time_us":33200,"raft.queue_time_us":370381,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":3470,"spinlock_wait_cycles":6491136,"thread_start_us":530849,"threads_started":2,"wal-append.queue_time_us":260213}
I20260709 13:01:40.408375 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.409340 20014 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 12.
I20260709 13:01:40.416041 20857 raft_consensus.cc:1275] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:01:40.416478 20857 pending_rounds.cc:85] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 6
I20260709 13:01:40.416697 20857 pending_rounds.cc:107] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:01:40.416930 20857 raft_consensus.cc:2889] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.417613 24448 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.377684 24574 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:40.379288 24676 raft_consensus.cc:3055] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:40.420151 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:40.388634 24679 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:40.433861 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:40.434451 20129 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:40.433080 24688 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:01:40.433373 24262 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.439513 24713 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:40.393389 24437 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:40.451308 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.454344 24568 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:40.454989 24568 raft_consensus.cc:2804] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:40.455524 24568 raft_consensus.cc:697] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:40.433974 24676 raft_consensus.cc:740] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:40.456322 24437 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.455595 24698 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:01:40.457553 24676 consensus_queue.cc:260] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.458514 24437 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:40.440023 24576 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.460489 24576 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.428124 24671 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:40.394423 24664 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.433225 24694 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:40.450852 24561 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.479538 24694 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:40.480088 24671 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.481879 24664 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:40.441526 24574 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.445359 24657 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.485567 24657 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.450978 24262 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.486699 24616 raft_consensus.cc:3055] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:40.487236 24616 raft_consensus.cc:740] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:40.441974 24202 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:01:40.488412 20851 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:40.480108 24694 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:40.492709 20017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57974 (request call id 855) took 1156 ms. Trace:
I20260709 13:01:40.493463 24633 raft_consensus.cc:493] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.493249 20017 rpcz_store.cc:276] 0709 13:01:39.336823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.336961 (+   138us) service_pool.cc:224] Handling call
0709 13:01:39.337329 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.355852 (+ 18523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.492339 (+1136487us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54002463d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b2e01 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2 00007fa02caed48c 00007fa02caec8c6
0709 13:01:40.492600 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33662208}
I20260709 13:01:40.494284 24633 raft_consensus.cc:515] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.487704 24657 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:40.498626 24690 consensus_queue.cc:1059] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:40.474727 24622 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:40.500231 24622 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.500761 24622 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.446962 24679 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.502965 24690 consensus_queue.cc:1048] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:40.472492 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.504953 24690 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:40.506031 24674 raft_consensus.cc:993] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:40.506557 24674 raft_consensus.cc:1081] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:40.508294 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:40.509934 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.510649 20158 raft_consensus.cc:2393] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 16: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:40.511515 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 43598.3us
I20260709 13:01:40.512923 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:40.514417 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.515537 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.519286 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:40.521034 20009 raft_consensus.cc:2393] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:40.532277 20743 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:40.534034 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:40.538341 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.538378 20851 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:40.561221 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35516f88d604db2ad4a7f67d0017312" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.583557 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:40.584329 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 52137.1us
W20260709 13:01:40.589617 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.592598 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.549338 24690 consensus_queue.cc:1243] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:40.573158 24654 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:40.618537 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 105698us
I20260709 13:01:40.583777 24657 raft_consensus.cc:493] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:40.600474 24242 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:40.589922 24674 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.630371 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.593423 24191 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.634744 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d934e44cfcb452b9a7d72dc634e9e77" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.650461 20850 raft_consensus.cc:2393] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:40.636547 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.647348 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:40.652581 24574 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.623131 24664 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.654134 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.654664 24688 raft_consensus.cc:493] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.626073 24294 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.655169 24688 raft_consensus.cc:515] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.655512 24294 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.656180 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.656987 20853 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:40.657312 24688 leader_election.cc:290] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.657718 24294 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.657823 24688 raft_consensus.cc:493] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.657999 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:40.658603 20017 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:40.658452 24688 raft_consensus.cc:515] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.626747 24589 raft_consensus.cc:493] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:40.660056 24589 raft_consensus.cc:515] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.660893 20009 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:40.661975 24589 leader_election.cc:290] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:40.662549 24191 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:40.664772 24191 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:40.654130 24664 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:40.668027 24664 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:40.658529 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:40.669181 20013 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:40.670105 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.670383 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.670806 20090 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:40.671530 24576 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:40.672348 24674 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.672662 24568 consensus_queue.cc:237] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.674664 24674 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:40.675074 20850 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:01:40.675324 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.600183 23958 raft_consensus.cc:493] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:40.676802 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.677160 23958 raft_consensus.cc:515] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.679168 23958 leader_election.cc:290] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.679711 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.680337 20009 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:40.680446 24262 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.680599 20855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44216
I20260709 13:01:40.681380 20855 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:40.681578 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.682096 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.680354 20851 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:40.683563 24242 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:40.684187 24633 leader_election.cc:290] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.584746 24616 consensus_queue.cc:260] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:40.684593 24242 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.685563 24671 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.633587 24578 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:40.687005 24578 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.687408 24578 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.687904 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.686946 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.688268 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.689522 24191 consensus_queue.cc:1059] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:40.690435 24191 consensus_queue.cc:1243] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:01:40.691181 24747 log.cc:927] Time spent T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:40.691008 24191 consensus_queue.cc:1048] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:40.691610 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.693163 24262 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:40.693989 20084 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:40.694940 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.695276 19982 leader_election.cc:304] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:40.645758 24657 raft_consensus.cc:515] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.696024 24653 raft_consensus.cc:2749] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:40.697551 24679 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:40.697675 24657 leader_election.cc:290] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:40.699119 20855 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.700542 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.701030 19982 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:40.702476 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.703410 24653 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:40.704205 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.705451 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.706223 20164 raft_consensus.cc:2393] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 10: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:40.616087 24661 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.712975 24681 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:40.712823 24661 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.716538 20011 raft_consensus.cc:2410] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:01:40.724371 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:40.725760 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 291360us
I20260709 13:01:40.727342 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.728972 20008 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:01:40.729421 20008 pending_rounds.cc:85] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 9
I20260709 13:01:40.729666 20008 pending_rounds.cc:107] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:01:40.729933 20008 raft_consensus.cc:2889] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.731849 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8749de35944e3d92b86f7d4c472de2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.735105 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.723402 20055 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.736969 20016 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:40.737111 20055 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.741372 19982 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.742487 20859 raft_consensus.cc:2468] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:40.730401 19981 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:40.753499 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.754511 20086 raft_consensus.cc:2410] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:40.755270 20055 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.756238 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.756636 20055 leader_election.cc:304] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.752327 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.763151 20851 raft_consensus.cc:2393] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:40.765355 20017 raft_consensus.cc:2410] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:01:40.766031 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dd40d980c94ccdab3d167b33984c44" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.771616 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.764779 20855 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.771612 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.775609 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:40.778661 19982 leader_election.cc:304] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.780166 24653 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:40.781128 24680 raft_consensus.cc:2749] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:40.781994 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.783560 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:40.784456 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.716591 24661 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.762396 24674 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:40.780865 20055 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:40.807752 20850 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:40.806849 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:40.811491 20013 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.762378 24506 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:40.766546 24501 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.761765 24713 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:40.810490 20055 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:40.815713 19982 leader_election.cc:304] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.777150 24633 raft_consensus.cc:3060] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:40.818039 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 196089us
W20260709 13:01:40.819722 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.830678 24262 raft_consensus.cc:2749] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:40.820369 24664 raft_consensus.cc:2804] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:40.838577 20013 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:40.841111 24558 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:40.844026 24558 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:40.845738 24694 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:40.846146 24558 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:40.848974 24688 leader_election.cc:290] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:40.818318 24677 consensus_queue.cc:1059] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:01:40.789873 24566 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:40.865765 20744 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:40.868330 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.872112 20090 raft_consensus.cc:2393] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 11: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:40.868553 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acf254dbaee44d1bc844da86793ef92" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.842044 24713 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.868819 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:40.877032 19981 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.880172 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:40.881551 20855 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:40.805182 24680 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:40.877395 20852 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 11.
I20260709 13:01:40.818569 24589 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:40.913470 19981 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:40.913626 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:40.915577 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 49904.3us
I20260709 13:01:40.916041 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.920182 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:40.917809 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:40.924587 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 116330us
I20260709 13:01:40.851869 24677 raft_consensus.cc:3055] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:40.928328 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:40.813726 24053 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:01:40.911271 24690 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:40.860728 24664 raft_consensus.cc:697] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:40.996798 24664 consensus_queue.cc:237] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:40.998127 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 60348.1us
I20260709 13:01:40.878069 24617 consensus_queue.cc:1059] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 13:01:40.959302 19983 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.926746 24563 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:41.013098 24563 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:41.014086 24563 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:41.005666 24578 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:40.857019 24566 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.000519 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:41.023883 20129 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.860572 24622 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.025149 20129 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:40.938611 20086 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:41.010737 24516 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.930910 24677 raft_consensus.cc:740] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:40.934466 24615 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:40.917174 24589 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:41.020298 24294 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:41.020169 24643 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:41.021622 24578 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:41.015887 24674 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.031937 20744 leader_election.cc:304] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.047039 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.048575 20164 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:41.049783 20158 raft_consensus.cc:1240] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:41.109440 19982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.111279 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69406.8us
W20260709 13:01:41.112944 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 87820us
W20260709 13:01:41.125566 20083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.007s	sys 0.038s
I20260709 13:01:41.147027 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:41.152171 20859 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:41.149366 24615 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:41.108706 24754 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:41.163947 20083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.007s	sys 0.038s
I20260709 13:01:41.109303 24681 raft_consensus.cc:2749] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:41.168473 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 136665us
I20260709 13:01:41.166929 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:41.147322 24768 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.152980 24589 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.144160 24671 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
W20260709 13:01:41.183398 20130 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.159379 23958 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:41.174032 24615 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.180897 24754 raft_consensus.cc:493] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.165737 24697 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:41.180687 24595 raft_consensus.cc:2749] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:41.208067 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 279844us
I20260709 13:01:41.218271 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.219168 24617 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.179612 24428 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:01:41.225966 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af76b6f84f54d249f7949cf3c104e32" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:41.196468 24437 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:41.241705 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:41.244799 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde87ca1d98f4bd8b90d7f1502cc67f1" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:41.246258 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:41.248801 20088 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 11: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:41.188369 24578 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:01:41.269032 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b9595f271b64d9caeb8e95eb6890e38" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:41.270993 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:41.271826 20013 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:41.276250 20129 leader_election.cc:304] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.270151 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:41.218611 24754 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.279131 20009 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:41.277998 20056 leader_election.cc:304] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.196051 24671 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.289542 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.192835 24677 consensus_queue.cc:260] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.280355 24767 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.282933 24568 raft_consensus.cc:2749] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:41.290686 24762 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.239078 24697 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.296196 24294 raft_consensus.cc:2749] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:01:41.329579 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 153704us
I20260709 13:01:41.318713 24767 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.338785 24767 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.339412 24679 consensus_queue.cc:1059] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:41.340677 24679 raft_consensus.cc:3055] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:41.341068 24679 raft_consensus.cc:740] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:41.341969 24679 consensus_queue.cc:260] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.343223 24679 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:41.350044 24448 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:41.353467 24768 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.354004 24697 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:41.364473 24501 consensus_meta.cc:287] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.003s
I20260709 13:01:41.365036 24501 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.375559 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:41.358316 24674 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:41.373180 24622 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:41.402894 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 125205us
I20260709 13:01:41.331784 24762 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:41.404803 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 115392us
I20260709 13:01:41.380656 24674 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.383137 24566 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.414388 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.419180 20129 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.421075 24671 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.411744 24674 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:41.431844 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.434315 24589 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.437531 24762 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.440443 20086 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 10.
I20260709 13:01:41.441871 24589 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.444103 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054afb193dbf43d48b1b0ba6ce809ee3" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.448763 24739 raft_consensus.cc:3055] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:01:41.449416 24739 raft_consensus.cc:740] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:41.450491 24739 consensus_queue.cc:260] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:41.419865 24566 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.457753 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.459470 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.460180 20156 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:41.461894 24676 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:41.462360 24789 log.cc:927] Time spent T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:01:41.462771 24633 consensus_meta.cc:287] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.000s
I20260709 13:01:41.467149 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:41.467337 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.451896 24739 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:01:41.467969 20162 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:41.472201 24544 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.472396 24787 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:41.472818 24544 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.473970 20743 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:41.475003 24544 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:41.476622 20010 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:41.477465 24563 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:41.478084 24563 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.479542 24516 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.470052 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 49766.6us
I20260709 13:01:41.480952 20743 leader_election.cc:304] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.479717 24576 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:01:41.468994 20056 leader_election.cc:304] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.482357 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.480932 24516 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.489485 24694 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:41.463636 24633 raft_consensus.cc:2749] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.491408 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.492013 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:41.492357 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:41.496522 24664 raft_consensus.cc:2749] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:41.498433 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:41.504195 24576 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.513497 24576 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:41.514919 23958 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:41.515520 23958 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:41.498251 24754 raft_consensus.cc:515] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.520983 24754 leader_election.cc:290] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.521752 20156 raft_consensus.cc:2368] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11.
I20260709 13:01:41.523615 20851 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:41.526386 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.527132 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.527735 20858 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:41.528669 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:41.529664 24616 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:41.530117 24563 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:41.515573 24664 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.532198 24664 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:41.532585 24664 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.497884 24516 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.497063 24694 raft_consensus.cc:2749] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:41.537683 20857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44218 (request call id 1313) took 1022 ms. Trace:
W20260709 13:01:41.539064 19981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.538094 20857 rpcz_store.cc:276] 0709 13:01:40.515377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.518061 (+  2684us) service_pool.cc:224] Handling call
0709 13:01:40.576004 (+ 57943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.594760 (+ 18756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.594806 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:40.620077 (+ 25271us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.076792 (+456715us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:41.077271 (+   479us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.116949 (+ 39678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.117060 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.511006 (+393946us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b18000de820. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02b6c53f3 00007fa02b6c77c1 00007fa02b6c7779 00007fa02b6c7709 00007fa02b6c76d1 00007fa02b6c69cd 000000000038f354 000000000038f199 00007fa02b6a883a
0709 13:01:41.537397 (+ 26391us) raft_consensus.cc:1685] finished
0709 13:01:41.537502 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.537628 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53308,"prepare.queue_time_us":163,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":2800,"raft.queue_time_us":60422,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":31687,"spinlock_wait_cycles":17949952,"thread_start_us":456217,"threads_started":1,"wal-append.queue_time_us":545292}
I20260709 13:01:41.540884 20855 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:41.542114 20055 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:41.543193 24762 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.543571 24762 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.543907 24762 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.545323 20055 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:41.549225 24760 raft_consensus.cc:3055] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:41.549824 24760 raft_consensus.cc:740] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:41.550838 24760 consensus_queue.cc:260] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:41.552536 24654 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:41.553105 24654 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:41.553964 24654 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.563632 24760 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.576294 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:41.577219 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 75029.5us
W20260709 13:01:41.579982 20084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36124 (request call id 1679) took 1397 ms (client timeout 1624 ms). Trace:
W20260709 13:01:41.580456 20084 rpcz_store.cc:269] 0709 13:01:40.182274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.182395 (+   121us) service_pool.cc:224] Handling call
0709 13:01:41.579807 (+1397412us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540016c1d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:41.579900 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135700864}
I20260709 13:01:41.602008 20858 raft_consensus.cc:1240] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:01:41.606669 20056 leader_election.cc:400] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11.
I20260709 13:01:41.612365 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.614791 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.617899 20013 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:41.619433 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.626782 20850 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:41.626945 20012 raft_consensus.cc:1240] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:41.620837 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.629825 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:41.631781 24506 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.632620 24506 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:41.636350 24506 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:41.666854 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:41.661743 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:41.647396 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:41.653707 19980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.679162 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.679768 20851 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:41.637106 24576 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.647791 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:41.636785 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:41.681023 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:41.667770 20091 raft_consensus.cc:1275] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:41.684000 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:41.684294 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:41.682235 20859 raft_consensus.cc:2468] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:41.635251 23958 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:41.688882 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:41.688416 20084 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:01:41.690238 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.692881 20857 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
W20260709 13:01:41.691258 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 54595.5us
I20260709 13:01:41.706754 20091 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:41.696044 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:41.700094 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.709800 20855 raft_consensus.cc:2393] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:41.693372 20010 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:41.697594 24679 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:41.697465 24677 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.724807 24664 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:41.722532 20017 raft_consensus.cc:1240] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:41.720526 24679 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.728885 24679 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.729894 24633 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.730379 24633 raft_consensus.cc:493] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.730706 24633 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.701746 24624 raft_consensus.cc:493] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:41.734144 20857 raft_consensus.cc:1240] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:41.732874 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:41.738914 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05d19c460b3d4d2cab96fa3f1372b25b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:41.734993 19981 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.739571 20851 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.731567 24780 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:41.746750 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:41.747452 20012 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.750577 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:41.750921 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:41.751492 20009 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:41.755878 20855 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:41.750074 20856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44216 (request call id 1380) took 1076 ms. Trace:
I20260709 13:01:41.695466 24712 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:41.762511 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.764058 20016 raft_consensus.cc:1240] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:41.788089 23958 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:41.790606 19981 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.792285 24664 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:41.793680 24583 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:41.794144 24583 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.794481 24583 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:41.743566 24624 raft_consensus.cc:515] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.796958 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.798049 24624 leader_election.cc:290] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.798308 24625 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.799837 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:41.765950 24561 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.800685 24561 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:41.801182 24633 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:41.802819 24561 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:41.803263 24633 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:41.757344 24501 consensus_queue.cc:1059] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:41.804481 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.805819 24622 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:41.806181 24622 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.806478 24622 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:41.769738 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:41.775995 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:41.776842 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.743698 24645 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:41.813654 20009 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 13: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:41.743654 24692 raft_consensus.cc:2764] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:01:41.813637 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.814381 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:41.817914 20857 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:41.814582 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:41.815429 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:41.815573 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 53177.1us
I20260709 13:01:41.815806 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd1b4e7346b47ffaa37a1f24e287d3a" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:41.821125 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:41.757162 24756 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:01:41.784500 20856 rpcz_store.cc:276] 0709 13:01:40.673225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.676536 (+  3311us) service_pool.cc:224] Handling call
0709 13:01:41.747702 (+1071166us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b540000a0d4. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02e71b571 00007fa02e7b62fa 00007fa033777b17 00007fa02ccf26d4 00007fa02ccf2662 00007fa02ccf2591 00007fa02ccf250c 00007fa02ccf17c2 00007fa02caed48c 00007fa02caec8c6 00007fa02caec26f
0709 13:01:41.750008 (+  2306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507910528}
I20260709 13:01:41.819176 20164 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 12.
I20260709 13:01:41.818213 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:41.839015 20851 raft_consensus.cc:2468] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 11.
I20260709 13:01:41.819196 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:41.846266 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:41.819870 20160 raft_consensus.cc:2393] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:41.825109 20855 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.825196 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.852378 20162 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:41.816648 24768 consensus_queue.cc:1048] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:41.817044 24645 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:41.820796 24754 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.783810 24643 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:01:41.846109 24681 consensus_queue.cc:1059] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:01:41.854817 24360 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.873572 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:41.845798 24782 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.867349 24563 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:41.884168 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 57121us
I20260709 13:01:41.870744 24437 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:41.889225 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:41.868491 24645 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.884387 20129 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:41.873092 24360 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.857467 24674 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:41.881683 24202 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.883430 24643 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:41.894474 20855 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:41.906656 20129 leader_election.cc:304] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:41.907560 20055 leader_election.cc:304] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:41.899646 24660 raft_consensus.cc:3055] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:41.901831 24501 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:41.931351 20012 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:41.931802 20083 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:41.910166 24202 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:41.945003 20015 raft_consensus.cc:1240] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:01:41.900288 24754 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:01:41.897193 24782 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:01:41.923588 24654 raft_consensus.cc:2804] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:41.904947 24360 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:41.956063 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 109867us
I20260709 13:01:41.928710 24660 raft_consensus.cc:740] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:41.947098 24681 raft_consensus.cc:2804] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:41.967677 20012 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:41.910431 24712 raft_consensus.cc:2804] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:41.908947 24643 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:41.969849 20744 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:41.979146 20742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.979770 24681 raft_consensus.cc:697] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:41.980754 24674 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:41.980953 24681 consensus_queue.cc:237] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:41.984133 24563 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:41.984477 24660 consensus_queue.cc:260] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:41.985342 24793 log.cc:927] Time spent T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:01:41.985041 24576 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:41.985936 24660 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.985968 24712 raft_consensus.cc:697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 15 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:41.956364 24501 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:41.987293 24501 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:41.989624 24563 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:41.991113 24202 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:41.987097 24712 consensus_queue.cc:237] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:41.994022 24615 consensus_meta.cc:287] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.788s	user 0.009s	sys 0.003s
I20260709 13:01:41.994504 24615 raft_consensus.cc:515] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:41.995699 20014 raft_consensus.cc:2468] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
W20260709 13:01:41.997262 24739 consensus_meta.cc:287] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.000s
I20260709 13:01:41.998332 24679 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:42.000809 24679 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:42.002892 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:42.008503 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.008809 20855 raft_consensus.cc:1275] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:42.010660 20055 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:41.983314 24762 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.021358 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:42.027001 20091 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.031543 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:41.989699 24566 consensus_meta.cc:287] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.001s
I20260709 13:01:41.969691 24654 raft_consensus.cc:697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:42.033094 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:42.044094 20008 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.044387 24654 consensus_queue.cc:237] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:42.046355 24566 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:42.033660 24787 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:42.030109 24626 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.035027 24643 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.036104 24633 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:42.059043 20088 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:42.030040 24589 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:41.995047 24622 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.047303 20161 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 11.
I20260709 13:01:42.049794 24626 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:42.047240 24787 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.059304 20008 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:42.036657 24754 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:42.064076 20085 raft_consensus.cc:2368] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13.
I20260709 13:01:42.074072 24626 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:42.066601 24589 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:42.077099 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:42.077755 20087 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:42.078886 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:41.991042 24563 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.071203 20744 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:42.126652 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55542.8us
I20260709 13:01:42.127209 24694 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:42.084724 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:42.128443 20851 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 19: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:42.128547 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.130914 24732 raft_consensus.cc:3055] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:42.131422 24732 raft_consensus.cc:740] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:42.134402 20890 heartbeater.cc:646] Failed to heartbeat to 127.12.30.126:46523 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.30.126:46523 timed out after 15.000s (SENT)
I20260709 13:01:42.029055 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:42.084666 20855 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:01:42.086328 24622 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:42.087236 24589 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:42.088696 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:42.058205 24760 consensus_queue.cc:1048] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:01:42.142479 20015 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 19.
I20260709 13:01:42.080369 24754 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:42.144757 24754 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:42.095945 24626 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:42.145392 24626 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.145780 24626 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:42.146693 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:42.149762 24697 consensus_queue.cc:1059] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:42.157140 24697 raft_consensus.cc:3055] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:42.157650 24697 raft_consensus.cc:740] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:42.158705 24697 consensus_queue.cc:260] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:42.160060 24697 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:42.164619 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:42.166663 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:42.168339 19982 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:42.105259 24615 leader_election.cc:290] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:42.176473 20158 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:42.177491 20162 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:42.114665 24643 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.116055 24544 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.179666 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.183151 20858 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.163061 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:42.184719 19982 leader_election.cc:304] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.116887 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:42.117690 24762 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.121104 24787 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:42.123991 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:42.127795 24694 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:42.151103 24202 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.153664 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:42.212435 20164 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:42.097056 24360 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.107810 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb7d330cd8b4195aba1f8c1348bfcc2" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.186440 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.183845 20160 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:42.055871 24583 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.156437 24645 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.223438 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:42.228607 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 43953.4us
W20260709 13:01:42.234254 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 49294.8us
I20260709 13:01:42.234563 20158 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.232141 20856 raft_consensus.cc:2368] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
W20260709 13:01:42.240284 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 53936.1us
I20260709 13:01:42.242671 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.247219 20091 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:42.225368 24688 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:01:42.249429 20017 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:42.250190 24633 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:01:42.242574 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:42.250653 24633 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:42.251397 24633 raft_consensus.cc:3060] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:01:42.253721 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:42.255533 20159 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:42.256162 20087 raft_consensus.cc:2393] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 15: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:42.257263 20010 raft_consensus.cc:1240] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting Update request from peer 8b5b4e2a92b046d096f2661c54321d41 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:01:42.263829 24603 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:42.264384 24603 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.264716 24603 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.238289 24615 raft_consensus.cc:3060] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.269265 24694 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:42.273043 24671 consensus_queue.cc:1059] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:42.228556 24202 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.247889 24674 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:42.274075 24674 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:42.274423 24645 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.275389 24202 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.276213 20011 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:42.254904 24395 consensus_queue.cc:1059] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:01:42.277091 24395 consensus_queue.cc:1243] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer 8b5b4e2a92b046d096f2661c54321d41 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:42.277586 24395 raft_consensus.cc:3055] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:42.277904 24395 raft_consensus.cc:740] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:42.278781 24395 consensus_queue.cc:260] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.282011 20856 raft_consensus.cc:1240] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:42.211540 24679 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:01:42.198233 24544 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.289788 20087 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:01:42.284544 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.291200 20858 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:42.227602 24595 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:01:42.299016 20055 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:42.274509 24674 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.239730 24626 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:42.295651 24799 log.cc:927] Time spent T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:01:42.334620 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 92139.4us
I20260709 13:01:42.340405 20857 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 14: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:42.296022 24242 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:42.345464 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:42.352921 20159 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:42.347843 20744 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:42.358132 24802 log.cc:927] Time spent T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:01:42.247763 24732 consensus_queue.cc:260] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.323860 24762 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:42.359943 24762 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:42.360826 24762 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.337896 24589 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.364580 24589 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:42.366636 24589 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:42.341357 24395 raft_consensus.cc:3060] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.368059 24797 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:42.368620 24797 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:42.370705 24797 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:42.324942 24768 raft_consensus.cc:3055] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:42.373039 19981 leader_election.cc:400] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
I20260709 13:01:42.373320 24768 raft_consensus.cc:740] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:42.373499 19981 leader_election.cc:403] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:42.374266 24768 consensus_queue.cc:260] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.375428 24768 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.376494 23958 raft_consensus.cc:3060] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.377203 24544 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:42.380357 24242 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.381120 24360 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.381482 20856 raft_consensus.cc:1240] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:42.382663 24242 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:42.349582 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:42.339545 24679 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:01:42.384779 24679 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:42.385157 24679 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:42.385246 24617 consensus_queue.cc:1059] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:42.386047 24679 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.387149 24679 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.389966 24615 raft_consensus.cc:2749] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:42.390223 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:42.391053 20164 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:42.390223 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:42.392246 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:42.392974 20012 raft_consensus.cc:2410] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:01:42.393054 20011 raft_consensus.cc:2393] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:42.396211 24693 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.396772 24693 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.399044 24583 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.400529 24693 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.402235 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.403327 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:42.403991 20855 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
W20260709 13:01:42.404803 24803 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:01:42.404987 24732 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:42.405278 19981 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:42.407495 24676 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.407877 24676 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.408259 24676 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.359135 24622 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:42.416374 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.417366 20856 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.418251 20009 raft_consensus.cc:2368] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16.
I20260709 13:01:42.421314 20857 raft_consensus.cc:2393] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 15: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:42.392256 20744 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:42.392254 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:42.430236 20160 raft_consensus.cc:2410] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 13:01:42.431416 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 84043.1us
I20260709 13:01:42.440249 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:42.440454 23958 raft_consensus.cc:2749] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607) with higher term. Message: Invalid argument: T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for earlier term 11. Current term is 12.
I20260709 13:01:42.443447 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:42.443624 20158 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
W20260709 13:01:42.460016 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 67827.8us
I20260709 13:01:42.461566 20129 leader_election.cc:400] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13.
I20260709 13:01:42.461946 20129 leader_election.cc:403] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:42.462010 20744 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.462754 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:42.464718 20856 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:42.476204 24677 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 12. Current term is 13.
I20260709 13:01:42.476464 24626 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:42.485798 20856 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:42.487165 24661 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:42.488914 24603 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:42.491003 24603 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:42.514781 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 50716.4us
I20260709 13:01:42.520521 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:42.525005 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:42.521433 20010 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
I20260709 13:01:42.500798 24622 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:42.545120 20087 raft_consensus.cc:1275] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:01:42.530644 24576 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:01:42.538172 24749 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:42.544546 24633 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:42.500167 24544 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:42.546253 20087 pending_rounds.cc:85] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Aborting all ops after (but not including) 5
I20260709 13:01:42.561483 20087 pending_rounds.cc:107] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:01:42.561898 20087 raft_consensus.cc:2889] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.546559 20010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
W20260709 13:01:42.547029 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 98990.4us
I20260709 13:01:42.487586 24674 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.551970 20012 raft_consensus.cc:1240] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:42.565232 20010 raft_consensus.cc:606] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:42.563546 19982 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.550968 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:42.547823 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.562484 20743 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:42.574595 20082 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:01:42.575454 20128 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.578114 24674 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.578966 20084 raft_consensus.cc:1240] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:42.547713 24749 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:42.581391 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.582158 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:42.585264 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.582106 20851 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:42.586985 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.561328 24544 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:42.584656 20160 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
W20260709 13:01:42.622769 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 267657us
I20260709 13:01:42.623987 24685 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:42.624127 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.624591 24685 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.625417 24783 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:42.625799 24783 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:42.626147 24783 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.626781 24685 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:42.627365 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 55090us
I20260709 13:01:42.629998 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:42.630245 19981 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:42.630790 20850 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:42.632005 20012 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:42.631471 19835 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 9 to 14, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 14 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.631474 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:42.633035 20084 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:42.630795 19838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32772 (request call id 153) took 15630 ms (client timeout 15000 ms). Trace:
I20260709 13:01:42.636247 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:42.642297 24242 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:01:42.634092 19838 rpcz_store.cc:269] 0709 13:01:27.000640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.003759 (+  3119us) service_pool.cc:224] Handling call
0709 13:01:42.630695 (+15626936us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:38.569625 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:38.624378 (+ 54753us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:38.624423 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:01:38.624439 (+    16us) tablet.cc:662] Decoding operations
0709 13:01:38.626189 (+  1750us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:38.626249 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:38.626273 (+    24us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:01:38.762256 (+135983us) tablet.cc:701] Row locks acquired
0709 13:01:38.762277 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:01:38.762446 (+   169us) write_op.cc:270] Start()
0709 13:01:38.762595 (+   149us) write_op.cc:276] Timestamp: P: 1783602098762401 usec, L: 0
0709 13:01:38.762611 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:01:38.844420 (+ 81809us) log.cc:844] Serialized 10544 byte log entry
0709 13:01:39.387041 (+542621us) op_driver.cc:464] REPLICATION: finished
0709 13:01:39.962797 (+575756us) write_op.cc:301] APPLY: starting
0709 13:01:39.962903 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:40.418105 (+455202us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:40.418233 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:40.855602 (+437369us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:40.898582 (+ 42980us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 13:01:40.915653 (+ 17071us) write_op.cc:312] APPLY: finished
0709 13:01:41.014714 (+ 99061us) log.cc:844] Serialized 235 byte log entry
0709 13:01:41.303296 (+288582us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:41.330062 (+ 26766us) write_op.cc:454] Released schema lock
0709 13:01:41.336438 (+  6376us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:41.336999 (+   561us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:41.337039 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1613253,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":575438,"apply.run_cpu_time_us":90990,"apply.run_wall_time_us":1374172,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116879,"num_ops":27,"prepare.queue_time_us":137516,"prepare.run_cpu_time_us":70043,"prepare.run_wall_time_us":660234,"raft.queue_time_us":87090,"raft.run_cpu_time_us":62261,"raft.run_wall_time_us":407188,"replication_time_us":624218,"thread_start_us":1100500,"threads_started":5,"wal-append.queue_time_us":770340,"wal-append.run_cpu_time_us":8996,"wal-append.run_wall_time_us":2156728}]]}
I20260709 13:01:42.637264 19835 catalog_manager.cc:5697] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.645735 19835 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 10 to 13, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:42.570493 24589 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:42.547842 24676 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.573124 24428 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:42.648705 20130 leader_election.cc:400] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16.
W20260709 13:01:42.652343 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 65408.1us
I20260709 13:01:42.657364 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.652235 19835 catalog_manager.cc:5697] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 4 to 9, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:42.640678 24783 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.680459 20160 raft_consensus.cc:3060] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.581003 24595 raft_consensus.cc:2749] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:42.651257 20130 leader_election.cc:403] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:42.685465 20012 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:42.687402 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.623252 24544 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:42.650715 24589 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.647575 24680 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.693373 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:42.693993 20158 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:42.697314 19835 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 14 to 15, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 15 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:42.592168 24681 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 13:01:42.621984 24576 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:01:42.680205 24501 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.623483 24674 consensus_queue.cc:1059] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:01:42.684854 24660 consensus_queue.cc:1048] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:01:42.708717 19835 catalog_manager.cc:5697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.711329 19835 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 8 to 11, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 11 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:42.705813 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a42b8897c4224305af65104f541be1fc" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.704523 20164 raft_consensus.cc:2410] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:01:42.691433 24589 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:42.693292 24671 raft_consensus.cc:3060] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:01:42.719233 24810 log.cc:927] Time spent T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:01:42.695597 24680 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.720991 24680 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.724562 24589 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:42.727769 24589 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:42.731745 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.736446 20160 raft_consensus.cc:2468] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:42.737535 20084 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:42.743664 24676 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:42.686252 24563 consensus_queue.cc:1059] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:01:42.748500 19835 catalog_manager.cc:5697] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.750435 19835 catalog_manager.cc:5697] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 3 to 13. New cstate: current_term: 13 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:42.750862 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:42.751503 24697 consensus_meta.cc:287] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:01:42.748551 24625 raft_consensus.cc:3055] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:42.752568 24625 raft_consensus.cc:740] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:42.753383 24625 consensus_queue.cc:260] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:42.754611 24625 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.756260 19835 catalog_manager.cc:5697] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 8 to 12, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 12 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.758109 19835 catalog_manager.cc:5697] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 10 to 13, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 13 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.759001 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.760711 20162 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:42.768756 19835 catalog_manager.cc:5697] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 4 to 5, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 5 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:42.773023 19835 catalog_manager.cc:5697] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:42.776368 24633 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:42.783154 19835 catalog_manager.cc:5697] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 6 to 12, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 12 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.785246 19835 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 8 to 11, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 11 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.788631 19835 catalog_manager.cc:5697] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:42.790096 19835 catalog_manager.cc:5697] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 6 to 9, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 9 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.794050 20851 raft_consensus.cc:1275] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:42.796052 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.797076 20856 raft_consensus.cc:2393] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 19: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:42.810671 20743 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.815730 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.816849 20851 raft_consensus.cc:2393] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 14: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
W20260709 13:01:42.821375 24643 consensus_meta.cc:287] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.001s
I20260709 13:01:42.828972 20164 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:42.840564 20856 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:42.841933 20162 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 17.
I20260709 13:01:42.843595 20056 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:42.849903 24617 raft_consensus.cc:2804] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:42.849686 24633 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:01:42.880848 24202 raft_consensus.cc:2804] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:42.884517 24395 raft_consensus.cc:3055] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:42.885064 24395 raft_consensus.cc:740] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:42.886690 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 238173us
I20260709 13:01:42.887305 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:42.891084 24202 raft_consensus.cc:697] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:42.892203 24202 consensus_queue.cc:237] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.905910 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "313fdc28d1e44d6393c8388c74843a28" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:42.906533 20009 raft_consensus.cc:3060] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:42.907393 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:42.908022 20160 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.877296 20084 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:42.909946 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:42.847854 24654 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:01:42.886258 24395 consensus_queue.cc:260] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:42.918392 19981 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:42.914587 20011 raft_consensus.cc:2468] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 19.
I20260709 13:01:42.924786 20014 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:01:42.926360 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 70658us
I20260709 13:01:42.929886 20160 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:42.933449 20056 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.946372 20009 raft_consensus.cc:2468] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 17.
I20260709 13:01:42.951040 20009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81fd99f42a47497f9d4372157320611b"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:42.954875 20009 raft_consensus.cc:606] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:42.948076 24664 raft_consensus.cc:2804] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:01:42.894336 24671 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003) with higher term. Message: Invalid argument: T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 15. Current term is 16.
I20260709 13:01:42.955871 20009 raft_consensus.cc:614] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:42.920679 24617 raft_consensus.cc:697] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:42.957607 19982 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:42.898316 24680 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:42.954670 24578 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:01:42.943117 24624 raft_consensus.cc:2804] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:43.012807 24624 raft_consensus.cc:697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:43.013914 24624 consensus_queue.cc:237] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:43.015967 24680 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.018990 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:43.019619 20859 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.935566 24732 consensus_queue.cc:1048] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:42.994963 24617 consensus_queue.cc:237] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:43.007722 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 50319.9us
I20260709 13:01:42.951205 24395 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.039186 20164 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:01:43.039552 19982 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:43.040112 20164 pending_rounds.cc:85] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Aborting all ops after (but not including) 7
I20260709 13:01:43.040633 20164 pending_rounds.cc:107] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:01:43.040993 20164 raft_consensus.cc:2889] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.992107 24749 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:43.027959 20859 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:43.042286 24739 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.050235 19982 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:43.025635 24626 raft_consensus.cc:2749] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:43.063202 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:43.065868 19981 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:43.067880 20158 raft_consensus.cc:2468] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:43.075659 20009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:43.076434 20009 raft_consensus.cc:606] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:43.076771 20009 raft_consensus.cc:614] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:43.076994 20164 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:01:43.077920 20057 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.079564 24822 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:01:43.079689 24783 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:43.061650 24676 raft_consensus.cc:2804] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:43.043108 24595 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:01:43.088464 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:43.089119 20160 raft_consensus.cc:3060] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.061851 24821 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:01:43.080536 24739 raft_consensus.cc:2804] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:43.108503 24739 raft_consensus.cc:697] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:43.109575 24739 consensus_queue.cc:237] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:43.083753 24583 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:43.111424 24583 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:43.111842 24583 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.100710 24676 raft_consensus.cc:697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:43.117587 24676 consensus_queue.cc:237] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:43.091979 24633 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 13:01:43.121709 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.155131 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:43.100871 20160 raft_consensus.cc:2468] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:43.082490 24664 raft_consensus.cc:697] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 19 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:43.162744 24822 log.cc:927] Time spent T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:01:43.163048 20010 raft_consensus.cc:1275] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:01:43.164263 24823 log.cc:927] Time spent T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:01:43.172830 20010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:43.173882 20010 raft_consensus.cc:606] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:43.179703 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:43.242671 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 63067us
W20260709 13:01:43.250989 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 71108.1us
W20260709 13:01:43.266530 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 100215us
I20260709 13:01:43.266950 20010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:43.267596 20010 raft_consensus.cc:606] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:43.267937 20010 raft_consensus.cc:614] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:43.167819 24544 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:43.254940 24566 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.285887 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:43.289485 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:43.301597 20856 raft_consensus.cc:3060] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.322877 20856 raft_consensus.cc:2468] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:43.326385 24544 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:43.353641 24837 consensus_queue.cc:1048] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:43.302310 24664 consensus_queue.cc:237] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:43.348714 24583 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:43.379297 24583 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.381155 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:43.381984 20859 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 13: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:43.400981 24595 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.401511 24595 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:43.408737 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:43.403553 24595 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:43.482553 24653 raft_consensus.cc:993] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:43.483127 24653 raft_consensus.cc:1081] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:43.486192 24625 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.487883 24625 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.490008 24625 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:43.531421 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:43.532563 20158 raft_consensus.cc:2368] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15.
W20260709 13:01:43.529429 24832 log.cc:927] Time spent T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:43.549686 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:43.551564 20744 leader_election.cc:400] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15.
I20260709 13:01:43.552042 20744 leader_election.cc:403] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:01:43.558863 24664 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:43.559723 24664 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:43.569909 20745 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.572458 24595 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15.
I20260709 13:01:43.574079 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:43.574988 20858 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:43.577718 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:43.562634 24664 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.579788 24732 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:43.582556 24732 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:43.582923 24732 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.596688 24837 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:43.601348 24837 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:43.614960 20857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44218
I20260709 13:01:43.615806 20857 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:43.616412 20857 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.618964 24677 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:43.619555 24677 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.621706 24677 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.628473 20085 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:43.625315 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:43.626039 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:43.629981 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:43.630772 20011 raft_consensus.cc:2410] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:01:43.623458 24589 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.641067 24589 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.649051 20156 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:43.650422 20161 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:43.647562 20158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:43.652878 24783 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.653434 24783 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:43.654376 24688 raft_consensus.cc:993] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:43.672246 24589 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.675806 20857 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:43.700145 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:43.700976 20011 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 17.
I20260709 13:01:43.702185 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:43.702877 20856 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
I20260709 13:01:43.706393 20161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:43.709314 20161 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:43.709602 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:43.710134 20857 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:43.711966 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:43.717748 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:43.611766 24732 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.719620 20165 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:01:43.721760 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:43.725418 20158 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:01:43.671173 24783 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:43.670823 24661 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:43.728909 24633 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:43.740717 24624 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:43.739842 24676 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:43.686719 24812 raft_consensus.cc:1081] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:43.732509 24693 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:43.721539 24692 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 13:01:43.748808 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 53322.5us
I20260709 13:01:43.759181 24693 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:43.760849 20857 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:43.722159 24762 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:43.761586 24693 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:43.763754 20744 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:43.777173 20857 raft_consensus.cc:1275] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:43.772210 24625 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:01:43.761976 24762 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.784811 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:43.785022 20082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:01:43.798391 20082 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:43.798851 20082 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.769029 24830 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:43.804425 24830 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:43.807049 24830 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:43.745097 24661 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.781261 24732 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.794682 20856 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:01:43.827221 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:43.823488 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:43.829145 20851 raft_consensus.cc:2393] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 9: Already voted for candidate e5eb27db496b47129df3e689e546d94a in this term.
I20260709 13:01:43.796257 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:43.793677 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:43.832208 20010 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:43.832566 20010 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:43.816831 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:43.833598 20083 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 16: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:43.820045 24661 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.821209 20159 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 8.
I20260709 13:01:43.837723 24732 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:43.838446 24732 consensus_queue.cc:1243] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:43.808930 24762 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.849999 20082 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:43.854027 20851 pending_rounds.cc:85] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 8
I20260709 13:01:43.854475 20851 pending_rounds.cc:107] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:01:43.854732 20851 raft_consensus.cc:2889] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:43.856922 20857 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:43.858201 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:43.834226 20010 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:43.860091 20010 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.861735 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:43.859115 24625 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.864531 24625 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:43.872836 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:43.875731 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 137829us
I20260709 13:01:43.881453 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:43.871800 24589 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:43.885334 24589 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:43.886336 24589 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.870572 24854 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:01:43.871560 24688 consensus_queue.cc:1048] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:01:43.892825 24643 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:43.893473 24643 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.895625 24643 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:43.898558 24837 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:43.901937 24749 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:43.904749 24625 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:43.905220 24811 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.948185 20743 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:43.949155 20009 raft_consensus.cc:1275] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:01:43.877759 24787 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:43.913786 20163 raft_consensus.cc:1275] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:43.914597 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:43.914762 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:43.950819 20091 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.918659 20159 raft_consensus.cc:1275] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:43.907740 20130 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:43.922931 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:43.952404 20859 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:43.953646 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:43.925333 20010 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:43.957985 20009 raft_consensus.cc:2393] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 17: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:43.923928 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:43.941685 20082 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:43.961196 20017 raft_consensus.cc:1275] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:01:43.906522 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:43.935245 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:43.962476 20160 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
W20260709 13:01:43.959618 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51963.6us
I20260709 13:01:43.956182 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:43.969774 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:43.964726 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:43.973466 20087 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:43.976323 24563 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:43.976776 24563 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.977133 24563 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.951956 24821 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.973886 24762 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:43.983058 20085 raft_consensus.cc:2368] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:01:43.986322 20850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:43.987036 24811 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:43.988027 24811 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:43.988497 20082 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:43.990058 24842 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:43.990741 24842 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:43.991113 24842 raft_consensus.cc:3060] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.987053 20850 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.009497 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:44.009799 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:44.020713 20091 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:44.024696 20165 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
W20260709 13:01:44.035065 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 64490.3us
I20260709 13:01:43.973004 24681 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:44.051210 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:44.051890 20160 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.996380 24762 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:44.057616 20129 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:44.072471 24783 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:44.073184 24856 consensus_queue.cc:1048] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:01:44.075176 20160 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:44.051815 24681 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.079834 24681 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:44.089813 24787 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:44.090536 24787 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.022747 24664 consensus_queue.cc:1048] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:01:44.092396 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:44.092857 24787 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:44.096167 24828 consensus_queue.cc:1048] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:44.034600 24692 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:01:44.057567 24756 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.124605 20850 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:44.133004 20055 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
W20260709 13:01:44.135582 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 170906us
W20260709 13:01:44.122906 24846 log.cc:927] Time spent T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:44.147604 20744 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:44.139531 24664 raft_consensus.cc:2804] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:44.049769 24680 consensus_queue.cc:1048] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:01:44.164022 20850 raft_consensus.cc:1275] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:01:44.174296 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 175695us
I20260709 13:01:44.180033 20129 leader_election.cc:400] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:01:44.180495 20129 leader_election.cc:403] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:44.182605 24784 raft_consensus.cc:3060] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.185736 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:44.186658 20091 raft_consensus.cc:2410] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:01:44.190996 20056 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:44.191839 24589 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:44.192601 24589 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:44.196419 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 141849us
I20260709 13:01:44.198396 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:44.199275 20010 raft_consensus.cc:2393] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:01:44.169598 24664 raft_consensus.cc:697] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:44.173007 24578 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:44.138212 24756 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:44.216274 20163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:44.193734 24589 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.179557 24583 consensus_queue.cc:1048] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 13:01:44.255302 24861 log.cc:927] Time spent T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:01:44.276906 20130 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:44.257618 24578 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:44.286001 24762 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:44.288625 24869 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:01:44.291908 20159 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:01:44.297554 24862 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:01:44.300331 24664 consensus_queue.cc:237] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:44.306579 24563 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:44.309103 24756 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:44.330160 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:44.351239 20010 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
W20260709 13:01:44.358916 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 143597us
I20260709 13:01:44.353502 20130 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:44.358978 24681 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.371250 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:44.376457 20855 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
W20260709 13:01:44.340018 24858 log.cc:927] Time spent T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:01:44.382615 20162 raft_consensus.cc:2368] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 17. Current term is 18.
I20260709 13:01:44.344619 24839 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.377964 24756 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:44.401477 20159 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:44.391292 24563 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:44.393085 24578 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:44.395008 24860 log.cc:927] Time spent T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:44.357704 24784 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831) with higher term. Message: Invalid argument: T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for earlier term 9. Current term is 10.
I20260709 13:01:44.408300 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
W20260709 13:01:44.409219 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 72122.8us
I20260709 13:01:44.410082 20159 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.336447 24762 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:44.422529 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:44.423393 20010 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:44.399756 24756 raft_consensus.cc:493] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.398582 24732 raft_consensus.cc:493] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.434921 20010 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 18.
I20260709 13:01:44.368970 24842 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:44.472649 24681 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:44.409574 24865 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:01:44.433907 24756 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:44.486064 24762 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:01:44.489107 24749 consensus_meta.cc:287] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 13:01:44.494261 20159 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:44.494776 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:44.499666 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:44.499743 20163 raft_consensus.cc:2468] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:01:44.503486 20858 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:44.507220 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:44.441094 24732 raft_consensus.cc:515] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:44.512187 24697 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.526672 24882 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.526816 24697 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:44.527576 24882 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.529820 24697 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:44.530107 24882 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:44.531796 24882 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:44.506633 24863 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.546595 20744 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:44.551573 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:44.552304 20857 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:44.554745 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:44.569039 20008 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:44.569558 20008 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:44.510568 24680 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:01:44.563937 24544 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:44.577035 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 69889.4us
I20260709 13:01:44.567654 24837 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:44.589980 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:44.590749 20009 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:44.597332 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:44.598032 20091 raft_consensus.cc:3060] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:01:44.559556 24863 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:44.602051 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:44.605862 20056 leader_election.cc:400] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 17. Current term is 18.
I20260709 13:01:44.606211 20056 leader_election.cc:403] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:44.606000 20856 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:44.606946 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:44.582369 24837 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.591811 24544 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:44.619066 24544 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:44.593631 24760 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:44.597702 24882 raft_consensus.cc:493] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:44.639032 20091 raft_consensus.cc:2468] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 18.
I20260709 13:01:44.640776 24732 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:44.642031 24882 raft_consensus.cc:515] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:44.644685 24882 leader_election.cc:290] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:44.621690 24837 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:44.660485 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 54821.9us
I20260709 13:01:44.663115 24888 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223) with higher term. Message: Invalid argument: T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for earlier term 17. Current term is 18.
I20260709 13:01:44.663587 24842 leader_election.cc:290] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:44.630838 24760 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.669454 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:44.670321 20858 raft_consensus.cc:2410] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:44.670588 24760 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
W20260709 13:01:44.671890 24859 log.cc:927] Time spent T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.470s	user 0.001s	sys 0.001s
I20260709 13:01:44.672433 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:44.673260 24756 raft_consensus.cc:515] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:44.677064 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:44.661438 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:01:44.686471 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 128971us
I20260709 13:01:44.691764 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:44.692507 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8a881d176a0426b9337a6b7c224b88c" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:44.694073 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77ca1b6b08b74f04948397dece34f882" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:44.694844 20082 raft_consensus.cc:2393] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 12: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:44.695462 20856 raft_consensus.cc:2468] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:01:44.695884 20162 raft_consensus.cc:2410] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:44.687088 24876 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:44.696928 24876 raft_consensus.cc:493] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.697296 24876 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:44.703542 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
W20260709 13:01:44.706789 24864 log.cc:927] Time spent T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.436s	user 0.000s	sys 0.002s
I20260709 13:01:44.712689 24839 raft_consensus.cc:2749] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:44.713198 20129 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:44.713973 20164 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:44.626689 24589 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:01:44.715242 20129 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:44.716121 24879 raft_consensus.cc:2804] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:01:44.716284 24783 raft_consensus.cc:2804] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:44.716892 24879 raft_consensus.cc:697] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:44.718096 24879 consensus_queue.cc:237] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.716948 24783 raft_consensus.cc:697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:44.343480 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:44.730820 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:44.736521 24890 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.737171 24890 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:44.739463 24890 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:44.743901 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:44.751116 24756 leader_election.cc:290] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:44.753655 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:44.754580 20011 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:44.761883 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe4ca3a058c431f8e6ecbfc90468313" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:44.762634 20856 raft_consensus.cc:3060] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:44.777647 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 68823.1us
I20260709 13:01:44.782188 20056 leader_election.cc:304] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:44.788695 24783 consensus_queue.cc:237] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:44.795786 20011 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:01:44.799006 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:44.799461 20856 raft_consensus.cc:2468] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:01:44.803081 20129 leader_election.cc:304] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:44.804836 20056 leader_election.cc:304] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:44.823422 20855 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:44.799052 24664 raft_consensus.cc:2749] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:44.820358 24839 raft_consensus.cc:2804] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:44.820026 24589 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.820631 24732 raft_consensus.cc:2749] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.843158 24839 raft_consensus.cc:697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:44.870141 24839 consensus_queue.cc:237] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:44.872267 20008 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.873682 20008 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:44.879344 24589 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:44.880023 24589 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:44.882145 24589 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:44.883770 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:44.824741 24837 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:44.886987 24863 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:44.887815 24837 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:44.893915 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
W20260709 13:01:44.897431 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 58995.3us
I20260709 13:01:44.898865 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:44.901294 23958 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:44.901670 23958 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:44.901978 23958 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:44.907444 24692 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:44.908339 24692 consensus_queue.cc:1243] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer e5eb27db496b47129df3e689e546d94a log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:44.908782 24692 raft_consensus.cc:3055] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:44.909162 24692 raft_consensus.cc:740] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:44.926505 20008 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:44.946286 20156 raft_consensus.cc:1240] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:44.956698 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 56079.7us
W20260709 13:01:44.941244 24812 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:01:44.847149 24876 raft_consensus.cc:515] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:44.963369 23958 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:44.987639 23958 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:44.909926 24692 consensus_queue.cc:260] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:45.168901 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 180910us
W20260709 13:01:45.294378 24896 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:01:45.345173 24894 log.cc:927] Time spent T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a: Append to log took a long time: real 0.422s	user 0.002s	sys 0.000s
W20260709 13:01:45.364975 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 184496us
I20260709 13:01:45.366560 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
W20260709 13:01:45.380025 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 392834us
I20260709 13:01:45.381188 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d2ab233d0664594955baf948686386b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:45.383265 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:45.383965 20011 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 16.
I20260709 13:01:45.392537 20851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:45.409768 20156 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:45.387283 24891 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:45.383636 24900 consensus_queue.cc:1059] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:01:45.411201 24876 leader_election.cc:290] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:45.414867 24589 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:45.454583 20851 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:45.444196 24603 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:45.428857 24891 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:45.461632 24881 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:45.466022 20087 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:45.467156 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:45.480813 20082 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 16: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:01:45.446345 24589 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:45.489960 20859 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:45.475991 24603 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:45.518378 24603 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:45.482870 24881 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:45.579813 24912 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:45.587070 24589 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:45.589064 24891 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:45.592103 24881 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:45.594946 24903 log.cc:927] Time spent T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.326s	user 0.001s	sys 0.002s
W20260709 13:01:45.690723 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 270007us
I20260709 13:01:45.694129 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:45.717882 20165 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50786 (request call id 1601) took 1450 ms. Trace:
I20260709 13:01:45.718580 20165 rpcz_store.cc:276] 0709 13:01:44.267708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.268087 (+   379us) service_pool.cc:224] Handling call
0709 13:01:44.268458 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.268779 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.268806 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.412228 (+1143422us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b2c001e7048. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa02d0aba11 00007fa02d0bb372 00007fa02e7eda01 00007fa02e7b243a 00007fa02e7b183e 00007fa033776419 00007fa02ccedbd4 00007fa02ccedb62 00007fa02cceda91 00007fa02cceda0c 00007fa02cceccc2
0709 13:01:45.412775 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.573333 (+160558us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:45.573389 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.573511 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.716994 (+143483us) raft_consensus.cc:1685] finished
0709 13:01:45.717053 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.717553 (+   500us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17425,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":6371200,"thread_start_us":159974,"threads_started":1,"wal-append.queue_time_us":188034}
I20260709 13:01:45.720876 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:45.732786 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:45.733292 20017 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:01:45.744830 20850 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:45.760397 24685 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:45.761533 24685 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:45.764225 24749 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:45.765416 24749 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:45.768596 24923 raft_consensus.cc:493] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8b5b4e2a92b046d096f2661c54321d41)
I20260709 13:01:45.769749 24923 raft_consensus.cc:515] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:45.776162 24760 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:45.777256 24760 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:45.782121 20856 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:45.785995 20009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34542 (request call id 1289) took 1006 ms. Trace:
I20260709 13:01:45.794962 24820 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:45.800815 24920 raft_consensus.cc:493] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:45.802109 24920 raft_consensus.cc:515] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:45.805586 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:45.807126 20017 raft_consensus.cc:1275] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:45.807580 24679 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:01:45.808204 24811 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:45.809091 24811 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:45.810721 24676 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.811543 24676 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:45.815114 24676 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:45.820430 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:45.771986 24685 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:45.831086 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 56197.1us
I20260709 13:01:45.832823 19981 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:45.836455 24812 raft_consensus.cc:2749] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:45.840533 20017 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:45.796353 24820 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:45.831218 24692 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.861351 24867 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:45.853124 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:45.864748 20011 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:45.864785 20082 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 16.
I20260709 13:01:45.875663 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:45.876156 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:45.876603 20091 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:45.904210 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:45.904835 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4864bdb85f20457e9ef1bfb12f991554" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:45.905973 20165 raft_consensus.cc:3060] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:45.925493 24749 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:45.946216 24923 leader_election.cc:290] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:45.955536 24820 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:45.956302 20014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:45.786482 20009 rpcz_store.cc:276] 0709 13:01:44.779276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.779433 (+   157us) service_pool.cc:224] Handling call
0709 13:01:44.779747 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.780181 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:44.803905 (+ 23724us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.705509 (+901604us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b1800435e20. stack: 00007fa02b693343 00007fa02b693240 00007fa02b0221d4 00007fa034d60304 00007fa034d60299 00007fa03458dde1 00007fa03459a5b1 00007fa02e35a4bc 00007fa02b6a848b 00007fa034502ab9 00007fa02b6c469b 00007fa02b6c293f 00007fa02b6c4c02 00007fa02e34c20b 00007fa02e351d23 00007fa02e35223a
0709 13:01:45.705951 (+   442us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.707322 (+  1371us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.707622 (+   300us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:45.715559 (+  7937us) log.cc:844] Serialized 14 byte log entry
0709 13:01:45.715934 (+   375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:45.716081 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.785463 (+ 69382us) raft_consensus.cc:1685] finished
0709 13:01:45.785578 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.785764 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65964,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1321,"raft.queue_time_us":85112,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2533,"spinlock_wait_cycles":11056000,"thread_start_us":28174,"threads_started":1,"wal-append.queue_time_us":34847}
I20260709 13:01:45.959697 20156 raft_consensus.cc:1240] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:01:45.950551 24811 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:45.969583 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:45.979772 20010 raft_consensus.cc:2410] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:01:45.983073 24878 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:45.983983 24878 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:45.987215 24878 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:45.991659 24837 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:01:45.994299 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 46293.4us
I20260709 13:01:45.983652 20165 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:45.997041 20017 raft_consensus.cc:2468] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:01:45.999991 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.001312 24912 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:46.003558 24912 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:46.003978 24920 leader_election.cc:290] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:46.004350 24760 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:46.006201 20165 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:46.007241 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:46.017881 24879 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:46.023785 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 359675us
I20260709 13:01:46.030203 20854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.030516 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.030617 24681 consensus_queue.cc:1048] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:01:46.031982 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 52508.2us
I20260709 13:01:46.036638 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.034215 20744 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:46.040875 24863 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:46.044262 20744 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:46.045215 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d767704576e43bb9f6bcd864fa9a6fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
W20260709 13:01:46.049873 19835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35376 (request call id 99) took 19051 ms (client timeout 15000 ms). Trace:
I20260709 13:01:46.044271 20056 leader_election.cc:304] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:46.062446 24922 raft_consensus.cc:2804] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:46.063647 24922 raft_consensus.cc:697] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:46.065148 24922 consensus_queue.cc:237] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:46.071290 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:46.084698 24681 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.095726 24863 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:46.138586 20087 raft_consensus.cc:1275] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:46.125195 24863 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.153183 20091 raft_consensus.cc:1275] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:01:46.164465 24865 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:46.165135 24865 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:46.165557 24704 raft_consensus.cc:493] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:46.166466 24704 raft_consensus.cc:515] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:46.167145 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 75419.2us
W20260709 13:01:46.168110 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 48136.5us
I20260709 13:01:46.173815 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.175697 20055 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.183480 24922 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:46.156430 24863 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:46.183826 24912 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.197815 24922 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:46.208243 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:46.207998 24922 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:46.215983 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.217037 24704 leader_election.cc:290] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:46.224538 24865 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:46.224363 24760 consensus_queue.cc:1059] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:46.146775 24681 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:46.050498 19835 rpcz_store.cc:269] 0709 13:01:26.998640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.998825 (+   185us) service_pool.cc:224] Handling call
0709 13:01:46.049579 (+19050754us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:43.342920 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:43.356173 (+ 13253us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:43.356212 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:01:43.356232 (+    20us) tablet.cc:662] Decoding operations
0709 13:01:43.357798 (+  1566us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:43.357872 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:43.357903 (+    31us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:01:43.360510 (+  2607us) tablet.cc:701] Row locks acquired
0709 13:01:43.360526 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:01:43.360680 (+   154us) write_op.cc:270] Start()
0709 13:01:43.360794 (+   114us) write_op.cc:276] Timestamp: P: 1783602103360629 usec, L: 0
0709 13:01:43.360811 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:01:43.361577 (+   766us) log.cc:844] Serialized 6694 byte log entry
0709 13:01:43.673795 (+312218us) op_driver.cc:464] REPLICATION: finished
0709 13:01:43.777259 (+103464us) write_op.cc:301] APPLY: starting
0709 13:01:43.777353 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:43.945122 (+167769us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:43.945218 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:44.254642 (+309424us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:44.310368 (+ 55726us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:01:44.310429 (+    61us) write_op.cc:312] APPLY: finished
0709 13:01:44.310879 (+   450us) log.cc:844] Serialized 155 byte log entry
0709 13:01:44.311812 (+   933us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:44.312017 (+   205us) write_op.cc:454] Released schema lock
0709 13:01:44.312083 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:44.339667 (+ 27584us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:44.346058 (+  6391us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1864303,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":103200,"apply.run_cpu_time_us":27005,"apply.run_wall_time_us":546019,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75119,"num_ops":17,"prepare.queue_time_us":259798,"prepare.run_cpu_time_us":20438,"prepare.run_wall_time_us":131777,"raft.queue_time_us":180378,"raft.run_cpu_time_us":30705,"raft.run_wall_time_us":80274,"replication_time_us":312775,"thread_start_us":467264,"threads_started":4,"wal-append.queue_time_us":104712,"wal-append.run_cpu_time_us":79629,"wal-append.run_wall_time_us":1855711}]]}
W20260709 13:01:46.241693 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 66125.2us
I20260709 13:01:46.253258 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.257767 24681 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:46.261732 20130 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.263639 24783 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:46.269982 24922 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:01:46.279318 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 48706.2us
I20260709 13:01:46.299010 20009 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:46.296277 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.301009 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
W20260709 13:01:46.303882 20744 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.30.66:39003 timed out after 1.665s (SENT) [suppressed 2 similar messages]
I20260709 13:01:46.224529 19837 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:46.337038 20009 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:01:46.340498 24911 raft_consensus.cc:493] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:46.341490 24911 raft_consensus.cc:515] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:46.337814 20009 pending_rounds.cc:85] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 8
I20260709 13:01:46.360277 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb22f25deefa4dce94a6ca7ffecaee7c" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.362787 24891 raft_consensus.cc:493] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:46.363436 24891 raft_consensus.cc:515] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:46.366253 24891 leader_election.cc:290] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:46.333143 24922 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:46.358206 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.347821 20009 pending_rounds.cc:107] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:01:46.420892 20859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.419832 20009 raft_consensus.cc:2889] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:46.433358 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.427255 20008 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:01:46.436131 24911 leader_election.cc:290] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:46.436060 24828 raft_consensus.cc:3055] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:46.422394 24923 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:46.429569 19837 catalog_manager.cc:5697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:46.474066 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:46.461637 24760 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:01:46.434192 24692 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.488909 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.495733 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e04fb0be94cccad001ccc48f9bd85" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:46.500257 20163 raft_consensus.cc:1275] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:46.500836 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:46.502048 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88eccd78839140bba038b898679a00b0" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.469512 24923 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:46.453087 24828 raft_consensus.cc:740] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
W20260709 13:01:46.509240 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 70436.9us
I20260709 13:01:46.514828 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:46.515578 20091 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:46.518455 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:46.538687 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.541772 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.492183 24692 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:46.478878 24760 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:46.552524 24923 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:46.552825 24633 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:46.553572 24633 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:46.554787 24760 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:46.530748 20130 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:46.522603 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.435497 24922 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:46.573414 24692 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:46.574034 24878 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:46.574502 24878 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.574872 24878 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:46.574944 24922 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:46.580309 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.551272 24863 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:46.608012 24633 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:46.608950 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:46.609434 24828 consensus_queue.cc:260] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:01:46.611195 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 88714.5us
I20260709 13:01:46.612610 24913 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:46.596786 24692 raft_consensus.cc:493] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:46.616169 24692 raft_consensus.cc:515] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:46.618635 24692 leader_election.cc:290] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:46.627163 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:01:46.643501 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 339656us
I20260709 13:01:46.674990 24863 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:46.700253 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.705650 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.723126 24908 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:46.703011 24828 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.720233 19981 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.716383 20008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.724037 24908 raft_consensus.cc:493] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.728295 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.720108 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:46.698557 24891 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:01:46.737016 24685 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:46.738058 24908 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:46.794683 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 119352us
W20260709 13:01:46.799755 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 145559us
W20260709 13:01:46.826184 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 74044.6us
W20260709 13:01:46.836103 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 141234us
I20260709 13:01:46.830055 20744 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
W20260709 13:01:46.843561 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 83543.1us
I20260709 13:01:46.840263 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:46.847908 24820 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:46.860715 19981 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:01:46.867477 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 161602us
I20260709 13:01:46.869082 20743 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:46.872901 24863 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:46.874689 24939 log.cc:927] Time spent T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:01:46.883091 24956 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:46.883702 24956 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:46.885278 24951 consensus_queue.cc:1048] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:46.886247 24956 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:46.847041 24948 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:46.914002 20083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f761e6c8cc73410eb153cfbcb79afb90" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:01:46.928203 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 67612.8us
I20260709 13:01:46.931236 20857 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:46.933261 24920 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:46.939105 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e1c1a13f585422ea6d2daf771874d19" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:46.939754 24878 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:46.940661 24908 raft_consensus.cc:515] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:46.952702 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:46.958474 24908 leader_election.cc:290] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:46.963774 20743 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:46.964074 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:46.964389 20088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7bc64cbbc04798be25ec85d5de3b0d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:46.965036 24891 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:46.967350 24891 raft_consensus.cc:2749] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:46.970516 24920 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:01:46.971240 24920 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:46.980652 19981 leader_election.cc:304] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.989291 19981 leader_election.cc:304] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:46.988855 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:46.993304 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:46.995836 20009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5e6c5a68d4b07b3d5830de6d5be23" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:46.997931 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:46.995549 24927 raft_consensus.cc:2749] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.004698 20088 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:46.986348 24865 raft_consensus.cc:493] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.012892 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:47.009351 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:47.009778 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:47.013670 20091 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:47.015093 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:47.016332 24920 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:47.008509 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ea05922db6473289d3a25f7b32e86e" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:47.018059 20089 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.010540 20084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d924541574d6086a6ae086476a1e7" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:47.012629 24865 raft_consensus.cc:515] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:47.010041 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:47.029870 20016 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:47.001843 24692 raft_consensus.cc:2749] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:01:47.033398 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 183813us
I20260709 13:01:47.034492 20056 leader_election.cc:304] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.038108 20160 raft_consensus.cc:1275] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:01:47.040414 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:47.046505 20091 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 17.
I20260709 13:01:47.053756 24878 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:47.055135 24918 consensus_queue.cc:1059] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:01:47.043740 24922 raft_consensus.cc:2749] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.058861 24912 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:47.057981 24865 leader_election.cc:290] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:47.072366 20089 raft_consensus.cc:2468] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 15.
I20260709 13:01:47.080435 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:47.088706 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:47.089061 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.074641 24867 consensus_queue.cc:1048] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:01:47.095562 24956 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.096426 20853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:47.098078 24956 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:47.101002 20082 raft_consensus.cc:1240] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Rejecting Update request from peer 48da76e971b84219a6ba80d1ba83b7eb for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260709 13:01:47.082425 24927 raft_consensus.cc:3055] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:47.104147 24927 raft_consensus.cc:740] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:47.105058 24927 consensus_queue.cc:260] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:47.106061 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 257223us
I20260709 13:01:47.106359 24927 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:47.089234 24922 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:01:47.115726 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:47.113118 24879 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.124665 20017 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:47.124480 24879 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:47.126883 24879 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:47.138343 20011 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:01:47.160740 19981 leader_election.cc:304] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.162274 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.164687 24920 raft_consensus.cc:2804] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:47.165287 24920 raft_consensus.cc:697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:47.166181 24920 consensus_queue.cc:237] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:47.168226 24953 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:01:47.178195 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 96733.2us
I20260709 13:01:47.200052 20855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e5eb27db496b47129df3e689e546d94a" is_pre_election: true
I20260709 13:01:47.201432 24882 consensus_queue.cc:1048] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:47.197867 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.218292 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.223033 20056 leader_election.cc:304] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:47.225628 24965 raft_consensus.cc:2749] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:47.225796 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45335.7us
I20260709 13:01:47.228091 24820 consensus_queue.cc:1059] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:47.228981 24820 raft_consensus.cc:3055] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:47.229334 24820 raft_consensus.cc:740] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:47.230268 24820 consensus_queue.cc:260] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:47.231499 24820 raft_consensus.cc:3060] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.217016 24760 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:47.247725 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.250280 20056 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:47.253545 24911 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.236120 24912 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.262456 24760 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:47.266592 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:47.270902 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.275250 24912 raft_consensus.cc:2749] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:47.273437 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:47.277853 20743 leader_election.cc:304] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:47.279428 24911 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:47.281176 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:47.282702 20160 raft_consensus.cc:1240] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Rejecting Update request from peer 41af8b41f8a348c5b60ec12d2e323698 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:01:47.280107 24965 consensus_queue.cc:1048] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:01:47.280551 20010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7296fb4f71a34242af6ae5071d9f508e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:47.293951 20089 raft_consensus.cc:1240] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Rejecting Update request from peer e5eb27db496b47129df3e689e546d94a for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:01:47.273391 24882 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.308990 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:47.316684 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6f643e855f54a4d8def1836bbdcf39b" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:47.318943 20743 leader_election.cc:304] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5eb27db496b47129df3e689e546d94a; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:47.299064 24891 raft_consensus.cc:2749] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:01:47.348140 24967 log.cc:927] Time spent T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: Append to log took a long time: real 0.084s	user 0.002s	sys 0.002s
I20260709 13:01:47.352257 24820 raft_consensus.cc:2749] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.353716 20743 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:47.359613 24863 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:47.368824 24863 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 17 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:47.369830 24863 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:01:47.373078 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 55030.1us
I20260709 13:01:47.378722 20056 leader_election.cc:304] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:47.379980 24912 consensus_queue.cc:1059] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:47.380990 24912 raft_consensus.cc:3055] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:47.381386 24912 raft_consensus.cc:740] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:47.382303 24912 consensus_queue.cc:260] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:47.402573 24912 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.398147 24828 raft_consensus.cc:2749] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:01:47.444643 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:47.447118 24915 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:47.448423 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.450193 24915 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.489022 20015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:47.489849 20015 raft_consensus.cc:606] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:47.490178 20015 raft_consensus.cc:614] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:47.545199 20015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:47.551203 20015 raft_consensus.cc:606] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:01:47.554486 19837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32774 (request call id 154) took 20290 ms (client timeout 15000 ms). Trace:
W20260709 13:01:47.554980 19837 rpcz_store.cc:269] 0709 13:01:27.264538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.267311 (+  2773us) service_pool.cc:224] Handling call
0709 13:01:47.554404 (+20287093us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:46.852484 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:46.903960 (+ 51476us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:46.904030 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:01:46.904057 (+    27us) tablet.cc:662] Decoding operations
0709 13:01:46.904912 (+   855us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:46.905021 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:46.905065 (+    44us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:01:46.905734 (+   669us) tablet.cc:701] Row locks acquired
0709 13:01:46.905785 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:01:46.906120 (+   335us) write_op.cc:270] Start()
0709 13:01:46.906343 (+   223us) write_op.cc:276] Timestamp: P: 1783602106906025 usec, L: 0
0709 13:01:46.906484 (+   141us) op_driver.cc:348] REPLICATION: starting
0709 13:01:46.907658 (+  1174us) log.cc:844] Serialized 922 byte log entry
0709 13:01:47.353936 (+446278us) op_driver.cc:464] REPLICATION: finished
0709 13:01:47.445575 (+ 91639us) write_op.cc:301] APPLY: starting
0709 13:01:47.445696 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:47.453306 (+  7610us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:47.453448 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:47.455136 (+  1688us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:47.455637 (+   501us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:01:47.455708 (+    71us) write_op.cc:312] APPLY: finished
0709 13:01:47.476013 (+ 20305us) log.cc:844] Serialized 32 byte log entry
0709 13:01:47.477120 (+  1107us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:47.477222 (+   102us) write_op.cc:454] Released schema lock
0709 13:01:47.477286 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:47.504237 (+ 26951us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:47.504270 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1105050,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":86133,"apply.run_cpu_time_us":6439,"apply.run_wall_time_us":58629,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12479,"num_ops":2,"prepare.queue_time_us":307620,"prepare.run_cpu_time_us":17418,"prepare.run_wall_time_us":242972,"raft.queue_time_us":179093,"raft.run_cpu_time_us":3259,"raft.run_wall_time_us":75075,"replication_time_us":447256,"thread_start_us":445075,"threads_started":4,"wal-append.queue_time_us":244628}]]}
I20260709 13:01:47.575778 20164 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:01:47.600216 20164 raft_consensus.cc:606] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:47.569464 19836 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 8 to 9, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 9 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:47.613235 24863 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:47.618760 24863 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:47.627960 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:47.626956 19836 catalog_manager.cc:5697] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:47.628717 20091 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:47.629786 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:47.639321 24969 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:47.639837 24969 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:47.640450 24969 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.646767 19836 catalog_manager.cc:5697] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.654419 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:47.655423 20164 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:47.657205 24863 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:47.681417 19836 catalog_manager.cc:5697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 5 to 9, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 9 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.691893 24969 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:47.695120 20089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:47.696511 24969 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:47.699321 20164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:47.700160 20164 raft_consensus.cc:2393] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e5eb27db496b47129df3e689e546d94a in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:47.705389 20089 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 15.
I20260709 13:01:47.706854 20743 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:47.707765 24578 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:47.708863 24578 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:47.709807 24578 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:47.729705 20855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:34112
I20260709 13:01:47.730418 20855 raft_consensus.cc:606] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:47.734830 20015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81fd99f42a47497f9d4372157320611b"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:47.735502 20015 raft_consensus.cc:606] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:47.735872 20015 raft_consensus.cc:614] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:47.741250 20015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:47.741851 20015 raft_consensus.cc:606] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:47.742166 20015 raft_consensus.cc:614] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:47.749262 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "8b5b4e2a92b046d096f2661c54321d41"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:47.749951 20017 raft_consensus.cc:606] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Received request to transfer leadership to TS 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:01:47.763989 20089 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:47.770846 24918 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:47.772068 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:01:47.778822 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:47.803825 20855 raft_consensus.cc:1275] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:01:47.804317 20855 pending_rounds.cc:85] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Aborting all ops after (but not including) 9
I20260709 13:01:47.804535 20855 pending_rounds.cc:107] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:01:47.804780 20855 raft_consensus.cc:2889] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:47.807751 24927 consensus_queue.cc:1048] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:47.821996 19838 catalog_manager.cc:5697] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:47.826859 19838 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 15, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 15 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
W20260709 13:01:47.827770 19836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 160) took 31654 ms (client timeout 15000 ms). Trace:
W20260709 13:01:47.831092 19836 rpcz_store.cc:269] 0709 13:01:16.173177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.173942 (+   765us) service_pool.cc:224] Handling call
0709 13:01:47.827681 (+31653739us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:47.717338 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:47.717935 (+   597us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:47.717974 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:01:47.717993 (+    19us) tablet.cc:662] Decoding operations
0709 13:01:47.718626 (+   633us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:47.718704 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:47.718734 (+    30us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:01:47.719138 (+   404us) tablet.cc:701] Row locks acquired
0709 13:01:47.719161 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:01:47.719337 (+   176us) write_op.cc:270] Start()
0709 13:01:47.719454 (+   117us) write_op.cc:276] Timestamp: P: 1783602107719288 usec, L: 0
0709 13:01:47.719477 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:01:47.720202 (+   725us) log.cc:844] Serialized 1693 byte log entry
0709 13:01:47.723768 (+  3566us) op_driver.cc:464] REPLICATION: finished
0709 13:01:47.724233 (+   465us) write_op.cc:301] APPLY: starting
0709 13:01:47.724322 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:47.726927 (+  2605us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:47.727050 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:47.734409 (+  7359us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:47.734611 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:01:47.734665 (+    54us) write_op.cc:312] APPLY: finished
0709 13:01:47.738808 (+  4143us) log.cc:844] Serialized 48 byte log entry
0709 13:01:47.738984 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:47.739082 (+    98us) write_op.cc:454] Released schema lock
0709 13:01:47.739126 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:47.739550 (+   424us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:47.739578 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":577912,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":252,"apply.run_cpu_time_us":6986,"apply.run_wall_time_us":15254,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16704,"num_ops":4,"prepare.queue_time_us":2654,"prepare.run_cpu_time_us":3178,"prepare.run_wall_time_us":3185,"replication_time_us":4147,"thread_start_us":2261,"threads_started":1}]]}
I20260709 13:01:47.836587 19838 catalog_manager.cc:5697] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 15, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 15 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.844691 19838 catalog_manager.cc:5697] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 12, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 12 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:47.846249 19838 catalog_manager.cc:5697] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.848013 19838 catalog_manager.cc:5697] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.850179 19838 catalog_manager.cc:5697] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.854770 19838 catalog_manager.cc:5697] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 15, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 15 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.856658 24969 raft_consensus.cc:993] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:47.857244 24969 raft_consensus.cc:1081] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:47.859833 19838 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 13, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.861909 19838 catalog_manager.cc:5697] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.865482 19838 catalog_manager.cc:5697] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 15 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:47.864375 20091 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:44890
I20260709 13:01:47.867779 20091 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.868198 20091 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:47.870623 19838 catalog_manager.cc:5697] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 14, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 14 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:47.881806 19838 catalog_manager.cc:5697] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.906551 20089 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:01:47.909648 20017 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:01:47.910516 20017 pending_rounds.cc:85] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting all ops after (but not including) 10
I20260709 13:01:47.910847 20017 pending_rounds.cc:107] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:01:47.911103 20017 raft_consensus.cc:2889] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:47.912142 19838 catalog_manager.cc:5697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.912705 24969 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:01:47.923560 24578 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:47.937789 19838 catalog_manager.cc:5697] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 11, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 11 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.939760 19838 catalog_manager.cc:5697] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 13 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 16 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.941437 19838 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 13 to 16, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:47.942966 19838 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 12, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 12 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.944638 19838 catalog_manager.cc:5697] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:47.931586 20091 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:47.948518 24685 raft_consensus.cc:993] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:47.949285 24920 raft_consensus.cc:1081] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:47.951118 24913 raft_consensus.cc:993] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:47.951366 20090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:01:47.951989 20090 raft_consensus.cc:493] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.952410 20090 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.952721 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:47.954180 20160 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:47.951748 24913 raft_consensus.cc:1081] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:47.959331 19838 catalog_manager.cc:5697] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.961079 19838 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:47.962500 19838 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 15, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 15 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:47.967648 20091 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:01:47.968890 20856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7a345503524963ae2cf82630978b5a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:47.967590 20855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44210
I20260709 13:01:47.971076 20856 raft_consensus.cc:3055] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:47.972076 20856 raft_consensus.cc:740] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:47.971707 20855 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:47.973658 20855 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.966775 20160 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 17.
I20260709 13:01:47.972657 20090 raft_consensus.cc:515] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:47.977954 20856 consensus_queue.cc:260] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:47.986011 20852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:47.986662 20852 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.988709 20056 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:47.989645 24912 raft_consensus.cc:2804] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:47.990142 24912 raft_consensus.cc:697] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 17 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:47.991163 24912 consensus_queue.cc:237] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:48.008637 19838 catalog_manager.cc:5697] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 15, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 15 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:48.017971 20090 leader_election.cc:290] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:48.019662 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25b25157ae9c4e9d87d2ff89c2c0c4dd" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:48.020366 20016 raft_consensus.cc:3055] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:48.010671 19838 catalog_manager.cc:5697] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:48.020710 20016 raft_consensus.cc:740] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:48.021644 20016 consensus_queue.cc:260] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:48.023036 20016 raft_consensus.cc:3060] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:48.023708 19838 catalog_manager.cc:5697] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 7 to 14, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 14 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:48.050091 19838 catalog_manager.cc:5697] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 13, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 13 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:48.057232 19838 catalog_manager.cc:5697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 9 to 13, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.058983 19838 catalog_manager.cc:5697] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 8 to 12, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 12 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:48.049002 20856 raft_consensus.cc:3060] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:48.060730 19838 catalog_manager.cc:5697] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 10 to 13, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 13 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.073660 24685 raft_consensus.cc:993] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 8b5b4e2a92b046d096f2661c54321d41 to start an election
I20260709 13:01:48.074321 24685 raft_consensus.cc:1081] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Signalling peer 8b5b4e2a92b046d096f2661c54321d41 to start an election
I20260709 13:01:48.087358 20160 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
 from {username='slave'} at 127.0.0.1:50786
I20260709 13:01:48.088045 20160 raft_consensus.cc:493] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:48.088447 20160 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:48.093290 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 118455us
I20260709 13:01:48.121416 20160 raft_consensus.cc:515] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:48.121879 20165 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:48.164247 20160 leader_election.cc:290] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:48.176067 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:48.176993 20017 raft_consensus.cc:3055] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:48.177389 20017 raft_consensus.cc:740] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:48.178155 20017 consensus_queue.cc:260] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:48.200587 20855 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:48.206933 20856 raft_consensus.cc:2468] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 17.
I20260709 13:01:48.246450 20016 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
I20260709 13:01:48.279670 20056 leader_election.cc:304] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:48.280541 24760 raft_consensus.cc:2804] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:48.281188 24760 raft_consensus.cc:697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:48.282138 24760 consensus_queue.cc:237] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:48.256543 20852 raft_consensus.cc:2468] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
I20260709 13:01:48.264209 20017 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.270495 20855 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:48.271628 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc68b8861f1d4bf09ec026e1ba26cdf6" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:48.286633 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:48.287297 20165 raft_consensus.cc:3055] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:48.287602 20165 raft_consensus.cc:740] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:48.285233 20857 raft_consensus.cc:3060] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.288221 20165 consensus_queue.cc:260] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:48.294553 20165 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.271219 24986 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:01:48.351588 20165 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:48.359077 20017 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:01:48.366238 20857 raft_consensus.cc:2468] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
W20260709 13:01:48.371634 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 88584.3us
W20260709 13:01:48.366179 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 103662us
I20260709 13:01:48.385111 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:48.385812 20016 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.422889 20130 leader_election.cc:304] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:48.425956 24993 raft_consensus.cc:2804] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.426554 24993 raft_consensus.cc:697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:48.427721 24993 consensus_queue.cc:237] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:01:48.392916 20744 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:48.445940 20016 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 13.
I20260709 13:01:48.454998 24995 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:48.455665 24995 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:48.456645 24995 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:01:48.453306 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 60472.9us
W20260709 13:01:48.464938 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 71909.5us
W20260709 13:01:48.466513 20129 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.478499 20164 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:01:48.480139 20091 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:48.484526 24828 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:01:48.510214 24994 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
W20260709 13:01:48.613979 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 83637.3us
I20260709 13:01:48.631850 20852 raft_consensus.cc:1275] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:01:48.715081 24760 consensus_queue.cc:1048] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:48.718763 20016 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:48.725641 24996 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:48.740800 19838 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32772 (request call id 154) took 3758 ms. Trace:
I20260709 13:01:48.740446 19840 catalog_manager.cc:5697] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:48.742997 19840 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 11 to 12, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 12 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:48.746685 19840 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 9 to 11, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 11 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:48.750526 19840 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:48.741266 19838 rpcz_store.cc:276] 0709 13:01:44.982942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.984484 (+  1542us) service_pool.cc:224] Handling call
0709 13:01:48.740713 (+3756229us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:48.124013 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:48.124554 (+   541us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:48.124589 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:01:48.124607 (+    18us) tablet.cc:662] Decoding operations
0709 13:01:48.126713 (+  2106us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:48.126787 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:48.126818 (+    31us) tablet.cc:685] Acquiring locks for 29 operations
0709 13:01:48.130829 (+  4011us) tablet.cc:701] Row locks acquired
0709 13:01:48.130864 (+    35us) write_op.cc:260] PREPARE: finished
0709 13:01:48.131017 (+   153us) write_op.cc:270] Start()
0709 13:01:48.131130 (+   113us) write_op.cc:276] Timestamp: P: 1783602108130972 usec, L: 0
0709 13:01:48.131154 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:01:48.132100 (+   946us) log.cc:844] Serialized 11318 byte log entry
0709 13:01:48.136097 (+  3997us) op_driver.cc:464] REPLICATION: finished
0709 13:01:48.136606 (+   509us) write_op.cc:301] APPLY: starting
0709 13:01:48.136698 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:48.319007 (+182309us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:48.319115 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:48.476644 (+157529us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:48.477027 (+   383us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 13:01:48.477076 (+    49us) write_op.cc:312] APPLY: finished
0709 13:01:48.477591 (+   515us) log.cc:844] Serialized 251 byte log entry
0709 13:01:48.496197 (+ 18606us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:48.512092 (+ 15895us) write_op.cc:454] Released schema lock
0709 13:01:48.524805 (+ 12713us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:48.525282 (+   477us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:48.525317 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":228809,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":282,"apply.run_cpu_time_us":42399,"apply.run_wall_time_us":388597,"cfile_cache_hit":116,"cfile_cache_hit_bytes":121104,"num_ops":29,"prepare.queue_time_us":211,"prepare.run_cpu_time_us":7097,"prepare.run_wall_time_us":36108,"replication_time_us":4809}]]}
I20260709 13:01:48.780082 20852 raft_consensus.cc:1275] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:48.781713 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:48.785789 24927 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:48.786370 24927 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:48.788949 24927 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:48.790140 24912 consensus_queue.cc:1048] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:48.790376 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:48.790547 20159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:01:48.791133 20091 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:48.791378 20159 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 16: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:48.792423 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:48.794081 24927 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:48.794588 24927 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:01:48.795046 24927 raft_consensus.cc:3060] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:48.855703 20017 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:48.864413 24927 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:48.866887 24927 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:48.868134 24998 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:48.871074 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:48.871758 20855 raft_consensus.cc:1275] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:48.873423 24991 consensus_queue.cc:1048] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:48.877171 20017 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:48.879096 24969 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:48.887419 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:48.888265 20165 raft_consensus.cc:2393] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 16: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:01:48.888216 20091 raft_consensus.cc:2468] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 16.
I20260709 13:01:48.890367 19981 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:48.915766 24918 raft_consensus.cc:2804] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:48.947165 24918 raft_consensus.cc:697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:48.952545 24918 consensus_queue.cc:237] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:48.985356 19839 catalog_manager.cc:5697] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 11 to 14, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 14 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:48.991091 19839 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 13 to 14, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 14 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:48.996423 19839 catalog_manager.cc:5697] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 13 to 14, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 14 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
W20260709 13:01:48.998085 19840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32784 (request call id 161) took 13420 ms (client timeout 15000 ms). Trace:
I20260709 13:01:49.006840 19839 catalog_manager.cc:5697] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 9 to 11, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 11 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:49.009416 19839 catalog_manager.cc:5697] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 15 to 19, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 19 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:49.010672 19839 catalog_manager.cc:5697] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 11 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 16 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:49.013797 19839 catalog_manager.cc:5697] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 12 to 13, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:49.015692 19839 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 12 to 14, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 14 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
W20260709 13:01:49.005301 19840 rpcz_store.cc:269] 0709 13:01:35.578193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.578613 (+   420us) service_pool.cc:224] Handling call
0709 13:01:48.997970 (+13419357us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:48.836916 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:48.839194 (+  2278us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:48.839230 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:01:48.839251 (+    21us) tablet.cc:662] Decoding operations
0709 13:01:48.840279 (+  1028us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:48.840345 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:48.840377 (+    32us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:01:48.840792 (+   415us) tablet.cc:701] Row locks acquired
0709 13:01:48.840812 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:01:48.840973 (+   161us) write_op.cc:270] Start()
0709 13:01:48.841088 (+   115us) write_op.cc:276] Timestamp: P: 1783602108840926 usec, L: 0
0709 13:01:48.841109 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:01:48.841836 (+   727us) log.cc:844] Serialized 1693 byte log entry
0709 13:01:48.865016 (+ 23180us) op_driver.cc:464] REPLICATION: finished
0709 13:01:48.865367 (+   351us) write_op.cc:301] APPLY: starting
0709 13:01:48.865453 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:48.867780 (+  2327us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:48.867892 (+   112us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:48.882362 (+ 14470us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:48.882563 (+   201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:01:48.882606 (+    43us) write_op.cc:312] APPLY: finished
0709 13:01:48.882866 (+   260us) log.cc:844] Serialized 48 byte log entry
0709 13:01:48.883049 (+   183us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:48.883177 (+   128us) write_op.cc:454] Released schema lock
0709 13:01:48.883225 (+    48us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":71879,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":128,"apply.run_cpu_time_us":6356,"apply.run_wall_time_us":18200,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16704,"num_ops":4,"prepare.queue_time_us":76143,"prepare.run_cpu_time_us":3341,"prepare.run_wall_time_us":15472,"replication_time_us":23736,"thread_start_us":75801,"threads_started":1}]]}
I20260709 13:01:49.037652 19839 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:49.052241 20159 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:01:49.054441 25001 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:49.092952 24998 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.093695 24998 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:49.097110 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:01:49.100381 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22d3f15573b24fd9aba34f52879de263" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:01:49.101248 20091 raft_consensus.cc:2410] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:01:49.104121 24998 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:01:49.103689 20129 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:49.106189 24998 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:49.101248 19838 catalog_manager.cc:5697] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 14 to 15, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 15 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.109489 19838 catalog_manager.cc:5697] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:49.110958 19838 catalog_manager.cc:5697] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 12 to 15, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 15 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:49.120225 19838 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:49.122042 19838 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 14 to 15, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 15 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:49.124504 19838 catalog_manager.cc:5697] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 13 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:49.127095 19838 catalog_manager.cc:5697] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 11 to 17, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 17 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } }
I20260709 13:01:49.133226 19838 catalog_manager.cc:5697] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 15 to 16, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 16 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } }
I20260709 13:01:49.176242 19839 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35376 (request call id 100) took 3067 ms. Trace:
I20260709 13:01:49.176697 19839 rpcz_store.cc:276] 0709 13:01:46.109004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.150645 (+ 41641us) service_pool.cc:224] Handling call
0709 13:01:49.176164 (+3025519us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:49.044612 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:49.045106 (+   494us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:49.045142 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:01:49.045159 (+    17us) tablet.cc:662] Decoding operations
0709 13:01:49.046437 (+  1278us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:49.046495 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:49.046521 (+    26us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:01:49.047221 (+   700us) tablet.cc:701] Row locks acquired
0709 13:01:49.047236 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:01:49.047379 (+   143us) write_op.cc:270] Start()
0709 13:01:49.047489 (+   110us) write_op.cc:276] Timestamp: P: 1783602109047324 usec, L: 0
0709 13:01:49.047507 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:01:49.048147 (+   640us) log.cc:844] Serialized 3618 byte log entry
0709 13:01:49.060830 (+ 12683us) op_driver.cc:464] REPLICATION: finished
0709 13:01:49.061180 (+   350us) write_op.cc:301] APPLY: starting
0709 13:01:49.061261 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:49.065755 (+  4494us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:49.065845 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:49.074876 (+  9031us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:49.075037 (+   161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:01:49.075104 (+    67us) write_op.cc:312] APPLY: finished
0709 13:01:49.075390 (+   286us) log.cc:844] Serialized 88 byte log entry
0709 13:01:49.076201 (+   811us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.076325 (+   124us) write_op.cc:454] Released schema lock
0709 13:01:49.076373 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:49.076700 (+   327us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.076725 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9617,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":111,"apply.run_cpu_time_us":10856,"apply.run_wall_time_us":15503,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37584,"num_ops":9,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":3878,"prepare.run_wall_time_us":14588,"replication_time_us":13174}]]}
I20260709 13:01:49.246361 19835 catalog_manager.cc:5697] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:49.313036 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 85015.6us
I20260709 13:01:49.359115 20159 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:49.361231 25024 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:49.383455 19840 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 14 to 15, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 15 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:49.385613 19840 catalog_manager.cc:5697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 12 to 13, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 13 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } }
I20260709 13:01:49.387972 19840 catalog_manager.cc:5697] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 16 to 17, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 17 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } }
I20260709 13:01:49.393572 19840 catalog_manager.cc:5697] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 13 to 14, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 14 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.392532 19835 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:32774 (request call id 155) took 2893 ms. Trace:
I20260709 13:01:49.397727 19835 rpcz_store.cc:276] 0709 13:01:46.499106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.499393 (+   287us) service_pool.cc:224] Handling call
0709 13:01:49.392464 (+2893071us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:49.283696 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:49.284543 (+   847us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:49.284604 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:01:49.284633 (+    29us) tablet.cc:662] Decoding operations
0709 13:01:49.285129 (+   496us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:49.285207 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:49.285258 (+    51us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:01:49.285519 (+   261us) tablet.cc:701] Row locks acquired
0709 13:01:49.285548 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:01:49.285751 (+   203us) write_op.cc:270] Start()
0709 13:01:49.285929 (+   178us) write_op.cc:276] Timestamp: P: 1783602109285688 usec, L: 0
0709 13:01:49.285982 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 13:01:49.286665 (+   683us) log.cc:844] Serialized 537 byte log entry
0709 13:01:49.300895 (+ 14230us) op_driver.cc:464] REPLICATION: finished
0709 13:01:49.301467 (+   572us) write_op.cc:301] APPLY: starting
0709 13:01:49.301588 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:49.303054 (+  1466us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:49.303202 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:49.304188 (+   986us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:49.304358 (+   170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:01:49.304412 (+    54us) write_op.cc:312] APPLY: finished
0709 13:01:49.304640 (+   228us) log.cc:844] Serialized 24 byte log entry
0709 13:01:49.304815 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.304889 (+    74us) write_op.cc:454] Released schema lock
0709 13:01:49.304926 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:49.305291 (+   365us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:49.305317 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":262,"apply.run_cpu_time_us":3779,"apply.run_wall_time_us":3779,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4176,"num_ops":1,"prepare.queue_time_us":108,"prepare.run_cpu_time_us":3412,"prepare.run_wall_time_us":16872,"replication_time_us":14744}]]}
I20260709 13:01:49.396353 19840 catalog_manager.cc:5697] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:49.429164 19838 catalog_manager.cc:5697] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 15 to 16, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 16 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:49.437155 20082 raft_consensus.cc:1275] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:49.439102 24927 consensus_queue.cc:1048] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:49.926641 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 57066.4us
W20260709 13:01:50.017958 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 57482.4us
I20260709 13:01:50.078924 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:50.308375 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:50.308990 20011 raft_consensus.cc:606] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:50.521569 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:50.522289 20011 raft_consensus.cc:606] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:01:50.584970 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 104019us
I20260709 13:01:50.589670 20156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:01:50.590315 20156 raft_consensus.cc:606] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
W20260709 13:01:50.593010 20112 debug-util.cc:398] Leaking SignalData structure 0x7b080027fcc0 after lost signal to thread 12412
W20260709 13:01:50.594264 20112 debug-util.cc:398] Leaking SignalData structure 0x7b0800054100 after lost signal to thread 19874
W20260709 13:01:50.595641 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08001f91c0 after lost signal to thread 20041
W20260709 13:01:50.599378 20112 debug-util.cc:398] Leaking SignalData structure 0x7b0800213ce0 after lost signal to thread 20115
W20260709 13:01:50.600615 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08001c1d40 after lost signal to thread 20189
W20260709 13:01:50.601925 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08000d0780 after lost signal to thread 20888
W20260709 13:01:50.602360 20112 debug-util.cc:398] Leaking SignalData structure 0x7b08002c0920 after lost signal to thread 24956
I20260709 13:01:50.608736 20011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:50.609413 20011 raft_consensus.cc:606] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:50.610741 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:01:50.612828 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:50.620216 19982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 85626.4us
W20260709 13:01:50.641680 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 49425.5us
I20260709 13:01:50.702464 25024 raft_consensus.cc:993] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.702958 25024 raft_consensus.cc:993] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.703398 25024 raft_consensus.cc:1081] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.703835 25031 raft_consensus.cc:1081] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.705442 25031 raft_consensus.cc:993] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.705821 25024 raft_consensus.cc:1081] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.706422 25051 raft_consensus.cc:993] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.707230 25037 raft_consensus.cc:1081] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:50.708017 20085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:01:50.708316 20082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:01:50.708619 20085 raft_consensus.cc:493] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.708936 20082 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.709038 20085 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.709342 20082 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.716324 20085 raft_consensus.cc:515] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:50.717937 20085 leader_election.cc:290] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:01:50.718544 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "e5eb27db496b47129df3e689e546d94a"
I20260709 13:01:50.718803 20090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:01:50.719249 20857 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.718833 20082 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.719656 20090 raft_consensus.cc:493] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.720083 20090 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.724321 20082 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:50.730129 20857 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:50.731532 20055 leader_election.cc:304] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:50.732756 25046 raft_consensus.cc:2804] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:50.733269 25046 raft_consensus.cc:697] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:50.734105 25046 consensus_queue.cc:237] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:50.735142 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111cd0966cde4d87a8a86d90eabf9c0a" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:50.735777 20015 raft_consensus.cc:3055] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:50.736092 20015 raft_consensus.cc:740] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:50.736850 20015 consensus_queue.cc:260] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:01:50.737949 20090 raft_consensus.cc:515] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.738345 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:50.739507 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:50.739804 20015 raft_consensus.cc:3060] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.739861 20017 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:50.740365 20017 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:50.741300 20160 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.741098 20017 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.742316 20017 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.749293 20017 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:50.755071 20082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36126
I20260709 13:01:50.759267 20015 raft_consensus.cc:2468] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:01:50.761794 20160 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:01:50.762121 20082 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:50.760794 20090 leader_election.cc:290] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:50.766876 20056 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:50.774127 20082 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:50.777720 25046 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:50.778537 25046 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:50.779412 25046 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.782256 20082 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.782594 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:50.782683 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5665e7206f7d4ad58380653cdf2490c2" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:50.783209 20156 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.783267 20017 raft_consensus.cc:3055] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:50.784138 20017 raft_consensus.cc:740] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:50.785005 20017 consensus_queue.cc:260] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.786298 20017 raft_consensus.cc:3060] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.801832 19839 catalog_manager.cc:5697] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 13 to 14, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 14 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:50.803442 20082 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:50.805737 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:50.806308 20015 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:50.807360 20156 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:50.808125 20017 raft_consensus.cc:2468] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:50.814071 20015 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 17.
I20260709 13:01:50.818217 20056 leader_election.cc:304] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:50.820147 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a78ca0f82940f0b3698fceba7b2ce4" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:50.820775 20156 raft_consensus.cc:3055] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:50.821339 25046 raft_consensus.cc:2804] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:50.823542 25046 raft_consensus.cc:697] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:50.824738 25046 consensus_queue.cc:237] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.828871 20156 raft_consensus.cc:740] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:01:50.822146 20056 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:50.830065 20156 consensus_queue.cc:260] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.832753 25009 raft_consensus.cc:2804] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:50.833328 25009 raft_consensus.cc:697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:50.834120 25009 consensus_queue.cc:237] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:50.844828 20156 raft_consensus.cc:3060] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:50.860771 20156 raft_consensus.cc:2468] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 17.
I20260709 13:01:50.915158 19839 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 12 to 13, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 13 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.921597 19839 catalog_manager.cc:5697] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 11 to 12, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 12 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:50.923453 19839 catalog_manager.cc:5697] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 16 to 17, leader changed from 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 17 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:51.197991 20017 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:51.201622 25046 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:51.279786 20160 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:51.283069 20156 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:01:51.285964 25046 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:51.287693 25046 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:51.319020 20017 raft_consensus.cc:1275] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:01:51.321666 25035 consensus_queue.cc:1048] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:01:51.345067 20160 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:51.346607 25035 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:01:51.356561 20186 debug-util.cc:398] Leaking SignalData structure 0x7b08002ca960 after lost signal to thread 12412
W20260709 13:01:51.357856 20186 debug-util.cc:398] Leaking SignalData structure 0x7b08002180c0 after lost signal to thread 19874
W20260709 13:01:51.359211 20186 debug-util.cc:398] Leaking SignalData structure 0x7b08002735c0 after lost signal to thread 20041
W20260709 13:01:51.360821 20186 debug-util.cc:398] Leaking SignalData structure 0x7b080021c3e0 after lost signal to thread 20115
I20260709 13:01:51.361935 20857 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:01:51.362149 20186 debug-util.cc:398] Leaking SignalData structure 0x7b0800372b40 after lost signal to thread 20189
W20260709 13:01:51.363718 20186 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0320 after lost signal to thread 20888
I20260709 13:01:51.368541 25027 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:51.381747 20017 raft_consensus.cc:1275] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:51.386247 25027 consensus_queue.cc:1048] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:51.454198 20016 raft_consensus.cc:1275] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:51.473106 25035 consensus_queue.cc:1048] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:51.614981 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:52.231976 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 68403us
W20260709 13:01:52.347322 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 71358.3us
I20260709 13:01:52.768831 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:52.977638 20082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:52.978328 20082 raft_consensus.cc:606] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Received request to transfer leadership to TS 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:01:52.979837 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:01:52.981742 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:52.987818 25035 raft_consensus.cc:993] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: : Instructing follower 48da76e971b84219a6ba80d1ba83b7eb to start an election
I20260709 13:01:52.988416 25035 raft_consensus.cc:1081] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Signalling peer 48da76e971b84219a6ba80d1ba83b7eb to start an election
I20260709 13:01:52.990499 20017 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
 from {username='slave'} at 127.0.0.1:34542
I20260709 13:01:52.991128 20017 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:52.991493 20017 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.128680 20017 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:01:53.128342 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 98662.8us
I20260709 13:01:53.132025 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:53.132678 20082 raft_consensus.cc:3055] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:53.133016 20082 raft_consensus.cc:740] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:53.133874 20082 consensus_queue.cc:260] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:53.135092 20082 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.156844 20017 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:53.157015 20160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258312f626c3416b90ed0d3ac81f31e1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:53.157639 20160 raft_consensus.cc:3060] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.159391 20082 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:53.160712 19981 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:01:53.162344 25081 raft_consensus.cc:2804] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:53.162823 25081 raft_consensus.cc:697] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:53.163610 25081 consensus_queue.cc:237] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:53.165340 20160 raft_consensus.cc:2468] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:01:53.300335 19839 catalog_manager.cc:5697] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 13 to 14, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 14 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.682072 20156 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:53.683768 25079 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:53.724472 20090 raft_consensus.cc:1275] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:53.726089 25079 consensus_queue.cc:1048] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:53.985365 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:54.553663 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 104119us
W20260709 13:01:54.606940 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 58789.9us
W20260709 13:01:54.607192 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 61828.2us
W20260709 13:01:54.620764 20743 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 75857.5us
I20260709 13:01:55.351874 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:55.360342 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08002db980 after lost signal to thread 12412
W20260709 13:01:55.361646 20038 debug-util.cc:398] Leaking SignalData structure 0x7b080023d620 after lost signal to thread 19874
W20260709 13:01:55.363112 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6ce0 after lost signal to thread 20041
W20260709 13:01:55.364456 20038 debug-util.cc:398] Leaking SignalData structure 0x7b0800134f80 after lost signal to thread 20115
W20260709 13:01:55.365741 20038 debug-util.cc:398] Leaking SignalData structure 0x7b08001e3720 after lost signal to thread 20189
W20260709 13:01:55.369545 20038 debug-util.cc:398] Leaking SignalData structure 0x7b0800366200 after lost signal to thread 20888
I20260709 13:01:55.487088 20017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15"
dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
mode: GRACEFUL
new_leader_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:60530
I20260709 13:01:55.487775 20017 raft_consensus.cc:606] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Received request to transfer leadership to TS 41af8b41f8a348c5b60ec12d2e323698
I20260709 13:01:55.489333 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:01:55.491516 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:55.847725 25099 raft_consensus.cc:993] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: : Instructing follower 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:55.848268 25099 raft_consensus.cc:1081] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Signalling peer 41af8b41f8a348c5b60ec12d2e323698 to start an election
I20260709 13:01:55.970063 20085 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
 from {username='slave'} at 127.0.0.1:36124
I20260709 13:01:55.971746 20085 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:56.006337 20085 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:56.125801 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 80624us
I20260709 13:01:56.136404 20085 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:56.141515 20085 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Requested vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:56.143832 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:56.144299 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "630bae767d5b40dfa02289e9b25f7c15" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:01:56.144896 20162 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.145085 20016 raft_consensus.cc:3055] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:56.145562 20016 raft_consensus.cc:740] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:01:56.146613 20016 consensus_queue.cc:260] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:56.148905 20016 raft_consensus.cc:3060] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.170351 20162 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
W20260709 13:01:56.175855 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 48324.8us
I20260709 13:01:56.179548 20056 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:01:56.197570 25113 raft_consensus.cc:2804] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:56.201880 25113 raft_consensus.cc:697] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:56.203536 25113 consensus_queue.cc:237] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:01:56.267885 20016 raft_consensus.cc:2468] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:01:56.303386 19835 catalog_manager.cc:5697] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 11 to 12, leader changed from 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 12 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:56.492724 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:56.826874 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 51046.3us
I20260709 13:01:56.837643 20016 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:56.848726 20162 raft_consensus.cc:1275] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:56.857576 25113 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:56.875087 25113 consensus_queue.cc:1048] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:57.621249 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:57.684926 20085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d"
dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
mode: GRACEFUL
new_leader_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:58322
I20260709 13:01:57.685665 20085 raft_consensus.cc:606] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Received request to transfer leadership to TS e5eb27db496b47129df3e689e546d94a
I20260709 13:01:57.687515 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:01:57.689291 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:57.885625 25104 raft_consensus.cc:993] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: : Instructing follower e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:57.886255 25104 raft_consensus.cc:1081] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Signalling peer e5eb27db496b47129df3e689e546d94a to start an election
I20260709 13:01:57.889070 20854 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d"
dest_uuid: "e5eb27db496b47129df3e689e546d94a"
 from {username='slave'} at 127.0.0.1:44216
I20260709 13:01:57.889775 20854 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:57.890313 20854 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.932693 20854 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:57.936854 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:01:57.937523 20085 raft_consensus.cc:3055] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:57.937819 20085 raft_consensus.cc:740] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:01:57.938685 20085 consensus_queue.cc:260] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:57.946244 20085 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.014079 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "e5eb27db496b47129df3e689e546d94a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:01:58.062208 20162 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.119494 20854 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:01:58.138504 20085 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:58.163714 20162 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5eb27db496b47129df3e689e546d94a in term 14.
I20260709 13:01:58.167026 20744 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a; no voters: 
I20260709 13:01:58.168128 25116 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:58.168969 25116 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Becoming Leader. State: Replica: e5eb27db496b47129df3e689e546d94a, State: Running, Role: LEADER
I20260709 13:01:58.170100 25116 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:01:58.250953 19835 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a reported cstate change: term changed from 13 to 14, leader changed from 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67) to e5eb27db496b47129df3e689e546d94a (127.12.30.65). New cstate: current_term: 14 leader_uuid: "e5eb27db496b47129df3e689e546d94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:58.685061 20162 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:58.686987 25134 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:58.690270 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:58.796341 20744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.68:46223, user_credentials={real_user=slave}} blocked reactor thread for 46887.2us
I20260709 13:01:58.805877 12409 tablet_server.cc:179] TabletServer@127.12.30.65:38607 shutting down...
W20260709 13:01:58.821642 20129 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:01:58.858690 20129 consensus_peers.cc:597] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:58.863029 20129 consensus_peers.cc:597] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:58.863771 20085 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer e5eb27db496b47129df3e689e546d94a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:01:58.865393 20055 consensus_peers.cc:597] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:58.867214 20129 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:58.868330 25145 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:58.938843 20129 consensus_peers.cc:597] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.023661 20055 consensus_peers.cc:597] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.033090 19981 consensus_peers.cc:597] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.034950 20129 consensus_peers.cc:597] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.041697 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 78167.2us
W20260709 13:01:59.047863 19981 consensus_peers.cc:597] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.047547 20129 consensus_peers.cc:597] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.071501 19981 consensus_peers.cc:597] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.073065 19981 consensus_peers.cc:597] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.086756 20055 consensus_peers.cc:597] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.097450 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:01:59.099800 19981 consensus_peers.cc:597] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.098831 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.103600 12409 raft_consensus.cc:2243] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.104446 12409 raft_consensus.cc:2272] T 2d767704576e43bb9f6bcd864fa9a6fe P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.111231 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.113600 12409 raft_consensus.cc:2243] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.114346 12409 raft_consensus.cc:2272] T 054afb193dbf43d48b1b0ba6ce809ee3 P e5eb27db496b47129df3e689e546d94a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.117735 12409 tablet_replica.cc:333] stopping tablet replica
W20260709 13:01:59.122820 20129 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.123700 20055 consensus_peers.cc:597] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.124720 12409 raft_consensus.cc:2243] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.126036 12409 raft_consensus.cc:2272] T bb22f25deefa4dce94a6ca7ffecaee7c P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.132807 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.133911 12409 raft_consensus.cc:2243] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.134626 12409 raft_consensus.cc:2272] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.140180 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.141227 12409 raft_consensus.cc:2243] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.144078 12409 raft_consensus.cc:2272] T af7bc64cbbc04798be25ec85d5de3b0d P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:59.150750 20129 consensus_peers.cc:597] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.153627 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.154901 12409 raft_consensus.cc:2243] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.157459 12409 raft_consensus.cc:2272] T 4ccd50eb99764ffc91bd06202fd0a5e7 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.163363 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.164390 12409 raft_consensus.cc:2243] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.165663 12409 raft_consensus.cc:2272] T ebe7c81078d24dfd8ebcfce8dd265486 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:59.166201 20129 consensus_peers.cc:597] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.176020 20055 consensus_peers.cc:597] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.181742 20055 consensus_peers.cc:597] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.185046 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.186141 12409 raft_consensus.cc:2243] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.186909 12409 raft_consensus.cc:2272] T 5acf254dbaee44d1bc844da86793ef92 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.190510 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.191617 12409 raft_consensus.cc:2243] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.192536 12409 raft_consensus.cc:2272] T 8b1a8f437ee9497c987ac7a6206d2338 P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:59.194875 20129 consensus_peers.cc:597] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.196193 12409 tablet_replica.cc:333] stopping tablet replica
W20260709 13:01:59.204115 20055 consensus_peers.cc:597] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.207010 12409 raft_consensus.cc:2243] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus shutting down.
W20260709 13:01:59.207502 20129 consensus_peers.cc:597] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.208180 12409 raft_consensus.cc:2272] T 0af76b6f84f54d249f7949cf3c104e32 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.212215 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.213141 12409 raft_consensus.cc:2243] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.215489 12409 raft_consensus.cc:2272] T b26fb84490474f98b31c0593ddf1b013 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.220323 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.221336 12409 raft_consensus.cc:2243] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.222110 12409 raft_consensus.cc:2272] T d6f643e855f54a4d8def1836bbdcf39b P e5eb27db496b47129df3e689e546d94a [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.225696 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.226671 12409 raft_consensus.cc:2243] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.227540 12409 raft_consensus.cc:2272] T 77ca1b6b08b74f04948397dece34f882 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.230885 12409 tablet_replica.cc:333] stopping tablet replica
W20260709 13:01:59.238812 19981 consensus_peers.cc:597] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.231786 12409 raft_consensus.cc:2243] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:01:59.239853 20055 consensus_peers.cc:597] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.241344 12409 raft_consensus.cc:2272] T 62a5f678444241d5a158a35557f9a25b P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:59.241739 20055 consensus_peers.cc:597] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.244932 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.245882 12409 raft_consensus.cc:2243] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.246712 12409 raft_consensus.cc:2272] T f761e6c8cc73410eb153cfbcb79afb90 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:59.258388 19981 consensus_peers.cc:597] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:01:59.264441 19981 consensus_peers.cc:597] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.268919 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.270291 12409 raft_consensus.cc:2243] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.272899 12409 raft_consensus.cc:2272] T 4b58ddc27cb54fd28e2324df51a5ff24 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:01:59.274178 20129 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.277900 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.278961 12409 raft_consensus.cc:2243] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.280117 12409 raft_consensus.cc:2272] T 746d924541574d6086a6ae086476a1e7 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.283486 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.284540 12409 raft_consensus.cc:2243] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.285454 12409 raft_consensus.cc:2272] T 2f8749de35944e3d92b86f7d4c472de2 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.288642 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.289450 12409 raft_consensus.cc:2243] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.290712 12409 raft_consensus.cc:2272] T 6240627eb43d4d039f730bc326fdc93e P e5eb27db496b47129df3e689e546d94a [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.294037 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.294849 12409 raft_consensus.cc:2243] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.295524 12409 raft_consensus.cc:2272] T 2fd1b4e7346b47ffaa37a1f24e287d3a P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.299034 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.299876 12409 raft_consensus.cc:2243] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.301154 12409 raft_consensus.cc:2272] T d94fa4e16c4f424c86e23d82c9408473 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.321458 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.322422 12409 raft_consensus.cc:2243] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.324019 12409 raft_consensus.cc:2272] T 1c733c25de16421ba3746f8c6de93bd5 P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.327440 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.328423 12409 raft_consensus.cc:2243] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.329738 12409 raft_consensus.cc:2272] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.334870 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.335779 12409 raft_consensus.cc:2243] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.337242 12409 raft_consensus.cc:2272] T 59c71a9c3bb24e9d92023242a19a363d P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.351095 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.353319 12409 raft_consensus.cc:2243] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.354530 12409 raft_consensus.cc:2272] T ea4e65a94f914c99a52cbf5467a84a13 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.361218 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.362135 12409 raft_consensus.cc:2243] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.372437 12409 raft_consensus.cc:2272] T 9d934e44cfcb452b9a7d72dc634e9e77 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.376001 12409 tablet_replica.cc:333] stopping tablet replica
W20260709 13:01:59.376533 20055 consensus_peers.cc:597] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.377075 12409 raft_consensus.cc:2243] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:01:59.365422 19981 consensus_peers.cc:597] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:01:59.383105 12409 raft_consensus.cc:2272] T d8a881d176a0426b9337a6b7c224b88c P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.386615 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.396683 12409 raft_consensus.cc:2243] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.397626 12409 raft_consensus.cc:2272] T fc68b8861f1d4bf09ec026e1ba26cdf6 P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.400830 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.401718 12409 raft_consensus.cc:2243] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.402420 12409 raft_consensus.cc:2272] T 05d19c460b3d4d2cab96fa3f1372b25b P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.405557 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.416652 12409 raft_consensus.cc:2243] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.417425 12409 raft_consensus.cc:2272] T 6c7a345503524963ae2cf82630978b5a P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.420704 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.421551 12409 raft_consensus.cc:2243] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.422472 12409 raft_consensus.cc:2272] T f4ea05922db6473289d3a25f7b32e86e P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.426132 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.427409 12409 raft_consensus.cc:2243] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.434836 12409 raft_consensus.cc:2272] T eb11384495c2485aaab243a655f75c86 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.449604 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.450663 12409 raft_consensus.cc:2243] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.451442 12409 raft_consensus.cc:2272] T bde87ca1d98f4bd8b90d7f1502cc67f1 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.455027 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.456218 12409 raft_consensus.cc:2243] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.456902 12409 raft_consensus.cc:2272] T 7b9595f271b64d9caeb8e95eb6890e38 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.462841 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.463828 12409 raft_consensus.cc:2243] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.464522 12409 raft_consensus.cc:2272] T 3e1c1a13f585422ea6d2daf771874d19 P e5eb27db496b47129df3e689e546d94a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.467703 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.468569 12409 raft_consensus.cc:2243] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.477116 12409 raft_consensus.cc:2272] T fac4d79f682d4f169331b9656bef55fe P e5eb27db496b47129df3e689e546d94a [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.483345 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.484290 12409 raft_consensus.cc:2243] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.485486 12409 raft_consensus.cc:2272] T 81fd99f42a47497f9d4372157320611b P e5eb27db496b47129df3e689e546d94a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.489048 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.489907 12409 raft_consensus.cc:2243] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:01:59.491300 12409 raft_consensus.cc:2272] T e4bbc03ab1f04c8ebbfad0c35770dad1 P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.495419 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.496471 12409 raft_consensus.cc:2243] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.497257 12409 raft_consensus.cc:2272] T 4c36c61154404ebebcf309833a8bf5ea P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.501250 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.502177 12409 raft_consensus.cc:2243] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.503199 12409 raft_consensus.cc:2272] T 8d2ab233d0664594955baf948686386b P e5eb27db496b47129df3e689e546d94a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.506708 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.507624 12409 raft_consensus.cc:2243] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.508371 12409 raft_consensus.cc:2272] T 45dd40d980c94ccdab3d167b33984c44 P e5eb27db496b47129df3e689e546d94a [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.512126 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.512962 12409 raft_consensus.cc:2243] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.513616 12409 raft_consensus.cc:2272] T 2fe4ca3a058c431f8e6ecbfc90468313 P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.520870 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.528798 12409 raft_consensus.cc:2243] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.530779 12409 raft_consensus.cc:2272] T 111cd0966cde4d87a8a86d90eabf9c0a P e5eb27db496b47129df3e689e546d94a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.534502 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.535391 12409 raft_consensus.cc:2243] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.536155 12409 raft_consensus.cc:2272] T 88eccd78839140bba038b898679a00b0 P e5eb27db496b47129df3e689e546d94a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:01:59.539306 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:01:59.540063 12409 raft_consensus.cc:2243] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:01:59.540699 12409 raft_consensus.cc:2272] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P e5eb27db496b47129df3e689e546d94a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:00.344774 25131 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.345696 25131 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.348304 25131 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.366529 25124 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.362771 25158 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.367220 25124 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.387212 25158 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.388746 25124 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.389865 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.390578 20165 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 12.
I20260709 13:02:00.392051 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.392935 25124 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:00.393304 25124 raft_consensus.cc:493] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.393642 25124 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.402695 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:00.403090 25124 raft_consensus.cc:515] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.404378 20091 raft_consensus.cc:2393] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 13: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:02:00.407732 25124 leader_election.cc:290] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.418589 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebe7c81078d24dfd8ebcfce8dd265486" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:02:00.419266 20165 raft_consensus.cc:3060] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.438617 25104 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.439258 25104 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.441311 25104 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.442061 25104 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.442358 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.442588 25104 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.444626 25104 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:02:00.452646 20055 leader_election.cc:336] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.456256 25148 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.457013 25148 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.459748 25148 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.464313 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.468626 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.483237 25107 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.484028 25107 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.486048 25107 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.497424 25140 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.524134 25140 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.527058 25140 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:00.498708 25157 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.529992 25157 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.532821 25157 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.503702 25148 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.554251 25148 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.556840 25148 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.507228 20165 raft_consensus.cc:2468] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:02:00.508390 25101 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.557791 25101 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.560554 25101 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.521899 25124 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.561919 25124 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.564816 25124 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.500031 25158 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:00.539412 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.567477 20162 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:02:00.581459 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:00.585177 19982 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.586887 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:00.587682 20015 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:02:00.589107 25140 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.589900 25140 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.592437 25140 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.593116 25140 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:00.593554 25140 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.594321 25140 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:00.605046 20055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 151519us
I20260709 13:02:00.632460 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:00.633385 20015 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
W20260709 13:02:00.549069 20129 leader_election.cc:336] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.550119 25104 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.635141 25104 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.637418 25104 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.551350 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:00.639087 20085 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:02:00.553375 25107 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.642612 25107 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.645114 25107 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.645833 20129 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:02:00.647221 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 98183.9us
I20260709 13:02:00.648931 25131 raft_consensus.cc:2749] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.652629 25131 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.653395 25131 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.656577 25131 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.666816 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.668771 25101 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:00.669349 25101 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.669799 25101 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.681990 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.683688 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.684870 25107 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.685529 25107 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.687587 25107 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.698336 25148 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.706300 25148 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.708904 25148 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.705494 20056 leader_election.cc:304] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.703449 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:00.709985 20085 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:02:00.715044 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:00.715201 25168 raft_consensus.cc:2804] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:00.715863 20015 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 13.
I20260709 13:02:00.716142 25168 raft_consensus.cc:697] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:02:00.724490 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:00.725102 25168 consensus_queue.cc:237] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.735146 25131 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
W20260709 13:02:00.718766 20055 leader_election.cc:336] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:00.739487 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 73323.8us
W20260709 13:02:00.749114 20055 leader_election.cc:336] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.757900 20055 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:00.761346 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:02:00.764427 12409 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 20884)
I20260709 13:02:00.752323 25131 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.775116 25168 raft_consensus.cc:2749] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.779093 25131 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:02:00.775367 20055 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.779723 20055 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:00.780318 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.780490 25168 raft_consensus.cc:2749] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:00.781041 20165 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 9.
W20260709 13:02:00.773201 20130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 45017us
I20260709 13:02:00.776852 25140 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.789144 25168 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:00.789626 25168 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.789979 25168 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.791429 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:02:00.792315 25167 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.795784 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:02:00.796650 25104 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:00.796990 20055 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.787420 25101 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.797092 25104 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.797976 25104 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.798084 25163 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.798640 25163 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.799068 25101 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.800734 25163 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:00.795042 25167 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.801600 20165 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 15: Already voted for candidate 8b5b4e2a92b046d096f2661c54321d41 in this term.
I20260709 13:02:00.803162 25167 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.806732 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:00.810349 20016 raft_consensus.cc:2393] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8b5b4e2a92b046d096f2661c54321d41 in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:02:00.812115 25168 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.815702 25168 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.787976 25140 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.797500 20055 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
W20260709 13:02:00.828047 20055 leader_election.cc:336] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.826840 25104 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:02:00.828606 20055 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.840502 25167 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.842265 25167 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.842954 25107 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.843514 25107 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.839720 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:02:00.845113 25157 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:00.845511 25157 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.845930 25157 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.846493 25167 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.830425 25168 raft_consensus.cc:2749] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.845695 25107 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.850369 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:00.851064 20016 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:02:00.849152 25174 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.855974 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:02:00.857127 25104 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.859938 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.861243 25166 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:00.861858 25166 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.862349 25166 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.863188 25107 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:00.863624 25107 raft_consensus.cc:493] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.864006 25107 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.874368 25107 raft_consensus.cc:515] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.854779 25174 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.879984 25174 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:02:00.880774 20056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.66:39003, user_credentials={real_user=slave}} blocked reactor thread for 89432us
I20260709 13:02:00.881181 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.882021 20165 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 13.
I20260709 13:02:00.854360 25157 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.885069 25176 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.886960 25107 leader_election.cc:290] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:00.886296 25157 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:00.887586 25176 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:02:00.889746 19981 leader_election.cc:336] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:00.890651 19981 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.880210 25166 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.898200 25166 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:00.895430 20017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:00.916227 20017 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 17.
I20260709 13:02:00.895903 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:00.917511 20011 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 15: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:02:00.889856 25176 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.887338 19839 catalog_manager.cc:5697] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 12 to 13, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 13 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:00.920307 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:00.905267 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b58ddc27cb54fd28e2324df51a5ff24" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:00.921028 20091 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:02:00.921294 20016 raft_consensus.cc:3060] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.923578 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:00.907085 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.924252 20091 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
W20260709 13:02:00.908195 20129 leader_election.cc:336] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:00.893577 20055 leader_election.cc:336] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.924914 20129 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:00.910749 20015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:00.925946 20015 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 14: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:02:00.912458 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:00.927130 20165 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
I20260709 13:02:00.927886 25176 raft_consensus.cc:2749] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:00.929565 20055 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:00.931003 20055 leader_election.cc:336] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:00.931880 20055 leader_election.cc:336] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.931954 25157 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:00.932559 25157 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.933153 25157 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:00.937533 20055 leader_election.cc:336] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:00.938171 20055 leader_election.cc:336] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.940415 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:00.941176 20091 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 9.
I20260709 13:02:00.941392 25157 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
W20260709 13:02:00.913882 19981 leader_election.cc:336] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.943794 25176 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.944504 25176 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.946734 25176 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.948702 20056 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:00.949715 25175 raft_consensus.cc:2749] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:00.957537 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:00.960728 25157 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:00.962697 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:02:00.963344 20129 leader_election.cc:336] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.963460 20085 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 17.
I20260709 13:02:00.965332 25175 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:02:00.965960 25175 raft_consensus.cc:493] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.966619 25175 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:00.975170 25175 raft_consensus.cc:515] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:00.979876 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb11384495c2485aaab243a655f75c86" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:02:00.980604 20165 raft_consensus.cc:3060] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:00.984748 20129 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:00.985592 20129 leader_election.cc:336] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:00.986850 20016 raft_consensus.cc:2468] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:02:00.987982 20165 raft_consensus.cc:2468] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48da76e971b84219a6ba80d1ba83b7eb in term 14.
I20260709 13:02:00.988855 25157 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:00.989396 25157 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:00.991379 25157 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:00.993237 19982 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.994246 20130 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:00.994364 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:02:00.995296 20085 raft_consensus.cc:2393] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 14: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:02:00.996251 25157 raft_consensus.cc:2804] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:00.996456 25176 raft_consensus.cc:2804] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:00.996848 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:00.997370 25176 raft_consensus.cc:697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:02:00.996919 25157 raft_consensus.cc:697] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Becoming Leader. State: Replica: 48da76e971b84219a6ba80d1ba83b7eb, State: Running, Role: LEADER
I20260709 13:02:00.998294 25176 consensus_queue.cc:237] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.999109 25157 consensus_queue.cc:237] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:00.999574 25173 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:01.000092 25173 raft_consensus.cc:493] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.000474 25173 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.008983 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:02:01.010057 20085 raft_consensus.cc:2393] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 15: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
I20260709 13:02:00.986058 20129 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.012465 25107 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.013958 25104 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.014460 25104 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:01.016279 25104 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:01.012250 20129 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.021462 20129 leader_election.cc:336] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.021971 20129 leader_election.cc:336] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.022409 20129 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.023160 25107 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:01.021984 20055 leader_election.cc:336] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.026609 25173 raft_consensus.cc:515] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.033948 25173 leader_election.cc:290] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:01.032141 25175 leader_election.cc:290] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Requested vote from peers 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:02:01.037967 19981 leader_election.cc:336] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.045004 20055 leader_election.cc:336] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.038516 19981 leader_election.cc:304] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.047745 25157 raft_consensus.cc:2749] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.050638 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.051649 25167 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:01.052137 25167 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.052604 25167 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:01.055053 20055 leader_election.cc:336] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.064599 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.065995 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.068178 19838 catalog_manager.cc:5697] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 13 to 14, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 14 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.070045 25175 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.070593 25173 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.070667 19835 catalog_manager.cc:5697] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb reported cstate change: term changed from 13 to 14, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66). New cstate: current_term: 14 leader_uuid: "48da76e971b84219a6ba80d1ba83b7eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.075712 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
W20260709 13:02:01.071245 20129 leader_election.cc:336] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.076421 20085 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
W20260709 13:02:01.077342 20129 leader_election.cc:336] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.078228 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:02:01.079066 25166 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:01.079128 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:01.079541 25166 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.079885 20085 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 12.
I20260709 13:02:01.080109 25166 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:01.082437 25167 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:01.084813 25167 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:01.091616 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:01.092675 20016 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:02:01.101038 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:01.101945 20162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59c71a9c3bb24e9d92023242a19a363d" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:02:01.102576 20162 raft_consensus.cc:3060] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.105047 20165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6240627eb43d4d039f730bc326fdc93e" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41"
I20260709 13:02:01.105748 20165 raft_consensus.cc:3060] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:02:01.105904 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.111234 25173 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:01.111732 25173 raft_consensus.cc:493] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.112195 25173 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.120537 25173 raft_consensus.cc:515] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
W20260709 13:02:01.123890 20055 leader_election.cc:336] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.133334 25166 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:01.134799 20016 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 41af8b41f8a348c5b60ec12d2e323698 in current term 18: Already voted for candidate 48da76e971b84219a6ba80d1ba83b7eb in this term.
I20260709 13:02:01.137940 20162 raft_consensus.cc:2468] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
W20260709 13:02:01.139345 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 64216.7us
I20260709 13:02:01.140233 25173 leader_election.cc:290] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:01.142037 20056 leader_election.cc:304] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.142879 25175 raft_consensus.cc:2804] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:01.143417 25175 raft_consensus.cc:697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:02:01.144310 25175 consensus_queue.cc:237] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:02:01.149360 20129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 72036.7us
W20260709 13:02:01.149981 20129 leader_election.cc:336] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.153514 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.155187 25104 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:01.158675 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:01.153553 20165 raft_consensus.cc:2468] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 10.
I20260709 13:02:01.159273 20016 raft_consensus.cc:3060] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.161693 25166 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:01.167184 20016 raft_consensus.cc:2468] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
I20260709 13:02:01.171305 20056 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.173602 25104 raft_consensus.cc:2804] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:02:01.174086 25104 raft_consensus.cc:697] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:02:01.174834 25104 consensus_queue.cc:237] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.182991 20056 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
W20260709 13:02:01.187512 20055 leader_election.cc:336] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.189082 25175 raft_consensus.cc:2804] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:01.189707 25175 raft_consensus.cc:697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:02:01.190523 25175 consensus_queue.cc:237] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:01.192178 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.194377 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4bbc03ab1f04c8ebbfad0c35770dad1" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698" is_pre_election: true
I20260709 13:02:01.197999 19838 catalog_manager.cc:5697] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 14 to 15, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 15 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.203368 25176 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:01.203888 25176 raft_consensus.cc:493] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.204311 25176 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.203886 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
W20260709 13:02:01.216475 19981 leader_election.cc:336] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.217250 25131 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.223533 25131 raft_consensus.cc:493] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.224046 25131 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.226462 20129 leader_election.cc:304] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.229713 25172 raft_consensus.cc:2764] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:02:01.233058 19838 catalog_manager.cc:5697] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 9 to 10, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 10 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.235291 19838 catalog_manager.cc:5697] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 15 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 16 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:01.238917 19981 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 8b5b4e2a92b046d096f2661c54321d41, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.215953 25176 raft_consensus.cc:515] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.239786 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.240571 25107 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:01.240895 25107 raft_consensus.cc:493] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.240967 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:01.241266 25107 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.246723 25171 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.247133 25171 raft_consensus.cc:493] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e5eb27db496b47129df3e689e546d94a)
I20260709 13:02:01.247499 25171 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.253366 20162 raft_consensus.cc:1275] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:01.255045 25104 consensus_queue.cc:1048] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:01.255599 25171 raft_consensus.cc:515] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.262243 25107 raft_consensus.cc:515] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.264509 25107 leader_election.cc:290] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:01.265720 25171 leader_election.cc:290] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:02:01.265385 20129 leader_election.cc:336] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.267133 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.65:38607, user_credentials={real_user=slave}} blocked reactor thread for 50938.3us
W20260709 13:02:01.269826 19981 leader_election.cc:336] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.270737 19981 leader_election.cc:336] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.271428 20085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4e65a94f914c99a52cbf5467a84a13" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:02:01.235782 25131 raft_consensus.cc:515] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.272060 20085 raft_consensus.cc:3060] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.272650 25166 raft_consensus.cc:2749] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.273514 25131 leader_election.cc:290] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
W20260709 13:02:01.274391 19981 leader_election.cc:336] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.275089 19981 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:02:01.276439 19981 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.276813 20091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fac4d79f682d4f169331b9656bef55fe" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:02:01.277048 19981 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.277490 20091 raft_consensus.cc:3060] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.278108 25166 raft_consensus.cc:2749] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.278203 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a2cace87d4c4d26be2f8bffdb5b6d50" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:02:01.279246 20090 raft_consensus.cc:3060] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.278357 20082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c733c25de16421ba3746f8c6de93bd5" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:02:01.281291 20082 raft_consensus.cc:3060] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.292164 25166 raft_consensus.cc:2749] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:01.305673 19981 leader_election.cc:336] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.306053 20085 raft_consensus.cc:2468] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 14.
I20260709 13:02:01.310915 20091 raft_consensus.cc:2468] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:02:01.308346 20082 raft_consensus.cc:2468] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 13.
I20260709 13:02:01.309343 20090 raft_consensus.cc:2468] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:02:01.304800 25176 leader_election.cc:290] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:01.321851 20090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "48da76e971b84219a6ba80d1ba83b7eb" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "41af8b41f8a348c5b60ec12d2e323698"
I20260709 13:02:01.323287 20090 raft_consensus.cc:2393] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 48da76e971b84219a6ba80d1ba83b7eb in current term 18: Already voted for candidate 41af8b41f8a348c5b60ec12d2e323698 in this term.
W20260709 13:02:01.327562 19981 leader_election.cc:336] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.343824 20129 consensus_peers.cc:597] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:01.348590 20129 leader_election.cc:304] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:01.353884 20129 leader_election.cc:304] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:01.354813 20129 leader_election.cc:304] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:01.355013 25171 raft_consensus.cc:2804] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:01.355651 25107 raft_consensus.cc:2804] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:01.354102 20055 consensus_peers.cc:597] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:01.356447 25172 raft_consensus.cc:2804] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:01.356093 25171 raft_consensus.cc:697] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:02:01.357914 25172 raft_consensus.cc:697] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:02:01.358386 25171 consensus_queue.cc:237] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.358789 25172 consensus_queue.cc:237] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.356352 20129 leader_election.cc:304] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:01.356953 25107 raft_consensus.cc:697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:02:01.366261 25107 consensus_queue.cc:237] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:01.366273 25131 raft_consensus.cc:2804] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:01.368677 25131 raft_consensus.cc:697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:02:01.369807 25131 consensus_queue.cc:237] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
W20260709 13:02:01.368093 20129 leader_election.cc:336] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.374266 20129 leader_election.cc:336] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.374987 20129 leader_election.cc:336] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.379359 19981 leader_election.cc:304] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.380561 25163 raft_consensus.cc:2749] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:01.392763 19981 leader_election.cc:336] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:01.393486 19981 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:01.395884 25174 raft_consensus.cc:2749] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:01.436414 20055 consensus_peers.cc:597] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.449211 20129 leader_election.cc:336] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:01.460472 20129 consensus_peers.cc:597] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.473546 19981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:39831, user_credentials={real_user=slave}} blocked reactor thread for 48742.9us
I20260709 13:02:01.449255 20015 raft_consensus.cc:1275] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:02:01.476409 20129 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:01.477455 25131 consensus_queue.cc:1048] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:01.476991 19838 catalog_manager.cc:5697] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 13 to 14, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 14 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:01.479907 20055 consensus_peers.cc:597] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.484346 19981 consensus_peers.cc:597] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:01.485524 19838 catalog_manager.cc:5697] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 12 to 13, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.496775 19838 catalog_manager.cc:5697] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 12 to 13, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 13 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:01.487970 20055 consensus_peers.cc:597] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.519951 20129 consensus_peers.cc:597] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:01.531201 20129 consensus_peers.cc:597] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:01.532198 19838 catalog_manager.cc:5697] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 15 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:01.575275 19981 consensus_peers.cc:597] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:01.578703 20162 raft_consensus.cc:1275] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Refusing update from remote peer 48da76e971b84219a6ba80d1ba83b7eb: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:01.580905 25174 consensus_queue.cc:1048] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:01.591593 20165 raft_consensus.cc:1275] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:02:01.596166 20055 consensus_peers.cc:597] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:01.598333 25173 consensus_queue.cc:1048] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:02:01.633004 20129 consensus_peers.cc:597] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.645993 19981 consensus_peers.cc:597] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.667223 20055 consensus_peers.cc:597] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:01.682502 20129 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.694216 19981 consensus_peers.cc:597] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.698038 19981 consensus_peers.cc:597] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.755199 20055 consensus_peers.cc:597] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:01.757182 20016 raft_consensus.cc:1275] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:02:01.767493 20129 consensus_peers.cc:597] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:01.771661 25104 consensus_queue.cc:1048] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:01.773725 20156 raft_consensus.cc:1275] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:02:01.774544 12409 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 20884)
I20260709 13:02:01.775763 20090 raft_consensus.cc:1275] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:01.778982 25131 consensus_queue.cc:1048] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:01.788546 20055 consensus_peers.cc:597] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:01.792896 25173 consensus_queue.cc:1048] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:01.818470 20129 consensus_peers.cc:597] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:01.897627 20129 consensus_peers.cc:597] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.906502 20055 consensus_peers.cc:597] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.930254 20129 consensus_peers.cc:597] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.935930 20129 consensus_peers.cc:597] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:01.936424 20129 consensus_peers.cc:597] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:01.936827 20129 consensus_peers.cc:597] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.937247 20129 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.937623 20129 consensus_peers.cc:597] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:01.963254 20090 raft_consensus.cc:1275] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:02:01.983892 20055 consensus_peers.cc:597] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.006968 20055 consensus_peers.cc:597] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.007859 20055 consensus_peers.cc:597] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:01.989336 19981 consensus_peers.cc:597] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:02.009949 20091 raft_consensus.cc:1275] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:02:02.016512 19981 consensus_peers.cc:597] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.017409 19981 consensus_peers.cc:597] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.018777 20129 consensus_peers.cc:597] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:02.022192 25176 consensus_queue.cc:1048] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:02.025054 25193 consensus_queue.cc:1048] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:02.001643 20090 raft_consensus.cc:1275] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:02.042506 25193 consensus_queue.cc:1048] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:02:02.060359 20055 consensus_peers.cc:597] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.088310 20055 consensus_peers.cc:597] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.167088 19981 consensus_peers.cc:597] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.397470 19981 consensus_peers.cc:597] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:02.774976 12409 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 20884)
I20260709 13:02:02.804430 25173 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.805081 25173 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:02.808683 25173 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:02.808773 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:02.809463 20011 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 18.
I20260709 13:02:02.810806 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:02:02.812865 25203 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:02.813238 25203 raft_consensus.cc:493] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.813544 25203 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:02.818536 20055 leader_election.cc:336] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:02.830327 25203 raft_consensus.cc:515] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:02.832616 25203 leader_election.cc:290] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:02.841583 20055 leader_election.cc:336] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:02.852985 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fd99f42a47497f9d4372157320611b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:02.854332 20011 raft_consensus.cc:3060] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:02.865162 20011 raft_consensus.cc:2468] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 19.
I20260709 13:02:02.868916 20056 leader_election.cc:304] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:02.871007 25173 raft_consensus.cc:2804] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:02.871605 25173 raft_consensus.cc:697] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 19 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:02:02.872514 25173 consensus_queue.cc:237] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:02.894513 19838 catalog_manager.cc:5697] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 17 to 19, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 19 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:02.956184 25131 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.956887 25131 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:02.961210 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:02.962000 20011 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 15.
I20260709 13:02:02.963125 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: 
I20260709 13:02:02.964015 25107 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:02.964368 25107 raft_consensus.cc:493] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.964676 25107 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.959221 25131 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:02.969344 20129 leader_election.cc:336] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:02.973141 25175 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.973296 25107 raft_consensus.cc:515] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:02.973786 25175 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:02.977566 20011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b26fb84490474f98b31c0593ddf1b013" candidate_uuid: "8b5b4e2a92b046d096f2661c54321d41" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:02.978268 20011 raft_consensus.cc:3060] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.978405 25107 leader_election.cc:290] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:02.987947 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:02.988955 20016 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 14.
W20260709 13:02:02.990352 20055 leader_election.cc:336] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:02.991367 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:02.992213 25173 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:02.992599 25173 raft_consensus.cc:493] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.992920 25173 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.002437 25173 raft_consensus.cc:515] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:02.975764 25175 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:03.009328 20129 leader_election.cc:336] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:03.024143 20055 leader_election.cc:336] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:03.029584 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62a5f678444241d5a158a35557f9a25b" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:03.030246 20016 raft_consensus.cc:3060] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.031508 20011 raft_consensus.cc:2468] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b5b4e2a92b046d096f2661c54321d41 in term 16.
I20260709 13:02:03.038507 25173 leader_election.cc:290] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:03.043334 20130 leader_election.cc:304] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:03.044526 25107 raft_consensus.cc:2804] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:03.045867 20016 raft_consensus.cc:2468] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:02:03.045261 25107 raft_consensus.cc:697] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Becoming Leader. State: Replica: 8b5b4e2a92b046d096f2661c54321d41, State: Running, Role: LEADER
I20260709 13:02:03.074100 25107 consensus_queue.cc:237] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:03.075333 20056 leader_election.cc:304] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:03.076550 25173 raft_consensus.cc:2804] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:03.077497 25173 raft_consensus.cc:697] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:02:03.078986 25173 consensus_queue.cc:237] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:03.102119 19838 catalog_manager.cc:5697] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 13 to 15, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 15 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.116760 25203 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.117516 25203 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:03.122643 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb" is_pre_election: true
I20260709 13:02:03.123435 20016 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 15.
I20260709 13:02:03.124864 19838 catalog_manager.cc:5697] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 reported cstate change: term changed from 14 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68). New cstate: current_term: 16 leader_uuid: "8b5b4e2a92b046d096f2661c54321d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:03.130097 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: 
I20260709 13:02:03.131860 25203 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:03.135962 20055 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:03.136373 25203 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:03.136958 25203 raft_consensus.cc:493] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.137332 25203 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:03.156668 25203 raft_consensus.cc:515] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:03.160172 25203 leader_election.cc:290] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Requested vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:03.160790 20016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94fa4e16c4f424c86e23d82c9408473" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "48da76e971b84219a6ba80d1ba83b7eb"
I20260709 13:02:03.161473 20016 raft_consensus.cc:3060] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:03.163233 20055 leader_election.cc:336] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:03.168020 20016 raft_consensus.cc:2468] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41af8b41f8a348c5b60ec12d2e323698 in term 16.
I20260709 13:02:03.170002 20056 leader_election.cc:304] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb; no voters: e5eb27db496b47129df3e689e546d94a
I20260709 13:02:03.170948 25203 raft_consensus.cc:2804] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:03.171471 25203 raft_consensus.cc:697] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Becoming Leader. State: Replica: 41af8b41f8a348c5b60ec12d2e323698, State: Running, Role: LEADER
I20260709 13:02:03.172331 25203 consensus_queue.cc:237] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } }
I20260709 13:02:03.182780 19838 catalog_manager.cc:5697] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 reported cstate change: term changed from 14 to 16, leader changed from e5eb27db496b47129df3e689e546d94a (127.12.30.65) to 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67). New cstate: current_term: 16 leader_uuid: "41af8b41f8a348c5b60ec12d2e323698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:03.432350 20016 raft_consensus.cc:1275] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:03.434808 25203 consensus_queue.cc:1048] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:03.554656 20016 raft_consensus.cc:1275] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 8b5b4e2a92b046d096f2661c54321d41: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:03.556890 25217 consensus_queue.cc:1048] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:03.654341 20011 raft_consensus.cc:1275] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:03.657640 25203 consensus_queue.cc:1048] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:02:03.665702 20129 consensus_peers.cc:597] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:03.690558 20129 consensus_peers.cc:597] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:03.775564 12409 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 20884)
I20260709 13:02:03.793666 20016 raft_consensus.cc:1275] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Refusing update from remote peer 41af8b41f8a348c5b60ec12d2e323698: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:03.795694 25175 consensus_queue.cc:1048] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:02:03.796500 20129 consensus_peers.cc:597] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:03.833387 19981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111) [suppressed 388 similar messages]
W20260709 13:02:03.880676 20129 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:03.908864 20129 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:03.966358 19981 consensus_peers.cc:597] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:03.969363 19981 consensus_peers.cc:597] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.172057 20129 consensus_peers.cc:597] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:04.179487 12409 tablet_server.cc:196] TabletServer@127.12.30.65:38607 shutdown complete.
W20260709 13:02:04.183318 20129 consensus_peers.cc:597] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.198418 20129 consensus_peers.cc:597] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:04.203079 20129 consensus_peers.cc:597] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:04.216742 20129 consensus_peers.cc:597] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:04.227085 19981 consensus_peers.cc:597] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:04.229281 20129 consensus_peers.cc:597] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.232286 20129 consensus_peers.cc:597] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:04.262969 19981 consensus_peers.cc:597] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.299301 20129 consensus_peers.cc:597] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:04.301384 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
W20260709 13:02:04.318302 20056 consensus_peers.cc:597] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.323130 19981 consensus_peers.cc:597] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.343997 19981 consensus_peers.cc:597] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.345959 20130 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.348310 20056 consensus_peers.cc:597] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.355461 20130 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.372605 20055 consensus_peers.cc:597] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.374323 20129 consensus_peers.cc:597] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.381891 20055 consensus_peers.cc:597] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.382686 20055 consensus_peers.cc:597] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.383178 20055 consensus_peers.cc:597] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.385329 20129 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.385914 20055 consensus_peers.cc:597] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.387751 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:04.389292 12409 tablet_replica.cc:333] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.392764 12409 raft_consensus.cc:2243] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.394191 12409 raft_consensus.cc:2272] T 258312f626c3416b90ed0d3ac81f31e1 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.397630 20130 consensus_peers.cc:597] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.397794 12409 tablet_replica.cc:333] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.399521 12409 raft_consensus.cc:2243] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.400409 12409 raft_consensus.cc:2272] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.404464 12409 tablet_replica.cc:333] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
W20260709 13:02:04.405382 20056 consensus_peers.cc:597] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.405391 12409 raft_consensus.cc:2243] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Raft consensus shutting down.
W20260709 13:02:04.407728 20130 consensus_peers.cc:597] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.412339 12409 raft_consensus.cc:2272] T 5acf254dbaee44d1bc844da86793ef92 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.417650 12409 tablet_replica.cc:333] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.418650 12409 raft_consensus.cc:2243] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.419318 12409 raft_consensus.cc:2272] T e35516f88d604db2ad4a7f67d0017312 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.422606 12409 tablet_replica.cc:333] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.423503 12409 raft_consensus.cc:2243] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.424625 12409 raft_consensus.cc:2272] T 4c36c61154404ebebcf309833a8bf5ea P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.428153 12409 tablet_replica.cc:333] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.429032 12409 raft_consensus.cc:2243] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.429602 12409 raft_consensus.cc:2272] T 5665e7206f7d4ad58380653cdf2490c2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.431474 20129 consensus_peers.cc:597] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.431517 20055 consensus_peers.cc:597] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:04.433619 12409 tablet_replica.cc:333] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.435027 12409 raft_consensus.cc:2243] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.435983 12409 raft_consensus.cc:2272] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.437754 19981 consensus_peers.cc:597] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:04.439126 12409 tablet_replica.cc:333] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.440371 12409 raft_consensus.cc:2243] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.445358 12409 raft_consensus.cc:2272] T 3e1c1a13f585422ea6d2daf771874d19 P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.452026 12409 tablet_replica.cc:333] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.453485 12409 raft_consensus.cc:2243] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.455734 12409 raft_consensus.cc:2272] T 7b9595f271b64d9caeb8e95eb6890e38 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.459105 12409 tablet_replica.cc:333] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.461450 12409 raft_consensus.cc:2243] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.462121 12409 raft_consensus.cc:2272] T d94fa4e16c4f424c86e23d82c9408473 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.466147 12409 tablet_replica.cc:333] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.467188 12409 raft_consensus.cc:2243] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.468366 12409 raft_consensus.cc:2272] T b26fb84490474f98b31c0593ddf1b013 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.470795 20056 consensus_peers.cc:597] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.476511 12409 tablet_replica.cc:333] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.477823 12409 raft_consensus.cc:2243] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:02:04.479374 20129 consensus_peers.cc:597] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:04.481918 12409 raft_consensus.cc:2272] T a42b8897c4224305af65104f541be1fc P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.484365 20130 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.486027 20056 consensus_peers.cc:597] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.489442 12409 tablet_replica.cc:333] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.490549 12409 raft_consensus.cc:2243] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.491371 12409 raft_consensus.cc:2272] T 62a5f678444241d5a158a35557f9a25b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.495030 12409 tablet_replica.cc:333] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.495903 12409 raft_consensus.cc:2243] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Raft consensus shutting down.
W20260709 13:02:04.496376 20056 consensus_peers.cc:597] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.496753 20130 consensus_peers.cc:597] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.497159 12409 raft_consensus.cc:2272] T 2d767704576e43bb9f6bcd864fa9a6fe P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.498445 20129 consensus_peers.cc:597] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:04.500885 12409 tablet_replica.cc:333] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.502786 12409 raft_consensus.cc:2243] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.504767 12409 raft_consensus.cc:2272] T d8a881d176a0426b9337a6b7c224b88c P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.508147 12409 tablet_replica.cc:333] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.508876 12409 raft_consensus.cc:2243] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.509706 12409 raft_consensus.cc:2272] T 4864bdb85f20457e9ef1bfb12f991554 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.512158 20056 consensus_peers.cc:597] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.513108 12409 tablet_replica.cc:333] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.513988 12409 raft_consensus.cc:2243] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.514817 12409 raft_consensus.cc:2272] T 88eccd78839140bba038b898679a00b0 P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.517139 12409 tablet_replica.cc:333] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.517736 12409 raft_consensus.cc:2243] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.518596 12409 raft_consensus.cc:2272] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.522334 12409 tablet_replica.cc:333] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.522958 12409 raft_consensus.cc:2243] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.523451 12409 raft_consensus.cc:2272] T eb6e04fb0be94cccad001ccc48f9bd85 P 48da76e971b84219a6ba80d1ba83b7eb [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.526037 12409 tablet_replica.cc:333] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.526825 12409 raft_consensus.cc:2243] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.527384 12409 raft_consensus.cc:2272] T 4b58ddc27cb54fd28e2324df51a5ff24 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.529652 12409 tablet_replica.cc:333] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.530601 12409 raft_consensus.cc:2243] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.531685 12409 raft_consensus.cc:2272] T 746d924541574d6086a6ae086476a1e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.533854 12409 tablet_replica.cc:333] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.534446 12409 raft_consensus.cc:2243] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.534937 12409 raft_consensus.cc:2272] T f761e6c8cc73410eb153cfbcb79afb90 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.537170 12409 tablet_replica.cc:333] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.537814 12409 raft_consensus.cc:2243] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.538419 12409 raft_consensus.cc:2272] T 2f8749de35944e3d92b86f7d4c472de2 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.540841 12409 tablet_replica.cc:333] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.541606 12409 raft_consensus.cc:2243] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.542927 12409 raft_consensus.cc:2272] T 22d3f15573b24fd9aba34f52879de263 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.545570 12409 tablet_replica.cc:333] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.546384 12409 raft_consensus.cc:2243] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.547622 12409 raft_consensus.cc:2272] T 153ccb361669440fa940c70fa6ae8d76 P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.549873 12409 tablet_replica.cc:333] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.550462 12409 raft_consensus.cc:2243] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.550981 12409 raft_consensus.cc:2272] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.553474 12409 tablet_replica.cc:333] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.554231 12409 raft_consensus.cc:2243] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.555022 12409 raft_consensus.cc:2272] T 2fe4ca3a058c431f8e6ecbfc90468313 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.558073 12409 tablet_replica.cc:333] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.559013 12409 raft_consensus.cc:2243] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.560024 12409 raft_consensus.cc:2272] T 45dd40d980c94ccdab3d167b33984c44 P 48da76e971b84219a6ba80d1ba83b7eb [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.563952 12409 tablet_replica.cc:333] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.564819 12409 raft_consensus.cc:2243] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.565956 12409 raft_consensus.cc:2272] T eb11384495c2485aaab243a655f75c86 P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.568078 12409 tablet_replica.cc:333] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.568706 12409 raft_consensus.cc:2243] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.569535 12409 raft_consensus.cc:2272] T f4ea05922db6473289d3a25f7b32e86e P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.571705 12409 tablet_replica.cc:333] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.572492 12409 raft_consensus.cc:2243] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.573186 12409 raft_consensus.cc:2272] T 8b1a8f437ee9497c987ac7a6206d2338 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.575347 12409 tablet_replica.cc:333] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.575975 12409 raft_consensus.cc:2243] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.576522 12409 raft_consensus.cc:2272] T 81fd99f42a47497f9d4372157320611b P 48da76e971b84219a6ba80d1ba83b7eb [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.579487 12409 tablet_replica.cc:333] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.580462 12409 raft_consensus.cc:2243] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.581096 12409 raft_consensus.cc:2272] T 313fdc28d1e44d6393c8388c74843a28 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.584225 12409 tablet_replica.cc:333] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.585002 12409 raft_consensus.cc:2243] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.585765 12409 raft_consensus.cc:2272] T 32a78ca0f82940f0b3698fceba7b2ce4 P 48da76e971b84219a6ba80d1ba83b7eb [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.589386 12409 tablet_replica.cc:333] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.590207 12409 raft_consensus.cc:2243] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.590957 12409 raft_consensus.cc:2272] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.595100 12409 tablet_replica.cc:333] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.595983 12409 raft_consensus.cc:2243] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.596583 12409 raft_consensus.cc:2272] T 630bae767d5b40dfa02289e9b25f7c15 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.599227 12409 tablet_replica.cc:333] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.600020 12409 raft_consensus.cc:2243] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.600740 12409 raft_consensus.cc:2272] T af7bc64cbbc04798be25ec85d5de3b0d P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.604259 12409 tablet_replica.cc:333] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.605027 12409 raft_consensus.cc:2243] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.605857 12409 raft_consensus.cc:2272] T 111cd0966cde4d87a8a86d90eabf9c0a P 48da76e971b84219a6ba80d1ba83b7eb [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.608775 12409 tablet_replica.cc:333] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.609683 12409 raft_consensus.cc:2243] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.610687 12409 raft_consensus.cc:2272] T 8d2ab233d0664594955baf948686386b P 48da76e971b84219a6ba80d1ba83b7eb [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.613427 12409 tablet_replica.cc:333] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.614351 12409 raft_consensus.cc:2243] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.615049 12409 raft_consensus.cc:2272] T 7296fb4f71a34242af6ae5071d9f508e P 48da76e971b84219a6ba80d1ba83b7eb [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.617748 12409 tablet_replica.cc:333] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.618459 12409 raft_consensus.cc:2243] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.619084 12409 raft_consensus.cc:2272] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 48da76e971b84219a6ba80d1ba83b7eb [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.622135 12409 tablet_replica.cc:333] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.622946 12409 raft_consensus.cc:2243] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:02:04.624121 12409 raft_consensus.cc:2272] T bb22f25deefa4dce94a6ca7ffecaee7c P 48da76e971b84219a6ba80d1ba83b7eb [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.625516 20130 consensus_peers.cc:597] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.627276 12409 tablet_replica.cc:333] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.628218 12409 raft_consensus.cc:2243] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 LEADER]: Raft consensus shutting down.
W20260709 13:02:04.629388 20056 consensus_peers.cc:597] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:04.629416 12409 raft_consensus.cc:2272] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 48da76e971b84219a6ba80d1ba83b7eb [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:04.633044 12409 tablet_replica.cc:333] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb: stopping tablet replica
I20260709 13:02:04.633770 12409 raft_consensus.cc:2243] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:04.634402 12409 raft_consensus.cc:2272] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 48da76e971b84219a6ba80d1ba83b7eb [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:04.655862 20055 consensus_peers.cc:597] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:04.661404 20055 consensus_peers.cc:597] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.674590 20056 consensus_peers.cc:597] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.676327 20055 consensus_peers.cc:597] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.677000 20055 consensus_peers.cc:597] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.677954 20130 consensus_peers.cc:597] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.694329 20130 consensus_peers.cc:597] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.697639 20056 consensus_peers.cc:597] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.721866 20056 consensus_peers.cc:597] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.734735 20130 consensus_peers.cc:597] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.737365 20055 consensus_peers.cc:597] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.756712 20130 consensus_peers.cc:597] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.773851 20056 consensus_peers.cc:597] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.784684 20056 consensus_peers.cc:597] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.785604 20056 consensus_peers.cc:597] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.794941 20130 consensus_peers.cc:597] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.880838 20130 consensus_peers.cc:597] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.897868 20056 consensus_peers.cc:597] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:04.908373 20055 consensus_peers.cc:597] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:04.908990 20055 consensus_peers.cc:597] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:04.939610 20055 consensus_peers.cc:597] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:05.188197 20055 consensus_peers.cc:597] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:05.262377 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
W20260709 13:02:05.282837 20055 consensus_peers.cc:597] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:05.344585 20055 consensus_peers.cc:597] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:05.388903 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
I20260709 13:02:05.397127 25107 raft_consensus.cc:493] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.397847 25107 raft_consensus.cc:515] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:05.400241 25107 leader_election.cc:290] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:02:05.405436 20129 leader_election.cc:336] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:02:05.407204 20130 leader_election.cc:336] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.407692 20130 leader_election.cc:304] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:05.410926 25107 raft_consensus.cc:2749] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:05.479041 20129 consensus_peers.cc:597] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.479728 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:05.480688 12409 tablet_replica.cc:333] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.481597 12409 raft_consensus.cc:2243] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.482118 12409 raft_consensus.cc:2272] T 258312f626c3416b90ed0d3ac81f31e1 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.484493 12409 tablet_replica.cc:333] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.485090 12409 raft_consensus.cc:2243] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.485616 12409 raft_consensus.cc:2272] T 05d19c460b3d4d2cab96fa3f1372b25b P 41af8b41f8a348c5b60ec12d2e323698 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.488828 12409 tablet_replica.cc:333] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.489823 12409 raft_consensus.cc:2243] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.490445 12409 raft_consensus.cc:2272] T e35516f88d604db2ad4a7f67d0017312 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.493132 12409 tablet_replica.cc:333] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.493744 12409 raft_consensus.cc:2243] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.494640 12409 raft_consensus.cc:2272] T ebe7c81078d24dfd8ebcfce8dd265486 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.497186 12409 tablet_replica.cc:333] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.498041 12409 raft_consensus.cc:2243] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.499194 12409 raft_consensus.cc:2272] T e4bbc03ab1f04c8ebbfad0c35770dad1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.501904 12409 tablet_replica.cc:333] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.502770 12409 raft_consensus.cc:2243] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.503696 12409 raft_consensus.cc:2272] T 3e1c1a13f585422ea6d2daf771874d19 P 41af8b41f8a348c5b60ec12d2e323698 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.506780 12409 tablet_replica.cc:333] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.507640 12409 raft_consensus.cc:2243] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.508467 12409 raft_consensus.cc:2272] T 7b9595f271b64d9caeb8e95eb6890e38 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.511533 12409 tablet_replica.cc:333] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
W20260709 13:02:05.512267 20129 consensus_peers.cc:597] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.512543 12409 raft_consensus.cc:2243] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.513329 12409 raft_consensus.cc:2272] T a42b8897c4224305af65104f541be1fc P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.516175 12409 tablet_replica.cc:333] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.517079 12409 raft_consensus.cc:2243] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.517901 12409 raft_consensus.cc:2272] T 77ca1b6b08b74f04948397dece34f882 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.521070 12409 tablet_replica.cc:333] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.522091 12409 raft_consensus.cc:2243] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.523089 12409 raft_consensus.cc:2272] T 62a5f678444241d5a158a35557f9a25b P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.526827 12409 tablet_replica.cc:333] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.527669 12409 raft_consensus.cc:2243] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.529121 12409 raft_consensus.cc:2272] T 4864bdb85f20457e9ef1bfb12f991554 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.532231 12409 tablet_replica.cc:333] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.533231 12409 raft_consensus.cc:2243] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.534049 12409 raft_consensus.cc:2272] T 88eccd78839140bba038b898679a00b0 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.537730 12409 tablet_replica.cc:333] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.538592 12409 raft_consensus.cc:2243] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.539337 12409 raft_consensus.cc:2272] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.542385 12409 tablet_replica.cc:333] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.543222 12409 raft_consensus.cc:2243] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.544142 12409 raft_consensus.cc:2272] T eb6e04fb0be94cccad001ccc48f9bd85 P 41af8b41f8a348c5b60ec12d2e323698 [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:05.545523 20129 consensus_peers.cc:597] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.546989 12409 tablet_replica.cc:333] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.547847 12409 raft_consensus.cc:2243] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.548446 12409 raft_consensus.cc:2272] T ea4e65a94f914c99a52cbf5467a84a13 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.550611 12409 tablet_replica.cc:333] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.551334 12409 raft_consensus.cc:2243] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.551386 25228 raft_consensus.cc:493] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.552073 12409 raft_consensus.cc:2272] T 9d934e44cfcb452b9a7d72dc634e9e77 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.551987 25228 raft_consensus.cc:515] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:05.552572 25217 raft_consensus.cc:493] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.553159 25217 raft_consensus.cc:515] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:05.555274 25107 raft_consensus.cc:493] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.555467 25217 leader_election.cc:290] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:05.555105 12409 tablet_replica.cc:333] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.555989 25107 raft_consensus.cc:515] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:05.562021 12409 raft_consensus.cc:2243] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.563109 12409 raft_consensus.cc:2272] T d8a881d176a0426b9337a6b7c224b88c P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.566120 12409 tablet_replica.cc:333] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.567006 12409 raft_consensus.cc:2243] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.568001 25107 leader_election.cc:290] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
I20260709 13:02:05.568182 12409 raft_consensus.cc:2272] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:05.568236 20130 leader_election.cc:336] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.555437 25228 leader_election.cc:290] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:05.569540 20130 leader_election.cc:336] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
W20260709 13:02:05.571188 20129 leader_election.cc:336] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:05.571859 12409 tablet_replica.cc:333] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.572818 12409 raft_consensus.cc:2243] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.573441 12409 raft_consensus.cc:2272] T 7296fb4f71a34242af6ae5071d9f508e P 41af8b41f8a348c5b60ec12d2e323698 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.571678 20129 leader_election.cc:304] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:05.574302 25228 raft_consensus.cc:2749] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:05.576233 20130 leader_election.cc:336] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
W20260709 13:02:05.577262 20129 leader_election.cc:336] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:05.577759 20129 leader_election.cc:304] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:05.578540 25228 raft_consensus.cc:2749] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:05.578709 12409 tablet_replica.cc:333] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.579594 12409 raft_consensus.cc:2243] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.580274 12409 raft_consensus.cc:2272] T 1c733c25de16421ba3746f8c6de93bd5 P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.583132 12409 tablet_replica.cc:333] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.583737 12409 raft_consensus.cc:2243] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.584328 12409 raft_consensus.cc:2272] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:05.584900 20129 leader_election.cc:336] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:05.585404 20129 leader_election.cc:304] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:05.586216 25228 raft_consensus.cc:2749] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:05.586807 12409 tablet_replica.cc:333] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.587703 12409 raft_consensus.cc:2243] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.588828 12409 raft_consensus.cc:2272] T 5665e7206f7d4ad58380653cdf2490c2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.591621 12409 tablet_replica.cc:333] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.592464 12409 raft_consensus.cc:2243] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.593499 12409 raft_consensus.cc:2272] T 59c71a9c3bb24e9d92023242a19a363d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.595667 12409 tablet_replica.cc:333] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.596293 12409 raft_consensus.cc:2243] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.596773 12409 raft_consensus.cc:2272] T 22d3f15573b24fd9aba34f52879de263 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.598762 12409 tablet_replica.cc:333] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.599323 12409 raft_consensus.cc:2243] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.600248 12409 raft_consensus.cc:2272] T d6f643e855f54a4d8def1836bbdcf39b P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.602304 12409 tablet_replica.cc:333] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.602870 12409 raft_consensus.cc:2243] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.603528 12409 raft_consensus.cc:2272] T 153ccb361669440fa940c70fa6ae8d76 P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.606170 12409 tablet_replica.cc:333] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.607038 12409 raft_consensus.cc:2243] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.608183 12409 raft_consensus.cc:2272] T 6240627eb43d4d039f730bc326fdc93e P 41af8b41f8a348c5b60ec12d2e323698 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.611129 12409 tablet_replica.cc:333] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.611925 12409 raft_consensus.cc:2243] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.613037 12409 raft_consensus.cc:2272] T 0af76b6f84f54d249f7949cf3c104e32 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.615093 12409 tablet_replica.cc:333] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.615864 12409 raft_consensus.cc:2243] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.616852 12409 raft_consensus.cc:2272] T 111cd0966cde4d87a8a86d90eabf9c0a P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.619699 12409 tablet_replica.cc:333] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.620280 12409 raft_consensus.cc:2243] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.621124 12409 raft_consensus.cc:2272] T af7bc64cbbc04798be25ec85d5de3b0d P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.623572 12409 tablet_replica.cc:333] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.624272 12409 raft_consensus.cc:2243] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.625291 12409 raft_consensus.cc:2272] T 6c7a345503524963ae2cf82630978b5a P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:05.626667 20129 consensus_peers.cc:597] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.628046 12409 tablet_replica.cc:333] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.629040 12409 raft_consensus.cc:2243] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.629594 12409 raft_consensus.cc:2272] T f4ea05922db6473289d3a25f7b32e86e P 41af8b41f8a348c5b60ec12d2e323698 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.632484 12409 tablet_replica.cc:333] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.633262 12409 raft_consensus.cc:2243] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.633826 12409 raft_consensus.cc:2272] T 054afb193dbf43d48b1b0ba6ce809ee3 P 41af8b41f8a348c5b60ec12d2e323698 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.636662 12409 tablet_replica.cc:333] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.637216 12409 raft_consensus.cc:2243] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.637672 12409 raft_consensus.cc:2272] T 746d924541574d6086a6ae086476a1e7 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.640059 12409 tablet_replica.cc:333] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.640851 12409 raft_consensus.cc:2243] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.641906 12409 raft_consensus.cc:2272] T f761e6c8cc73410eb153cfbcb79afb90 P 41af8b41f8a348c5b60ec12d2e323698 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.645105 12409 tablet_replica.cc:333] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.645692 12409 raft_consensus.cc:2243] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.646692 12409 raft_consensus.cc:2272] T 81fd99f42a47497f9d4372157320611b P 41af8b41f8a348c5b60ec12d2e323698 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.647948 25230 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.648526 25230 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:05.649221 12409 tablet_replica.cc:333] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.649879 12409 raft_consensus.cc:2243] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.650466 12409 raft_consensus.cc:2272] T fac4d79f682d4f169331b9656bef55fe P 41af8b41f8a348c5b60ec12d2e323698 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.653283 12409 tablet_replica.cc:333] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.654126 12409 raft_consensus.cc:2243] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.655193 12409 raft_consensus.cc:2272] T 313fdc28d1e44d6393c8388c74843a28 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:05.657861 20056 leader_election.cc:336] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.658169 12409 tablet_replica.cc:333] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.659013 12409 raft_consensus.cc:2243] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.660094 12409 raft_consensus.cc:2272] T 32a78ca0f82940f0b3698fceba7b2ce4 P 41af8b41f8a348c5b60ec12d2e323698 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.662819 12409 tablet_replica.cc:333] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.663514 12409 raft_consensus.cc:2243] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.664547 12409 raft_consensus.cc:2272] T 25b25157ae9c4e9d87d2ff89c2c0c4dd P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.666023 25230 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 8b5b4e2a92b046d096f2661c54321d41 (127.12.30.68:46223)
I20260709 13:02:05.667014 20156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dadb6b575d44e06ba9ab0804b1d51c6" candidate_uuid: "41af8b41f8a348c5b60ec12d2e323698" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8b5b4e2a92b046d096f2661c54321d41" is_pre_election: true
I20260709 13:02:05.667486 12409 tablet_replica.cc:333] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.669341 12409 raft_consensus.cc:2243] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.670544 12409 raft_consensus.cc:2272] T 630bae767d5b40dfa02289e9b25f7c15 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.669359 20056 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41af8b41f8a348c5b60ec12d2e323698; no voters: 48da76e971b84219a6ba80d1ba83b7eb, 8b5b4e2a92b046d096f2661c54321d41
I20260709 13:02:05.671991 25230 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:02:05.673204 12409 tablet_replica.cc:333] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.674180 12409 raft_consensus.cc:2243] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:05.674829 12409 raft_consensus.cc:2272] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 41af8b41f8a348c5b60ec12d2e323698 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.677834 12409 tablet_replica.cc:333] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.678524 12409 raft_consensus.cc:2243] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:02:05.678855 20129 consensus_peers.cc:597] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.679675 12409 raft_consensus.cc:2272] T d94fa4e16c4f424c86e23d82c9408473 P 41af8b41f8a348c5b60ec12d2e323698 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.681780 25228 raft_consensus.cc:493] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.682173 12409 tablet_replica.cc:333] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698: stopping tablet replica
I20260709 13:02:05.682407 25228 raft_consensus.cc:515] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:05.682941 12409 raft_consensus.cc:2243] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:02:05.684051 12409 raft_consensus.cc:2272] T 6eb7d330cd8b4195aba1f8c1348bfcc2 P 41af8b41f8a348c5b60ec12d2e323698 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:05.686777 25228 leader_election.cc:290] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:05.688486 20129 consensus_peers.cc:597] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:05.697171 20130 leader_election.cc:336] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
W20260709 13:02:05.701419 20129 leader_election.cc:336] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:05.702066 20129 leader_election.cc:304] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:05.702991 25228 raft_consensus.cc:2749] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:05.724570 20129 consensus_peers.cc:597] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:05.768184 20129 connection.cc:769] client connection to 127.12.30.67:39831 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:02:05.769888 25228 raft_consensus.cc:493] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.770490 25228 raft_consensus.cc:515] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:02:05.772502 20129 consensus_peers.cc:597] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.772905 25228 leader_election.cc:290] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:02:05.773031 20129 consensus_peers.cc:597] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:05.777150 20130 leader_election.cc:336] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.777678 25228 raft_consensus.cc:493] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.778455 25228 raft_consensus.cc:515] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:05.781996 25228 leader_election.cc:290] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:02:05.785230 20129 leader_election.cc:336] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
I20260709 13:02:05.786128 20129 leader_election.cc:304] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:05.787289 25228 raft_consensus.cc:2749] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:05.789748 20129 leader_election.cc:336] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
W20260709 13:02:05.790882 20130 leader_election.cc:336] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.791296 20130 leader_election.cc:304] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:05.792191 25228 raft_consensus.cc:2749] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:05.859843 25228 raft_consensus.cc:493] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.860430 25228 raft_consensus.cc:515] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } } peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } }
I20260709 13:02:05.862856 25228 leader_election.cc:290] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607), 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003)
W20260709 13:02:05.874011 20129 leader_election.cc:336] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
W20260709 13:02:05.878257 20130 leader_election.cc:336] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.878786 20130 leader_election.cc:304] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 48da76e971b84219a6ba80d1ba83b7eb, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:05.880376 25107 raft_consensus.cc:2749] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:05.889269 20129 consensus_peers.cc:597] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:05.895318 25107 raft_consensus.cc:493] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.896095 25107 raft_consensus.cc:515] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
W20260709 13:02:05.907173 20130 leader_election.cc:336] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.908221 25107 leader_election.cc:290] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:02:05.911638 20129 consensus_peers.cc:597] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Couldn't send request to peer 41af8b41f8a348c5b60ec12d2e323698. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:05.914043 20129 leader_election.cc:336] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
I20260709 13:02:05.914463 20129 leader_election.cc:304] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:05.915148 25107 raft_consensus.cc:2749] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:05.933671 20129 consensus_peers.cc:597] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:05.942612 25107 raft_consensus.cc:493] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.943264 25107 raft_consensus.cc:515] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:05.945482 25107 leader_election.cc:290] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:02:05.950516 20130 leader_election.cc:336] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
W20260709 13:02:05.951470 20129 leader_election.cc:336] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
I20260709 13:02:05.951874 20129 leader_election.cc:304] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:05.952615 25107 raft_consensus.cc:2749] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.989471 25107 raft_consensus.cc:493] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 48da76e971b84219a6ba80d1ba83b7eb)
I20260709 13:02:05.990056 25107 raft_consensus.cc:515] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:05.992964 25107 leader_election.cc:290] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
W20260709 13:02:05.997874 20129 leader_election.cc:336] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
W20260709 13:02:05.998373 20130 leader_election.cc:336] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:05.998822 20130 leader_election.cc:304] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:06.048377 25107 raft_consensus.cc:2749] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:02:06.329047 20129 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:02:06.378875 20129 consensus_peers.cc:597] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:02:06.379899 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
W20260709 13:02:06.452728 20129 consensus_peers.cc:597] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:02:06.468997 20130 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:06.519802 20129 consensus_peers.cc:597] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:06.576385 20129 consensus_peers.cc:597] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:02:06.579221 12409 tablet_server.cc:179] TabletServer@127.12.30.68:0 shutting down...
W20260709 13:02:06.615357 20130 consensus_peers.cc:597] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:06.638384 20129 consensus_peers.cc:597] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:02:06.657758 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:06.659019 12409 tablet_replica.cc:333] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.660537 12409 raft_consensus.cc:2243] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.661180 25217 raft_consensus.cc:493] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:02:06.661617 12409 raft_consensus.cc:2272] T 258312f626c3416b90ed0d3ac81f31e1 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.662165 25217 raft_consensus.cc:515] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:06.665829 25217 leader_election.cc:290] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:06.666100 12409 tablet_replica.cc:333] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.667881 12409 raft_consensus.cc:2243] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.671591 12409 raft_consensus.cc:2272] T ea4e65a94f914c99a52cbf5467a84a13 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:06.675423 20130 consensus_peers.cc:597] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:06.676730 12409 tablet_replica.cc:333] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.678071 12409 raft_consensus.cc:2243] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:02:06.678833 20129 leader_election.cc:336] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
W20260709 13:02:06.679807 20129 leader_election.cc:336] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:06.679822 12409 raft_consensus.cc:2272] T 9d934e44cfcb452b9a7d72dc634e9e77 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.680508 20129 leader_election.cc:304] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
W20260709 13:02:06.682932 20129 consensus_peers.cc:597] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:06.683472 25228 raft_consensus.cc:2749] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:06.684041 12409 tablet_replica.cc:333] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.685204 12409 raft_consensus.cc:2243] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.686676 12409 raft_consensus.cc:2272] T 05d19c460b3d4d2cab96fa3f1372b25b P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.689560 12409 tablet_replica.cc:333] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.690372 12409 raft_consensus.cc:2243] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.691574 12409 raft_consensus.cc:2272] T fc68b8861f1d4bf09ec026e1ba26cdf6 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.694396 12409 tablet_replica.cc:333] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.695094 12409 raft_consensus.cc:2243] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.696250 12409 raft_consensus.cc:2272] T e35516f88d604db2ad4a7f67d0017312 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.699069 12409 tablet_replica.cc:333] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.700054 12409 raft_consensus.cc:2243] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.700788 12409 raft_consensus.cc:2272] T 4c36c61154404ebebcf309833a8bf5ea P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.703774 12409 tablet_replica.cc:333] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.704581 12409 raft_consensus.cc:2243] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.706000 12409 raft_consensus.cc:2272] T 5acf254dbaee44d1bc844da86793ef92 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.709453 12409 tablet_replica.cc:333] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.710249 12409 raft_consensus.cc:2243] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.711735 12409 raft_consensus.cc:2272] T b26fb84490474f98b31c0593ddf1b013 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.714684 12409 tablet_replica.cc:333] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.715395 12409 raft_consensus.cc:2243] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.716390 12409 raft_consensus.cc:2272] T a42b8897c4224305af65104f541be1fc P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.719012 12409 tablet_replica.cc:333] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.719877 12409 raft_consensus.cc:2243] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.721187 12409 raft_consensus.cc:2272] T 4ad5e6c5a68d4b07b3d5830de6d5be23 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.725052 12409 tablet_replica.cc:333] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
W20260709 13:02:06.725543 20129 consensus_peers.cc:597] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:06.725981 12409 raft_consensus.cc:2243] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.727712 12409 raft_consensus.cc:2272] T eb6e04fb0be94cccad001ccc48f9bd85 P 8b5b4e2a92b046d096f2661c54321d41 [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:06.729400 20130 consensus_peers.cc:597] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:06.730909 12409 tablet_replica.cc:333] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.731683 12409 raft_consensus.cc:2243] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.733017 12409 raft_consensus.cc:2272] T 77ca1b6b08b74f04948397dece34f882 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.736218 12409 tablet_replica.cc:333] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.737161 12409 raft_consensus.cc:2243] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:02:06.737846 20129 consensus_peers.cc:597] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 -> Peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Couldn't send request to peer e5eb27db496b47129df3e689e546d94a. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:06.737978 12409 raft_consensus.cc:2272] T 2d767704576e43bb9f6bcd864fa9a6fe P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.740993 12409 tablet_replica.cc:333] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.741734 12409 raft_consensus.cc:2243] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.742383 12409 raft_consensus.cc:2272] T 4864bdb85f20457e9ef1bfb12f991554 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.744071 25228 raft_consensus.cc:493] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:02:06.745491 12409 tablet_replica.cc:333] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.744920 25228 raft_consensus.cc:515] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:06.746330 12409 raft_consensus.cc:2243] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.747943 12409 raft_consensus.cc:2272] T 4b58ddc27cb54fd28e2324df51a5ff24 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.748358 25228 leader_election.cc:290] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:06.751057 25228 raft_consensus.cc:493] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:02:06.751816 12409 tablet_replica.cc:333] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.752499 25228 raft_consensus.cc:515] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } } peers { permanent_uuid: "e5eb27db496b47129df3e689e546d94a" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 38607 } }
I20260709 13:02:06.753067 12409 raft_consensus.cc:2243] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:02:06.754771 20129 leader_election.cc:336] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
I20260709 13:02:06.755831 12409 raft_consensus.cc:2272] T 2f8749de35944e3d92b86f7d4c472de2 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:06.756219 20130 leader_election.cc:336] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:06.756811 20130 leader_election.cc:304] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:06.757630 25228 leader_election.cc:290] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831), e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607)
I20260709 13:02:06.759408 12409 tablet_replica.cc:333] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.760042 25217 raft_consensus.cc:2749] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.760835 12409 raft_consensus.cc:2243] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.762068 12409 raft_consensus.cc:2272] T bde87ca1d98f4bd8b90d7f1502cc67f1 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:06.763844 20129 leader_election.cc:336] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
W20260709 13:02:06.764602 20129 leader_election.cc:336] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e5eb27db496b47129df3e689e546d94a (127.12.30.65:38607): Network error: Client connection negotiation failed: client connection to 127.12.30.65:38607: connect: Connection refused (error 111)
I20260709 13:02:06.764966 20129 leader_election.cc:304] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, e5eb27db496b47129df3e689e546d94a
I20260709 13:02:06.765211 12409 tablet_replica.cc:333] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.765718 25228 raft_consensus.cc:2749] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:06.766031 12409 raft_consensus.cc:2243] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.766825 12409 raft_consensus.cc:2272] T 0af76b6f84f54d249f7949cf3c104e32 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.769528 12409 tablet_replica.cc:333] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.770129 12409 raft_consensus.cc:2243] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.771311 12409 raft_consensus.cc:2272] T 2fd1b4e7346b47ffaa37a1f24e287d3a P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.774655 12409 tablet_replica.cc:333] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.775580 12409 raft_consensus.cc:2243] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.777235 12409 raft_consensus.cc:2272] T 1c733c25de16421ba3746f8c6de93bd5 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.780584 12409 tablet_replica.cc:333] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.781448 12409 raft_consensus.cc:2243] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.783414 12409 raft_consensus.cc:2272] T 9a2cace87d4c4d26be2f8bffdb5b6d50 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.787384 12409 tablet_replica.cc:333] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.788476 12409 raft_consensus.cc:2243] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.789419 12409 raft_consensus.cc:2272] T 5665e7206f7d4ad58380653cdf2490c2 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.793321 12409 tablet_replica.cc:333] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.794101 12409 raft_consensus.cc:2243] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.795034 12409 raft_consensus.cc:2272] T 59c71a9c3bb24e9d92023242a19a363d P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.798733 12409 tablet_replica.cc:333] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.799793 12409 raft_consensus.cc:2243] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.800861 12409 raft_consensus.cc:2272] T eb11384495c2485aaab243a655f75c86 P 8b5b4e2a92b046d096f2661c54321d41 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.804520 12409 tablet_replica.cc:333] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.805473 12409 raft_consensus.cc:2243] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.807029 12409 raft_consensus.cc:2272] T 6c7a345503524963ae2cf82630978b5a P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:06.808441 20130 consensus_peers.cc:597] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 -> Peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Couldn't send request to peer 48da76e971b84219a6ba80d1ba83b7eb. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:02:06.810372 12409 tablet_replica.cc:333] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.811379 12409 raft_consensus.cc:2243] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.812849 12409 raft_consensus.cc:2272] T 2fe4ca3a058c431f8e6ecbfc90468313 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.816412 12409 tablet_replica.cc:333] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.817257 12409 raft_consensus.cc:2243] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.818662 12409 raft_consensus.cc:2272] T 45dd40d980c94ccdab3d167b33984c44 P 8b5b4e2a92b046d096f2661c54321d41 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.821913 12409 tablet_replica.cc:333] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.822800 12409 raft_consensus.cc:2243] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.823674 12409 raft_consensus.cc:2272] T 6240627eb43d4d039f730bc326fdc93e P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.828223 12409 tablet_replica.cc:333] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.829169 12409 raft_consensus.cc:2243] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.830197 12409 raft_consensus.cc:2272] T bb22f25deefa4dce94a6ca7ffecaee7c P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.833690 12409 tablet_replica.cc:333] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.834602 12409 raft_consensus.cc:2243] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.835775 12409 raft_consensus.cc:2272] T 8d2ab233d0664594955baf948686386b P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.839375 12409 tablet_replica.cc:333] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.840276 12409 raft_consensus.cc:2243] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.841957 12409 raft_consensus.cc:2272] T 7296fb4f71a34242af6ae5071d9f508e P 8b5b4e2a92b046d096f2661c54321d41 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.846454 12409 tablet_replica.cc:333] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.847400 12409 raft_consensus.cc:2243] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.848596 12409 raft_consensus.cc:2272] T ebe7c81078d24dfd8ebcfce8dd265486 P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.852020 12409 tablet_replica.cc:333] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.852959 12409 raft_consensus.cc:2243] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.854317 12409 raft_consensus.cc:2272] T 4ccd50eb99764ffc91bd06202fd0a5e7 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.857849 12409 tablet_replica.cc:333] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.858660 12409 raft_consensus.cc:2243] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.860383 12409 raft_consensus.cc:2272] T 8b1a8f437ee9497c987ac7a6206d2338 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.863885 12409 tablet_replica.cc:333] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.865070 12409 raft_consensus.cc:2243] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.867157 12409 raft_consensus.cc:2272] T 054afb193dbf43d48b1b0ba6ce809ee3 P 8b5b4e2a92b046d096f2661c54321d41 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.870486 12409 tablet_replica.cc:333] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.871383 12409 raft_consensus.cc:2243] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.872237 12409 raft_consensus.cc:2272] T 22d3f15573b24fd9aba34f52879de263 P 8b5b4e2a92b046d096f2661c54321d41 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.875491 12409 tablet_replica.cc:333] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.875759 25228 raft_consensus.cc:493] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:02:06.876457 12409 raft_consensus.cc:2243] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.876498 25228 raft_consensus.cc:515] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:06.877482 12409 raft_consensus.cc:2272] T d6f643e855f54a4d8def1836bbdcf39b P 8b5b4e2a92b046d096f2661c54321d41 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.879127 25217 raft_consensus.cc:493] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 41af8b41f8a348c5b60ec12d2e323698)
I20260709 13:02:06.879722 25217 raft_consensus.cc:515] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48da76e971b84219a6ba80d1ba83b7eb" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 39003 } } peers { permanent_uuid: "41af8b41f8a348c5b60ec12d2e323698" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 39831 } } peers { permanent_uuid: "8b5b4e2a92b046d096f2661c54321d41" member_type: VOTER last_known_addr { host: "127.12.30.68" port: 46223 } }
I20260709 13:02:06.880329 25228 leader_election.cc:290] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:06.880877 12409 tablet_replica.cc:333] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.882020 12409 raft_consensus.cc:2243] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.883329 12409 raft_consensus.cc:2272] T 153ccb361669440fa940c70fa6ae8d76 P 8b5b4e2a92b046d096f2661c54321d41 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.882087 25217 leader_election.cc:290] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003), 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831)
I20260709 13:02:06.886947 12409 tablet_replica.cc:333] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.888054 12409 raft_consensus.cc:2243] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Raft consensus shutting down.
W20260709 13:02:06.888417 20129 leader_election.cc:336] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
I20260709 13:02:06.889091 12409 raft_consensus.cc:2272] T 630bae767d5b40dfa02289e9b25f7c15 P 8b5b4e2a92b046d096f2661c54321d41 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:06.889133 20129 leader_election.cc:336] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 41af8b41f8a348c5b60ec12d2e323698 (127.12.30.67:39831): Network error: Client connection negotiation failed: client connection to 127.12.30.67:39831: connect: Connection refused (error 111)
W20260709 13:02:06.890470 20130 leader_election.cc:336] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:06.890945 20130 leader_election.cc:304] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:06.891798 25217 raft_consensus.cc:2749] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:06.891947 20130 leader_election.cc:336] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 48da76e971b84219a6ba80d1ba83b7eb (127.12.30.66:39003): Network error: Client connection negotiation failed: client connection to 127.12.30.66:39003: connect: Connection refused (error 111)
I20260709 13:02:06.892583 12409 tablet_replica.cc:333] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.892458 20130 leader_election.cc:304] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5b4e2a92b046d096f2661c54321d41; no voters: 41af8b41f8a348c5b60ec12d2e323698, 48da76e971b84219a6ba80d1ba83b7eb
I20260709 13:02:06.893409 12409 raft_consensus.cc:2243] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.893927 25217 raft_consensus.cc:2706] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Received pre-election callback for term 18 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 13:02:06.894593 12409 raft_consensus.cc:2272] T 32a78ca0f82940f0b3698fceba7b2ce4 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.897150 12409 tablet_replica.cc:333] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.898156 12409 raft_consensus.cc:2243] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:02:06.899628 12409 raft_consensus.cc:2272] T fac4d79f682d4f169331b9656bef55fe P 8b5b4e2a92b046d096f2661c54321d41 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.902995 12409 tablet_replica.cc:333] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.903671 12409 raft_consensus.cc:2243] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.904644 12409 raft_consensus.cc:2272] T 313fdc28d1e44d6393c8388c74843a28 P 8b5b4e2a92b046d096f2661c54321d41 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:06.907316 12409 tablet_replica.cc:333] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41: stopping tablet replica
I20260709 13:02:06.907826 12409 raft_consensus.cc:2243] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:06.908428 12409 raft_consensus.cc:2272] T 9dadb6b575d44e06ba9ab0804b1d51c6 P 8b5b4e2a92b046d096f2661c54321d41 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:07.425724 12409 tablet_server.cc:196] TabletServer@127.12.30.68:0 shutdown complete.
I20260709 13:02:07.517300 12409 master.cc:562] Master@127.12.30.126:46523 shutting down...
I20260709 13:02:07.575423 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:07.576383 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:07.576905 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P d84e5f78c3264c738b7966fc230d4878: stopping tablet replica
I20260709 13:02:07.791558 12409 master.cc:584] Master@127.12.30.126:46523 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (161056 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:02:07.901589 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:42253
I20260709 13:02:07.902951 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:07.909379 25238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:07.909708 25239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:07.910490 25241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:07.912369 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:07.915285 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:07.915501 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:07.915683 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602127915666 us; error 0 us; skew 500 ppm
I20260709 13:02:07.916319 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:07.919131 12409 webserver.cc:533] Webserver started at http://127.12.30.126:39875/ using document root <none> and password file <none>
I20260709 13:02:07.919687 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:07.919891 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:07.920293 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:07.921638 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "2eabe16999d147e6a8d4ea80b3054552"
format_stamp: "Formatted at 2026-07-09 13:02:07 on dist-test-slave-mlt7"
I20260709 13:02:07.926400 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:02:07.930661 25246 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:07.931440 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:02:07.931777 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "2eabe16999d147e6a8d4ea80b3054552"
format_stamp: "Formatted at 2026-07-09 13:02:07 on dist-test-slave-mlt7"
I20260709 13:02:07.932158 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:07.942883 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:07.944453 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:07.985716 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:42253
I20260709 13:02:07.985807 25297 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:42253 every 8 connection(s)
I20260709 13:02:07.990312 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.002204 25298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552: Bootstrap starting.
I20260709 13:02:08.007376 25298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.011729 25298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552: No bootstrap required, opened a new log
I20260709 13:02:08.014294 25298 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eabe16999d147e6a8d4ea80b3054552" member_type: VOTER }
I20260709 13:02:08.014707 25298 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.014901 25298 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eabe16999d147e6a8d4ea80b3054552, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.015441 25298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eabe16999d147e6a8d4ea80b3054552" member_type: VOTER }
I20260709 13:02:08.015882 25298 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:02:08.016181 25298 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:02:08.016407 25298 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.021512 25298 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eabe16999d147e6a8d4ea80b3054552" member_type: VOTER }
I20260709 13:02:08.022022 25298 leader_election.cc:304] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eabe16999d147e6a8d4ea80b3054552; no voters: 
I20260709 13:02:08.023159 25298 leader_election.cc:290] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:02:08.023766 25301 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:08.025282 25301 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 1 LEADER]: Becoming Leader. State: Replica: 2eabe16999d147e6a8d4ea80b3054552, State: Running, Role: LEADER
I20260709 13:02:08.026019 25301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eabe16999d147e6a8d4ea80b3054552" member_type: VOTER }
I20260709 13:02:08.026474 25298 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:02:08.031540 25302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eabe16999d147e6a8d4ea80b3054552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eabe16999d147e6a8d4ea80b3054552" member_type: VOTER } }
I20260709 13:02:08.031759 25303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eabe16999d147e6a8d4ea80b3054552. Latest consensus state: current_term: 1 leader_uuid: "2eabe16999d147e6a8d4ea80b3054552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eabe16999d147e6a8d4ea80b3054552" member_type: VOTER } }
I20260709 13:02:08.032415 25302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:08.032437 25303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:08.050081 25309 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:02:08.055877 25309 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:02:08.059211 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:02:08.066191 25309 catalog_manager.cc:1383] Generated new cluster ID: 87d1f374244b4c6591024e8a4b0c5b51
I20260709 13:02:08.066542 25309 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:02:08.088527 25309 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:02:08.089818 25309 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:02:08.105050 25309 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552: Generated new TSK 0
I20260709 13:02:08.105674 25309 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:02:08.126711 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:08.133842 25319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:08.135098 25320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:08.139292 25322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:08.139667 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:08.142473 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:08.142685 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:08.142894 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602128142877 us; error 0 us; skew 500 ppm
I20260709 13:02:08.143591 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:08.146353 12409 webserver.cc:533] Webserver started at http://127.12.30.65:35515/ using document root <none> and password file <none>
I20260709 13:02:08.146936 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:08.147140 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:08.147454 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:08.148842 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "e998827be8264d1d8681d5bfc1277e16"
format_stamp: "Formatted at 2026-07-09 13:02:08 on dist-test-slave-mlt7"
I20260709 13:02:08.154019 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:02:08.157878 25327 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.158775 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:02:08.159138 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "e998827be8264d1d8681d5bfc1277e16"
format_stamp: "Formatted at 2026-07-09 13:02:08 on dist-test-slave-mlt7"
I20260709 13:02:08.159483 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:08.184926 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:08.186590 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:08.188316 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:08.190714 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:08.190950 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.191219 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:08.191414 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.235395 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:35871
I20260709 13:02:08.235512 25389 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:35871 every 8 connection(s)
I20260709 13:02:08.240839 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:08.248389 25394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:08.252935 25395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:08.256316 25390 heartbeater.cc:344] Connected to a master server at 127.12.30.126:42253
I20260709 13:02:08.256549 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:08.256924 25390 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:08.257781 25390 heartbeater.cc:507] Master 127.12.30.126:42253 requested a full tablet report, sending...
W20260709 13:02:08.257797 25397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:08.260645 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:02:08.260886 25263 ts_manager.cc:194] Registered new tserver with Master: e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871)
W20260709 13:02:08.261107 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:08.261518 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602128261493 us; error 0 us; skew 500 ppm
I20260709 13:02:08.262727 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:08.264165 25263 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42896
I20260709 13:02:08.266517 12409 webserver.cc:533] Webserver started at http://127.12.30.66:39541/ using document root <none> and password file <none>
I20260709 13:02:08.267287 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:08.267536 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:08.267845 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:08.269100 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "815ff4fecd7844c2bfb4864215f033cf"
format_stamp: "Formatted at 2026-07-09 13:02:08 on dist-test-slave-mlt7"
I20260709 13:02:08.274657 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:02:08.279023 25402 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.279857 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:02:08.280267 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "815ff4fecd7844c2bfb4864215f033cf"
format_stamp: "Formatted at 2026-07-09 13:02:08 on dist-test-slave-mlt7"
I20260709 13:02:08.280607 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:08.297973 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:08.299268 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:08.300932 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:08.303521 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:08.303767 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.304041 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:08.304225 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.352291 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:37721
I20260709 13:02:08.352334 25464 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:37721 every 8 connection(s)
I20260709 13:02:08.358434 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:08.367570 25468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:08.370325 25469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:08.374042 12409 server_base.cc:1061] running on GCE node
W20260709 13:02:08.374394 25471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:08.375027 25465 heartbeater.cc:344] Connected to a master server at 127.12.30.126:42253
I20260709 13:02:08.375454 25465 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:08.375545 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:08.375962 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:08.376158 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602128376142 us; error 0 us; skew 500 ppm
I20260709 13:02:08.376318 25465 heartbeater.cc:507] Master 127.12.30.126:42253 requested a full tablet report, sending...
I20260709 13:02:08.376873 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:08.378862 25263 ts_manager.cc:194] Registered new tserver with Master: 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:08.379989 12409 webserver.cc:533] Webserver started at http://127.12.30.67:41755/ using document root <none> and password file <none>
I20260709 13:02:08.380788 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:08.381042 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:08.380925 25263 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42900
I20260709 13:02:08.381486 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:08.383070 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "7efd98fd5c384fa09947aac9da4b776b"
format_stamp: "Formatted at 2026-07-09 13:02:08 on dist-test-slave-mlt7"
I20260709 13:02:08.387996 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:02:08.391808 25476 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.392885 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:02:08.393249 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "7efd98fd5c384fa09947aac9da4b776b"
format_stamp: "Formatted at 2026-07-09 13:02:08 on dist-test-slave-mlt7"
I20260709 13:02:08.393612 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:08.413352 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:08.414661 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:08.416283 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:08.418679 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:08.418946 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.419190 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:08.419360 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:08.474102 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:41807
I20260709 13:02:08.474226 25538 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:41807 every 8 connection(s)
I20260709 13:02:08.492050 25539 heartbeater.cc:344] Connected to a master server at 127.12.30.126:42253
I20260709 13:02:08.492749 25539 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:08.493507 25539 heartbeater.cc:507] Master 127.12.30.126:42253 requested a full tablet report, sending...
I20260709 13:02:08.495959 25263 ts_manager.cc:194] Registered new tserver with Master: 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:08.496474 12409 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018015699s
I20260709 13:02:08.497953 25263 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42914
I20260709 13:02:08.530126 25263 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42916:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:08.533154 25263 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:08.611042 25355 tablet_service.cc:1511] Processing CreateTablet for tablet e367fc7664aa486fbbb057c89d54508f (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:02:08.612584 25355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e367fc7664aa486fbbb057c89d54508f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.612493 25504 tablet_service.cc:1511] Processing CreateTablet for tablet e367fc7664aa486fbbb057c89d54508f (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:02:08.613281 25354 tablet_service.cc:1511] Processing CreateTablet for tablet 7c16a4a39cd54a9eb36628706ea642e3 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:02:08.614573 25504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e367fc7664aa486fbbb057c89d54508f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.614708 25354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c16a4a39cd54a9eb36628706ea642e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.636574 25503 tablet_service.cc:1511] Processing CreateTablet for tablet 7c16a4a39cd54a9eb36628706ea642e3 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:02:08.639019 25503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c16a4a39cd54a9eb36628706ea642e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.642498 25354 tablet_service.cc:1511] Processing CreateTablet for tablet 56cfdecb47214faaa68fb5d2712780d7 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:02:08.644109 25354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cfdecb47214faaa68fb5d2712780d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.653453 25554 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:08.655521 25502 tablet_service.cc:1511] Processing CreateTablet for tablet 56cfdecb47214faaa68fb5d2712780d7 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:02:08.657445 25502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cfdecb47214faaa68fb5d2712780d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.657842 25353 tablet_service.cc:1511] Processing CreateTablet for tablet fd04fac15ca04d8db197a7b2f5b03d1c (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:02:08.659586 25353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd04fac15ca04d8db197a7b2f5b03d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.661969 25501 tablet_service.cc:1511] Processing CreateTablet for tablet fd04fac15ca04d8db197a7b2f5b03d1c (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:02:08.663807 25501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd04fac15ca04d8db197a7b2f5b03d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.665239 25352 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e606040784cb3af6133607f421fb4 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:02:08.671231 25352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e606040784cb3af6133607f421fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.671696 25430 tablet_service.cc:1511] Processing CreateTablet for tablet e367fc7664aa486fbbb057c89d54508f (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:02:08.673172 25430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e367fc7664aa486fbbb057c89d54508f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.677124 25429 tablet_service.cc:1511] Processing CreateTablet for tablet 7c16a4a39cd54a9eb36628706ea642e3 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:02:08.678737 25429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c16a4a39cd54a9eb36628706ea642e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.678366 25426 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e606040784cb3af6133607f421fb4 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:02:08.679256 25554 tablet_bootstrap.cc:654] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.679798 25426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e606040784cb3af6133607f421fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.681061 25428 tablet_service.cc:1511] Processing CreateTablet for tablet 56cfdecb47214faaa68fb5d2712780d7 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:02:08.684074 25428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cfdecb47214faaa68fb5d2712780d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.677134 25427 tablet_service.cc:1511] Processing CreateTablet for tablet fd04fac15ca04d8db197a7b2f5b03d1c (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:02:08.688263 25427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd04fac15ca04d8db197a7b2f5b03d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.697193 25500 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e606040784cb3af6133607f421fb4 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:02:08.698865 25500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e606040784cb3af6133607f421fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.705976 25355 tablet_service.cc:1511] Processing CreateTablet for tablet 35cac1690929460f81c7dda80e26cf37 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:02:08.707592 25355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cac1690929460f81c7dda80e26cf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.721812 25555 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:08.725716 25554 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:08.726452 25554 ts_tablet_manager.cc:1403] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.005s
I20260709 13:02:08.735550 25425 tablet_service.cc:1511] Processing CreateTablet for tablet 35cac1690929460f81c7dda80e26cf37 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:02:08.737136 25425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cac1690929460f81c7dda80e26cf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.736147 25554 raft_consensus.cc:359] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:08.747429 25554 raft_consensus.cc:385] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.747906 25554 raft_consensus.cc:740] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.749112 25554 consensus_queue.cc:260] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:08.746891 25555 tablet_bootstrap.cc:654] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.758576 25501 tablet_service.cc:1511] Processing CreateTablet for tablet 35cac1690929460f81c7dda80e26cf37 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:02:08.760386 25501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cac1690929460f81c7dda80e26cf37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.764386 25390 heartbeater.cc:499] Master 127.12.30.126:42253 was elected leader, sending a full tablet report...
I20260709 13:02:08.771173 25355 tablet_service.cc:1511] Processing CreateTablet for tablet db0af08988d44a99a14043e9b39508cf (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:02:08.772562 25355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0af08988d44a99a14043e9b39508cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.776969 25354 tablet_service.cc:1511] Processing CreateTablet for tablet 1caefa4abb3d435db0299d2ffadf77f5 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:02:08.778430 25354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1caefa4abb3d435db0299d2ffadf77f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.781628 25502 tablet_service.cc:1511] Processing CreateTablet for tablet 1caefa4abb3d435db0299d2ffadf77f5 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:02:08.783869 25502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1caefa4abb3d435db0299d2ffadf77f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.783847 25500 tablet_service.cc:1511] Processing CreateTablet for tablet db0af08988d44a99a14043e9b39508cf (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:02:08.784188 25429 tablet_service.cc:1511] Processing CreateTablet for tablet 1caefa4abb3d435db0299d2ffadf77f5 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:02:08.785359 25500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0af08988d44a99a14043e9b39508cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.789101 25353 tablet_service.cc:1511] Processing CreateTablet for tablet b87c17d2fac24478924ba677e41fdc68 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:02:08.790532 25353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87c17d2fac24478924ba677e41fdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.794149 25555 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:08.786130 25554 ts_tablet_manager.cc:1434] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.005s
I20260709 13:02:08.796497 25554 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:08.797034 25429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1caefa4abb3d435db0299d2ffadf77f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.801410 25559 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:08.804994 25425 tablet_service.cc:1511] Processing CreateTablet for tablet db0af08988d44a99a14043e9b39508cf (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:02:08.808863 25425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0af08988d44a99a14043e9b39508cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.812610 25428 tablet_service.cc:1511] Processing CreateTablet for tablet b87c17d2fac24478924ba677e41fdc68 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:02:08.814193 25428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87c17d2fac24478924ba677e41fdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.794735 25555 ts_tablet_manager.cc:1403] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.004s
I20260709 13:02:08.820174 25554 tablet_bootstrap.cc:654] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.819895 25555 raft_consensus.cc:359] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:08.824198 25352 tablet_service.cc:1511] Processing CreateTablet for tablet 39e51fa0d7af4f7ba2f0e1c4f1362152 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:02:08.828260 25559 tablet_bootstrap.cc:654] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.829439 25352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e51fa0d7af4f7ba2f0e1c4f1362152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.833240 25555 raft_consensus.cc:385] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.834674 25555 raft_consensus.cc:740] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.837020 25426 tablet_service.cc:1511] Processing CreateTablet for tablet 39e51fa0d7af4f7ba2f0e1c4f1362152 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:02:08.825023 25502 tablet_service.cc:1511] Processing CreateTablet for tablet b87c17d2fac24478924ba677e41fdc68 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:02:08.840893 25426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e51fa0d7af4f7ba2f0e1c4f1362152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.845086 25555 consensus_queue.cc:260] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:08.847836 25502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87c17d2fac24478924ba677e41fdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.852674 25554 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:08.853439 25554 ts_tablet_manager.cc:1403] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.057s	user 0.007s	sys 0.005s
I20260709 13:02:08.860469 25539 heartbeater.cc:499] Master 127.12.30.126:42253 was elected leader, sending a full tablet report...
I20260709 13:02:08.860689 25559 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:08.861357 25559 ts_tablet_manager.cc:1403] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.061s	user 0.023s	sys 0.013s
I20260709 13:02:08.864559 25559 raft_consensus.cc:359] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.865502 25559 raft_consensus.cc:385] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.865895 25559 raft_consensus.cc:740] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.856964 25500 tablet_service.cc:1511] Processing CreateTablet for tablet 39e51fa0d7af4f7ba2f0e1c4f1362152 (DEFAULT_TABLE table=test-workload [id=13f4adc4c23f4f9298866b4eebca5193]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:02:08.866988 25559 consensus_queue.cc:260] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.867956 25500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e51fa0d7af4f7ba2f0e1c4f1362152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:08.870975 25554 raft_consensus.cc:359] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.871889 25554 raft_consensus.cc:385] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.872236 25554 raft_consensus.cc:740] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.874835 25554 consensus_queue.cc:260] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.875623 25555 ts_tablet_manager.cc:1434] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.002s
I20260709 13:02:08.884410 25555 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:08.885071 25559 ts_tablet_manager.cc:1434] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.023s	user 0.013s	sys 0.004s
I20260709 13:02:08.885900 25465 heartbeater.cc:499] Master 127.12.30.126:42253 was elected leader, sending a full tablet report...
I20260709 13:02:08.886276 25559 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:08.894089 25554 ts_tablet_manager.cc:1434] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.005s
I20260709 13:02:08.895219 25554 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:08.918282 25554 tablet_bootstrap.cc:654] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.919232 25559 tablet_bootstrap.cc:654] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.907802 25555 tablet_bootstrap.cc:654] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.926791 25554 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:08.927386 25554 ts_tablet_manager.cc:1403] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.001s
I20260709 13:02:08.930038 25555 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:08.930588 25555 ts_tablet_manager.cc:1403] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.001s
I20260709 13:02:08.931252 25559 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:08.930842 25554 raft_consensus.cc:359] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.931640 25554 raft_consensus.cc:385] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.931809 25559 ts_tablet_manager.cc:1403] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.008s
I20260709 13:02:08.931989 25554 raft_consensus.cc:740] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.932886 25554 consensus_queue.cc:260] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.933645 25555 raft_consensus.cc:359] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.934415 25555 raft_consensus.cc:385] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.934662 25555 raft_consensus.cc:740] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.935467 25559 raft_consensus.cc:359] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.935483 25555 consensus_queue.cc:260] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.936381 25559 raft_consensus.cc:385] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.936749 25559 raft_consensus.cc:740] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.936789 25554 ts_tablet_manager.cc:1434] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:02:08.937676 25559 consensus_queue.cc:260] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.938380 25555 ts_tablet_manager.cc:1434] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:02:08.940325 25554 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:08.941022 25555 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:08.941708 25559 ts_tablet_manager.cc:1434] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:02:08.942746 25559 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:08.948392 25554 tablet_bootstrap.cc:654] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.949081 25562 raft_consensus.cc:493] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.949357 25555 tablet_bootstrap.cc:654] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.949695 25559 tablet_bootstrap.cc:654] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:08.949689 25562 raft_consensus.cc:515] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:08.976104 25562 leader_election.cc:290] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
W20260709 13:02:08.979673 25540 tablet.cc:2406] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:02:08.984100 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16" is_pre_election: true
I20260709 13:02:08.985345 25365 raft_consensus.cc:2468] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 0.
I20260709 13:02:08.987618 25477 leader_election.cc:304] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:08.990418 25562 raft_consensus.cc:2804] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:08.990859 25562 raft_consensus.cc:493] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.991184 25562 raft_consensus.cc:3060] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:08.993758 25559 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:08.994396 25559 ts_tablet_manager.cc:1403] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.052s	user 0.021s	sys 0.028s
I20260709 13:02:08.997887 25559 raft_consensus.cc:359] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:08.998626 25559 raft_consensus.cc:385] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:08.998843 25554 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:08.998947 25559 raft_consensus.cc:740] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:08.999579 25554 ts_tablet_manager.cc:1403] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.003s
I20260709 13:02:09.000072 25559 consensus_queue.cc:260] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:09.001293 25555 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.001942 25555 ts_tablet_manager.cc:1403] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.027s
I20260709 13:02:09.002846 25554 raft_consensus.cc:359] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.003630 25554 raft_consensus.cc:385] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.003942 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.005364 25559 ts_tablet_manager.cc:1434] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
W20260709 13:02:09.005774 25479 leader_election.cc:343] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.005729 25555 raft_consensus.cc:359] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.006569 25559 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:09.006901 25555 raft_consensus.cc:385] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.007305 25555 raft_consensus.cc:740] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.003969 25554 raft_consensus.cc:740] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.008461 25555 consensus_queue.cc:260] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.012394 25554 consensus_queue.cc:260] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.014066 25559 tablet_bootstrap.cc:654] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.015829 25554 ts_tablet_manager.cc:1434] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 13:02:09.021322 25555 ts_tablet_manager.cc:1434] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.005s
I20260709 13:02:09.022290 25554 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:09.022650 25555 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:09.027065 25562 raft_consensus.cc:515] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.030575 25555 tablet_bootstrap.cc:654] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.032562 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:09.033259 25365 raft_consensus.cc:3060] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.045329 25555 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.046044 25555 ts_tablet_manager.cc:1403] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 13:02:09.045867 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
W20260709 13:02:09.047513 25479 leader_election.cc:343] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.048319 25562 leader_election.cc:290] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:09.049430 25365 raft_consensus.cc:2468] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:09.049779 25555 raft_consensus.cc:359] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.050642 25555 raft_consensus.cc:385] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.050800 25477 leader_election.cc:304] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:09.051090 25555 raft_consensus.cc:740] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.052177 25562 raft_consensus.cc:2804] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.052254 25555 consensus_queue.cc:260] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
W20260709 13:02:09.053824 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:09.055964 25554 tablet_bootstrap.cc:654] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.056584 25562 raft_consensus.cc:697] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:09.057622 25562 consensus_queue.cc:237] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.058625 25555 ts_tablet_manager.cc:1434] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:02:09.059109 25559 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:09.059810 25555 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:09.059854 25559 ts_tablet_manager.cc:1403] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.054s	user 0.028s	sys 0.024s
I20260709 13:02:09.066032 25559 raft_consensus.cc:359] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.067204 25559 raft_consensus.cc:385] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.067644 25559 raft_consensus.cc:740] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.069403 25555 tablet_bootstrap.cc:654] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.068774 25559 consensus_queue.cc:260] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.072418 25559 ts_tablet_manager.cc:1434] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:02:09.074507 25559 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:09.079985 25557 raft_consensus.cc:493] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.080626 25557 raft_consensus.cc:515] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:09.069852 25262 catalog_manager.cc:5697] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 1 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.083189 25559 tablet_bootstrap.cc:654] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.100939 25557 leader_election.cc:290] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:09.105602 25562 raft_consensus.cc:493] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.106698 25562 raft_consensus.cc:515] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:09.111424 25562 leader_election.cc:290] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:09.111732 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.112598 25440 raft_consensus.cc:2468] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 0.
I20260709 13:02:09.114434 25479 leader_election.cc:304] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:09.115657 25562 raft_consensus.cc:2804] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.116173 25562 raft_consensus.cc:493] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.116626 25562 raft_consensus.cc:3060] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.121124 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16" is_pre_election: true
I20260709 13:02:09.122063 25365 raft_consensus.cc:2468] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 0.
I20260709 13:02:09.126569 25562 raft_consensus.cc:515] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:09.127729 25559 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:09.128449 25559 ts_tablet_manager.cc:1403] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.019s
I20260709 13:02:09.130018 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:09.132262 25365 raft_consensus.cc:3060] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.134259 25562 leader_election.cc:290] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:09.135207 25555 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.135743 25555 ts_tablet_manager.cc:1403] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.076s	user 0.028s	sys 0.029s
I20260709 13:02:09.138723 25555 raft_consensus.cc:359] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.139475 25555 raft_consensus.cc:385] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.139366 25439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.140250 25439 raft_consensus.cc:2468] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 0.
I20260709 13:02:09.139909 25555 raft_consensus.cc:740] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.142745 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.143472 25330 leader_election.cc:304] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.143728 25555 consensus_queue.cc:260] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.144227 25514 raft_consensus.cc:2393] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e998827be8264d1d8681d5bfc1277e16 in current term 1: Already voted for candidate 7efd98fd5c384fa09947aac9da4b776b in this term.
I20260709 13:02:09.146795 25555 ts_tablet_manager.cc:1434] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:02:09.147631 25555 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:09.131879 25559 raft_consensus.cc:359] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.147694 25365 raft_consensus.cc:2468] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:09.133940 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:09.148145 25559 raft_consensus.cc:385] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.148701 25559 raft_consensus.cc:740] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.148762 25440 raft_consensus.cc:3060] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.149843 25559 consensus_queue.cc:260] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.155385 25555 tablet_bootstrap.cc:654] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.156530 25559 ts_tablet_manager.cc:1434] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 13:02:09.133613 25554 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:09.157846 25559 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:09.160805 25554 ts_tablet_manager.cc:1403] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.139s	user 0.012s	sys 0.019s
I20260709 13:02:09.160727 25557 raft_consensus.cc:2764] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:02:09.163822 25477 leader_election.cc:304] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.160336 25440 raft_consensus.cc:2468] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:09.164860 25562 raft_consensus.cc:2804] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.165721 25562 raft_consensus.cc:697] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:09.166792 25562 consensus_queue.cc:237] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:09.168224 25554 raft_consensus.cc:359] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.169061 25554 raft_consensus.cc:385] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.169335 25554 raft_consensus.cc:740] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.168311 25555 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.170511 25555 ts_tablet_manager.cc:1403] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.011s
I20260709 13:02:09.172286 25554 consensus_queue.cc:260] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.173559 25559 tablet_bootstrap.cc:654] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.173722 25555 raft_consensus.cc:359] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.174496 25555 raft_consensus.cc:385] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.174857 25555 raft_consensus.cc:740] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.175287 25554 ts_tablet_manager.cc:1434] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:02:09.178746 25554 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:09.180326 25571 raft_consensus.cc:493] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.180922 25571 raft_consensus.cc:515] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.175868 25555 consensus_queue.cc:260] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.190071 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.190855 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16" is_pre_election: true
I20260709 13:02:09.190901 25440 raft_consensus.cc:2468] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 0.
I20260709 13:02:09.192965 25571 leader_election.cc:290] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
W20260709 13:02:09.193014 25477 leader_election.cc:343] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.195129 25559 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:09.195873 25559 ts_tablet_manager.cc:1403] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.006s
I20260709 13:02:09.197193 25479 leader_election.cc:304] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:09.199481 25555 ts_tablet_manager.cc:1434] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.007s
I20260709 13:02:09.199290 25559 raft_consensus.cc:359] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.200215 25559 raft_consensus.cc:385] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.200547 25559 raft_consensus.cc:740] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.201145 25555 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:09.201373 25559 consensus_queue.cc:260] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.203890 25559 ts_tablet_manager.cc:1434] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:02:09.204478 25554 tablet_bootstrap.cc:654] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.205096 25559 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:09.211091 25262 catalog_manager.cc:5697] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 1 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.212198 25559 tablet_bootstrap.cc:654] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.213620 25555 tablet_bootstrap.cc:654] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.200676 25586 raft_consensus.cc:2804] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.218108 25586 raft_consensus.cc:493] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.218473 25586 raft_consensus.cc:3060] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.220415 25554 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:09.221143 25554 ts_tablet_manager.cc:1403] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.005s
I20260709 13:02:09.225337 25554 raft_consensus.cc:359] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.226348 25554 raft_consensus.cc:385] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.226744 25554 raft_consensus.cc:740] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.227980 25557 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.228570 25557 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.228638 25554 consensus_queue.cc:260] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.230257 25559 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:09.230798 25557 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.232267 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.233026 25514 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 0.
I20260709 13:02:09.233250 25554 ts_tablet_manager.cc:1434] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:02:09.234561 25329 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.230825 25555 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.234966 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.235715 25590 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.235842 25555 ts_tablet_manager.cc:1403] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.008s
I20260709 13:02:09.236258 25590 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.236699 25590 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.238896 25586 raft_consensus.cc:515] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.235805 25440 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 0.
I20260709 13:02:09.241542 25554 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:09.230837 25559 ts_tablet_manager.cc:1403] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.014s
I20260709 13:02:09.244954 25586 leader_election.cc:290] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:09.245453 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:09.246155 25365 raft_consensus.cc:3060] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.245442 25590 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.246395 25559 raft_consensus.cc:359] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.247190 25559 raft_consensus.cc:385] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.247560 25559 raft_consensus.cc:740] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.248586 25559 consensus_queue.cc:260] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.249503 25439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:09.250106 25439 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.253713 25563 raft_consensus.cc:493] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.254526 25365 raft_consensus.cc:2468] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:09.254441 25563 raft_consensus.cc:515] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.256644 25477 leader_election.cc:304] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.257758 25554 tablet_bootstrap.cc:654] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.258528 25439 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 1.
I20260709 13:02:09.247805 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:09.259217 25440 raft_consensus.cc:3060] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.260914 25559 ts_tablet_manager.cc:1434] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:02:09.262713 25559 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:09.264137 25586 raft_consensus.cc:2804] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.265653 25586 raft_consensus.cc:697] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:09.265697 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:09.266331 25514 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.266666 25586 consensus_queue.cc:237] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.271602 25559 tablet_bootstrap.cc:654] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.273557 25330 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.274428 25557 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.274570 25590 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 election: Requested vote from peers 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.279632 25563 leader_election.cc:290] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.283669 25514 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 1.
I20260709 13:02:09.239374 25555 raft_consensus.cc:359] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.280442 25557 raft_consensus.cc:697] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 LEADER]: Becoming Leader. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Running, Role: LEADER
I20260709 13:02:09.287647 25440 raft_consensus.cc:2468] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:09.286142 25555 raft_consensus.cc:385] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.288198 25557 consensus_queue.cc:237] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.289700 25555 raft_consensus.cc:740] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.297510 25555 consensus_queue.cc:260] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.300998 25555 ts_tablet_manager.cc:1434] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.001s
I20260709 13:02:09.302937 25555 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:09.312670 25554 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:09.313416 25554 ts_tablet_manager.cc:1403] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.007s
I20260709 13:02:09.316320 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16" is_pre_election: true
I20260709 13:02:09.317147 25365 raft_consensus.cc:2468] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 0.
I20260709 13:02:09.318701 25403 leader_election.cc:304] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.319950 25563 raft_consensus.cc:2804] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.320334 25563 raft_consensus.cc:493] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.320292 25554 raft_consensus.cc:359] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.320993 25563 raft_consensus.cc:3060] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.321295 25554 raft_consensus.cc:385] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.312670 25559 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:09.322607 25559 ts_tablet_manager.cc:1403] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.018s
I20260709 13:02:09.321631 25263 catalog_manager.cc:5697] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to e998827be8264d1d8681d5bfc1277e16 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e998827be8264d1d8681d5bfc1277e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.325394 25559 raft_consensus.cc:359] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.326007 25559 raft_consensus.cc:385] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.326282 25559 raft_consensus.cc:740] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.326979 25559 consensus_queue.cc:260] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.328083 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.328795 25514 raft_consensus.cc:2468] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 0.
I20260709 13:02:09.330175 25262 catalog_manager.cc:5697] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 1 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.321661 25554 raft_consensus.cc:740] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.332427 25554 consensus_queue.cc:260] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.339967 25554 ts_tablet_manager.cc:1434] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 13:02:09.341624 25554 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:09.337126 25555 tablet_bootstrap.cc:654] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.345611 25563 raft_consensus.cc:515] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.353768 25557 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.354650 25557 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.356405 25554 tablet_bootstrap.cc:654] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.356564 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:09.360738 25365 raft_consensus.cc:3060] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.365662 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:09.366232 25557 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.351837 25563 leader_election.cc:290] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.369222 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.370245 25440 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 0.
I20260709 13:02:09.372288 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.372682 25330 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.373093 25513 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 0.
I20260709 13:02:09.375191 25557 raft_consensus.cc:2804] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.375692 25557 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.376154 25557 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.366844 25514 raft_consensus.cc:3060] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.396271 25563 raft_consensus.cc:493] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.396759 25559 ts_tablet_manager.cc:1434] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.074s	user 0.032s	sys 0.037s
I20260709 13:02:09.397199 25563 raft_consensus.cc:515] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.398129 25559 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:09.402275 25364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16" is_pre_election: true
W20260709 13:02:09.404032 25403 leader_election.cc:343] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.404520 25559 tablet_bootstrap.cc:654] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.406538 25563 leader_election.cc:290] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.406752 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.407846 25513 raft_consensus.cc:2468] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 0.
I20260709 13:02:09.409416 25404 leader_election.cc:304] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:09.410877 25598 raft_consensus.cc:2804] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.411291 25598 raft_consensus.cc:493] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.411613 25598 raft_consensus.cc:3060] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.413025 25365 raft_consensus.cc:2468] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 1.
I20260709 13:02:09.414428 25403 leader_election.cc:304] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.415349 25563 raft_consensus.cc:2804] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.417825 25514 raft_consensus.cc:2468] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 1.
I20260709 13:02:09.432117 25555 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.432487 25563 raft_consensus.cc:697] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:09.432971 25555 ts_tablet_manager.cc:1403] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.131s	user 0.042s	sys 0.036s
I20260709 13:02:09.433990 25563 consensus_queue.cc:237] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.435072 25557 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.437592 25554 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:09.438165 25555 raft_consensus.cc:359] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.439036 25555 raft_consensus.cc:385] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.439538 25555 raft_consensus.cc:740] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.438545 25598 raft_consensus.cc:515] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.440290 25555 consensus_queue.cc:260] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.441531 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:09.442173 25440 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.442397 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:09.438172 25554 ts_tablet_manager.cc:1403] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.097s	user 0.015s	sys 0.016s
I20260709 13:02:09.442999 25514 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.446498 25554 raft_consensus.cc:359] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.447515 25554 raft_consensus.cc:385] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.447798 25554 raft_consensus.cc:740] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.448228 25557 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 election: Requested vote from peers 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.448697 25554 consensus_queue.cc:260] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.453231 25514 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 1.
I20260709 13:02:09.453614 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:09.454332 25513 raft_consensus.cc:3060] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.443019 25555 ts_tablet_manager.cc:1434] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:02:09.455725 25439 raft_consensus.cc:3060] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.456084 25555 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:09.456907 25559 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:09.457497 25559 ts_tablet_manager.cc:1403] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.003s
I20260709 13:02:09.454764 25329 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.453614 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:09.461252 25594 raft_consensus.cc:2804] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.461808 25594 raft_consensus.cc:697] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 LEADER]: Becoming Leader. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Running, Role: LEADER
I20260709 13:02:09.462455 25555 tablet_bootstrap.cc:654] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.462663 25594 consensus_queue.cc:237] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
W20260709 13:02:09.464740 25403 leader_election.cc:343] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.468174 25554 ts_tablet_manager.cc:1434] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:02:09.469622 25554 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:09.470976 25513 raft_consensus.cc:2468] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 1.
I20260709 13:02:09.472646 25404 leader_election.cc:304] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:09.474167 25601 raft_consensus.cc:2804] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.472000 25559 raft_consensus.cc:359] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.474864 25559 raft_consensus.cc:385] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.475412 25559 raft_consensus.cc:740] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.475871 25586 consensus_queue.cc:1048] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:09.476344 25559 consensus_queue.cc:260] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.478966 25440 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 1.
I20260709 13:02:09.483283 25559 ts_tablet_manager.cc:1434] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 13:02:09.480542 25598 leader_election.cc:290] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.484754 25559 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf: Bootstrap starting.
I20260709 13:02:09.487003 25590 raft_consensus.cc:493] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.487592 25590 raft_consensus.cc:515] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.489876 25590 leader_election.cc:290] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.479995 25263 catalog_manager.cc:5697] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 1 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.496053 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.496114 25562 consensus_queue.cc:1048] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:09.502622 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.504716 25440 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 0.
I20260709 13:02:09.514890 25330 leader_election.cc:304] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.515735 25513 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 0.
I20260709 13:02:09.524732 25559 tablet_bootstrap.cc:654] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.518332 25601 raft_consensus.cc:697] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:09.530282 25601 consensus_queue.cc:237] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.528414 25594 raft_consensus.cc:2804] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.532385 25594 raft_consensus.cc:493] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.533147 25554 tablet_bootstrap.cc:654] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.532756 25594 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.544957 25555 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.545678 25555 ts_tablet_manager.cc:1403] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.090s	user 0.020s	sys 0.028s
I20260709 13:02:09.554073 25555 raft_consensus.cc:359] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.555048 25555 raft_consensus.cc:385] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.555393 25555 raft_consensus.cc:740] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.556411 25555 consensus_queue.cc:260] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.565637 25263 catalog_manager.cc:5697] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to e998827be8264d1d8681d5bfc1277e16 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e998827be8264d1d8681d5bfc1277e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.567857 25555 ts_tablet_manager.cc:1434] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 13:02:09.574425 25555 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b: Bootstrap starting.
I20260709 13:02:09.589286 25555 tablet_bootstrap.cc:654] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.592022 25562 raft_consensus.cc:493] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.593539 25562 raft_consensus.cc:515] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.596720 25554 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:09.597339 25554 ts_tablet_manager.cc:1403] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.128s	user 0.013s	sys 0.007s
I20260709 13:02:09.597976 25562 leader_election.cc:290] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:09.599502 25594 raft_consensus.cc:515] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.600180 25554 raft_consensus.cc:359] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.600936 25554 raft_consensus.cc:385] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.601266 25554 raft_consensus.cc:740] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.602084 25554 consensus_queue.cc:260] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.603716 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:09.604434 25594 leader_election.cc:290] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 election: Requested vote from peers 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.604526 25513 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.603755 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16" is_pre_election: true
I20260709 13:02:09.613992 25513 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 1.
I20260709 13:02:09.613690 25261 catalog_manager.cc:5697] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 1 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.612083 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:09.612493 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.616887 25438 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.617267 25555 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b: No bootstrap required, opened a new log
I20260709 13:02:09.617767 25555 ts_tablet_manager.cc:1403] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.000s
W20260709 13:02:09.618012 25479 leader_election.cc:343] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:09.612596 25559 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf: No bootstrap required, opened a new log
I20260709 13:02:09.615995 25365 raft_consensus.cc:2468] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 0.
I20260709 13:02:09.619135 25559 ts_tablet_manager.cc:1403] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf: Time spent bootstrapping tablet: real 0.135s	user 0.023s	sys 0.004s
I20260709 13:02:09.619843 25477 leader_election.cc:304] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:09.620569 25555 raft_consensus.cc:359] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.621119 25562 raft_consensus.cc:2804] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.621295 25555 raft_consensus.cc:385] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.621596 25562 raft_consensus.cc:493] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.621663 25555 raft_consensus.cc:740] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.622017 25562 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.622437 25555 consensus_queue.cc:260] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.624672 25555 ts_tablet_manager.cc:1434] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:09.624993 25329 leader_election.cc:304] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.621971 25559 raft_consensus.cc:359] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.628839 25559 raft_consensus.cc:385] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.629158 25559 raft_consensus.cc:740] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.629624 25562 raft_consensus.cc:515] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.629951 25559 consensus_queue.cc:260] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.615024 25554 ts_tablet_manager.cc:1434] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:02:09.632889 25562 leader_election.cc:290] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:09.633806 25559 ts_tablet_manager.cc:1434] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.006s
I20260709 13:02:09.633978 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:09.635373 25440 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.634158 25554 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:09.634680 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:09.641108 25365 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.639528 25594 raft_consensus.cc:2804] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.643219 25594 raft_consensus.cc:697] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 LEADER]: Becoming Leader. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Running, Role: LEADER
I20260709 13:02:09.644208 25594 consensus_queue.cc:237] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.643277 25438 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e998827be8264d1d8681d5bfc1277e16 in term 1.
I20260709 13:02:09.647310 25440 raft_consensus.cc:2468] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:09.648636 25479 leader_election.cc:304] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:09.649411 25562 raft_consensus.cc:2804] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.649895 25562 raft_consensus.cc:697] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:09.650755 25562 consensus_queue.cc:237] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.658640 25263 catalog_manager.cc:5697] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to e998827be8264d1d8681d5bfc1277e16 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "e998827be8264d1d8681d5bfc1277e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.661252 25365 raft_consensus.cc:2468] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:09.660629 25262 catalog_manager.cc:5697] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 1 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.663597 25554 tablet_bootstrap.cc:654] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:09.677209 25554 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: No bootstrap required, opened a new log
I20260709 13:02:09.678107 25554 ts_tablet_manager.cc:1403] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.006s
I20260709 13:02:09.678798 25594 raft_consensus.cc:493] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.679421 25594 raft_consensus.cc:515] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.681398 25554 raft_consensus.cc:359] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.682257 25554 raft_consensus.cc:385] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:09.682583 25554 raft_consensus.cc:740] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:09.682667 25440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:09.681774 25594 leader_election.cc:290] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.683477 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "e998827be8264d1d8681d5bfc1277e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.683387 25554 consensus_queue.cc:260] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.684378 25513 raft_consensus.cc:2393] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e998827be8264d1d8681d5bfc1277e16 in current term 1: Already voted for candidate 815ff4fecd7844c2bfb4864215f033cf in this term.
I20260709 13:02:09.685616 25329 leader_election.cc:304] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e998827be8264d1d8681d5bfc1277e16; no voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:09.685695 25554 ts_tablet_manager.cc:1434] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:02:09.686434 25594 raft_consensus.cc:3060] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.695021 25594 raft_consensus.cc:2749] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:09.695895 25562 consensus_queue.cc:1048] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:09.699239 25601 raft_consensus.cc:493] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.699822 25601 raft_consensus.cc:515] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.703831 25364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16" is_pre_election: true
I20260709 13:02:09.704705 25364 raft_consensus.cc:2468] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 0.
I20260709 13:02:09.705425 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:09.706144 25513 raft_consensus.cc:2468] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 0.
I20260709 13:02:09.706336 25403 leader_election.cc:304] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.707630 25601 leader_election.cc:290] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.707703 25606 raft_consensus.cc:2804] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:09.708395 25606 raft_consensus.cc:493] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.708887 25606 raft_consensus.cc:3060] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.711428 25562 consensus_queue.cc:1048] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:09.715095 25562 consensus_queue.cc:1048] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:09.720592 25606 raft_consensus.cc:515] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.724035 25365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:09.724694 25365 raft_consensus.cc:3060] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.726454 25606 leader_election.cc:290] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:09.731626 25586 consensus_queue.cc:1048] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:09.734830 25365 raft_consensus.cc:2468] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 1.
I20260709 13:02:09.736344 25403 leader_election.cc:304] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:09.737571 25601 raft_consensus.cc:2804] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:09.739773 25601 raft_consensus.cc:697] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:09.740717 25601 consensus_queue.cc:237] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:09.743643 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:09.744398 25513 raft_consensus.cc:3060] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:09.754011 25513 raft_consensus.cc:2468] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 1.
I20260709 13:02:09.759110 25260 catalog_manager.cc:5697] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 1 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:09.787590 25594 consensus_queue.cc:1048] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:09.799937 25557 consensus_queue.cc:1048] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:09.852597 25606 consensus_queue.cc:1048] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:09.864804 25601 consensus_queue.cc:1048] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:09.928052 25557 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:09.939033 25590 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:10.025079 12409 ts_manager.cc:295] Set tserver state for e998827be8264d1d8681d5bfc1277e16 to MAINTENANCE_MODE
I20260709 13:02:10.025929 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
W20260709 13:02:10.037230 25403 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 342 similar messages]
W20260709 13:02:10.040827 25403 consensus_peers.cc:597] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:10.054239 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:02:10.055207 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:10.055984 12409 tablet_replica.cc:333] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.057139 12409 raft_consensus.cc:2243] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:10.057637 12409 raft_consensus.cc:2272] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:10.059653 25477 consensus_peers.cc:597] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:10.060942 12409 tablet_replica.cc:333] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.061750 12409 raft_consensus.cc:2243] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:10.062238 12409 raft_consensus.cc:2272] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:10.064563 12409 tablet_replica.cc:333] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.065200 12409 raft_consensus.cc:2243] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:10.065869 12409 pending_rounds.cc:70] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Trying to abort 1 pending ops.
I20260709 13:02:10.066110 12409 pending_rounds.cc:77] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305634323032166400 op_type: NO_OP noop_request { }
I20260709 13:02:10.066447 12409 raft_consensus.cc:2889] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:02:10.066763 12409 raft_consensus.cc:2272] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:10.069360 12409 tablet_replica.cc:333] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.069988 12409 raft_consensus.cc:2243] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:10.070461 12409 raft_consensus.cc:2272] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:10.072846 12409 tablet_replica.cc:333] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.073632 12409 raft_consensus.cc:2243] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:10.074195 12409 raft_consensus.cc:2272] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:10.076436 25562 consensus_queue.cc:1048] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:10.076992 12409 tablet_replica.cc:333] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.077915 12409 raft_consensus.cc:2243] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:10.078887 12409 raft_consensus.cc:2272] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:10.081372 12409 tablet_replica.cc:333] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.082072 12409 raft_consensus.cc:2243] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:10.083153 12409 raft_consensus.cc:2272] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:10.086171 12409 tablet_replica.cc:333] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.086987 12409 raft_consensus.cc:2243] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:10.087481 12409 raft_consensus.cc:2272] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:10.090098 12409 tablet_replica.cc:333] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.090927 12409 raft_consensus.cc:2243] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:10.091444 12409 raft_consensus.cc:2272] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:10.093894 25477 consensus_peers.cc:597] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:10.094982 12409 tablet_replica.cc:333] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: stopping tablet replica
I20260709 13:02:10.096067 12409 raft_consensus.cc:2243] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:10.096879 12409 raft_consensus.cc:2272] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:10.124857 25477 consensus_peers.cc:597] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:10.132993 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
I20260709 13:02:10.156416 25601 consensus_queue.cc:1048] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:10.223737 25601 consensus_queue.cc:1048] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:02:10.237977 25403 consensus_peers.cc:597] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:10.240176 25477 consensus_peers.cc:597] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:10.350519 25403 consensus_peers.cc:597] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:11.057490 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:11.299108 25601 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e998827be8264d1d8681d5bfc1277e16)
I20260709 13:02:11.300045 25601 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.302270 25601 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:11.304021 25562 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e998827be8264d1d8681d5bfc1277e16)
I20260709 13:02:11.303869 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:11.304909 25562 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.305927 25513 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 1.
W20260709 13:02:11.307329 25403 leader_election.cc:336] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.307658 25562 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:11.308012 25404 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:11.308350 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:11.309010 25438 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:11.309649 25601 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:11.310032 25601 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting leader election (detected failure of leader e998827be8264d1d8681d5bfc1277e16)
I20260709 13:02:11.310432 25479 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:11.310439 25601 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.311422 25562 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:11.311775 25562 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e998827be8264d1d8681d5bfc1277e16)
I20260709 13:02:11.312170 25562 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:11.312122 25477 leader_election.cc:336] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.318976 25562 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.319856 25601 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.320914 25562 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:11.322204 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:11.322466 25601 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:11.323088 25438 raft_consensus.cc:2393] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7efd98fd5c384fa09947aac9da4b776b in current term 2: Already voted for candidate 815ff4fecd7844c2bfb4864215f033cf in this term.
I20260709 13:02:11.323622 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:11.324548 25513 raft_consensus.cc:2393] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 815ff4fecd7844c2bfb4864215f033cf in current term 2: Already voted for candidate 7efd98fd5c384fa09947aac9da4b776b in this term.
W20260709 13:02:11.326359 25477 leader_election.cc:336] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.326954 25477 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b; no voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:11.328097 25562 raft_consensus.cc:2749] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:02:11.329044 25403 leader_election.cc:336] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.329571 25403 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf; no voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:11.330416 25601 raft_consensus.cc:2749] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.438172 25562 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e998827be8264d1d8681d5bfc1277e16)
I20260709 13:02:11.438778 25562 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.440948 25562 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:11.442371 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:11.443173 25438 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:11.443665 25601 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e998827be8264d1d8681d5bfc1277e16)
I20260709 13:02:11.444196 25601 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.444451 25479 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:11.445371 25562 raft_consensus.cc:2804] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:02:11.445628 25477 leader_election.cc:336] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.445823 25562 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e998827be8264d1d8681d5bfc1277e16)
I20260709 13:02:11.446311 25562 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.447219 25601 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:11.447698 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
W20260709 13:02:11.451702 25403 leader_election.cc:336] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.456758 25562 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.458181 25513 raft_consensus.cc:2393] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 815ff4fecd7844c2bfb4864215f033cf in current term 2: Already voted for candidate 7efd98fd5c384fa09947aac9da4b776b in this term.
I20260709 13:02:11.459239 25562 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:11.460707 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:11.461382 25438 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.461900 25404 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf; no voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16
W20260709 13:02:11.464340 25477 leader_election.cc:336] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.469099 25438 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:11.469779 25601 raft_consensus.cc:2749] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:11.470677 25479 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:11.471724 25562 raft_consensus.cc:2804] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:11.472800 25562 raft_consensus.cc:697] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:11.473614 25562 consensus_queue.cc:237] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.481904 25260 catalog_manager.cc:5697] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 1 to 2, leader changed from e998827be8264d1d8681d5bfc1277e16 (127.12.30.65) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 2 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.701829 25562 raft_consensus.cc:493] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.702317 25562 raft_consensus.cc:515] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.704061 25562 leader_election.cc:290] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:11.705286 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf" is_pre_election: true
I20260709 13:02:11.705922 25438 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 1.
I20260709 13:02:11.707192 25479 leader_election.cc:304] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:11.708071 25562 raft_consensus.cc:2804] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:11.708379 25562 raft_consensus.cc:493] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:11.708523 25477 leader_election.cc:336] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.708688 25562 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:11.715233 25562 raft_consensus.cc:515] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.717217 25562 leader_election.cc:290] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:11.718369 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:11.719149 25438 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:11.721817 25477 leader_election.cc:336] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:11.725373 25438 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:11.726950 25479 leader_election.cc:304] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:11.728220 25562 raft_consensus.cc:2804] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:11.728924 25562 raft_consensus.cc:697] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:11.730295 25562 consensus_queue.cc:237] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:11.741671 25260 catalog_manager.cc:5697] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 1 to 2, leader changed from e998827be8264d1d8681d5bfc1277e16 (127.12.30.65) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 2 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.871728 25438 raft_consensus.cc:1275] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:11.873294 25562 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:11.890614 25477 consensus_peers.cc:597] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:12.058678 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:02:12.167236 25477 consensus_peers.cc:597] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:12.257973 25438 raft_consensus.cc:1275] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:02:12.259366 25650 consensus_queue.cc:1048] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:02:12.355984 25403 consensus_peers.cc:597] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:12.432747 25477 consensus_peers.cc:597] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:12.474292 25477 consensus_peers.cc:597] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:12.594281 25403 consensus_peers.cc:597] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:12.674382 25477 consensus_peers.cc:597] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:12.675119 25477 consensus_peers.cc:597] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:12.954574 25403 consensus_peers.cc:597] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:13.059758 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:13.220209 25601 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.220955 25601 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:13.223285 25601 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:13.224694 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b" is_pre_election: true
I20260709 13:02:13.225519 25513 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:13.226881 25404 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:13.227892 25601 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:02:13.228014 25403 leader_election.cc:336] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:13.228338 25601 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.228724 25601 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:13.234645 25601 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:13.236699 25601 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:13.238091 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:13.238869 25513 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:02:13.240513 25403 leader_election.cc:336] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111)
I20260709 13:02:13.245795 25513 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 3.
I20260709 13:02:13.247330 25404 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: e998827be8264d1d8681d5bfc1277e16
I20260709 13:02:13.248471 25601 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:13.249740 25601 raft_consensus.cc:697] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:13.250626 25601 consensus_queue.cc:237] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:13.259774 25260 catalog_manager.cc:5697] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 1 to 3, leader changed from e998827be8264d1d8681d5bfc1277e16 (127.12.30.65) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 3 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:13.655604 25513 raft_consensus.cc:1275] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:13.657327 25601 consensus_queue.cc:1048] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:02:13.682228 25403 consensus_peers.cc:597] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:14.061254 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:02:14.740365 25477 consensus_peers.cc:597] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:14.753788 25477 consensus_peers.cc:597] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:02:14.841156 25477 consensus_peers.cc:597] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:14.867146 25403 consensus_peers.cc:597] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:14.988471 25477 consensus_peers.cc:597] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:14.999264 25477 consensus_peers.cc:597] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:15.019590 25403 consensus_peers.cc:597] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:02:15.062531 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:02:15.151815 25477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111) [suppressed 100 similar messages]
I20260709 13:02:15.154912 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:15.163508 25663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:15.164499 25664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:15.168737 12409 server_base.cc:1061] running on GCE node
W20260709 13:02:15.170188 25666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:15.171401 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:15.171753 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:15.172088 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602135172005 us; error 0 us; skew 500 ppm
I20260709 13:02:15.172922 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:15.176543 12409 webserver.cc:533] Webserver started at http://127.12.30.65:35515/ using document root <none> and password file <none>
I20260709 13:02:15.177268 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:15.177848 12409 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:02:15.183162 12409 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:02:15.187582 25671 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:15.188614 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:02:15.188982 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "e998827be8264d1d8681d5bfc1277e16"
format_stamp: "Formatted at 2026-07-09 13:02:08 on dist-test-slave-mlt7"
I20260709 13:02:15.189342 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:15.217903 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
W20260709 13:02:15.218403 25477 consensus_peers.cc:597] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:15.219434 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:15.221256 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:15.225395 25678 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:02:15.296252 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:02:15.296923 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.073s	user 0.002s	sys 0.000s
I20260709 13:02:15.297681 12409 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:02:15.305642 25678 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.320847 25678 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:02:15.322052 25678 tablet_bootstrap.cc:492] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.322832 25678 ts_tablet_manager.cc:1403] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:02:15.327250 25678 raft_consensus.cc:359] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.328553 25678 raft_consensus.cc:740] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.330168 25678 consensus_queue.cc:260] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.333925 25678 ts_tablet_manager.cc:1434] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:02:15.335631 25678 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.350664 25678 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.351815 25678 tablet_bootstrap.cc:492] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.352609 25678 ts_tablet_manager.cc:1403] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:02:15.354996 25678 raft_consensus.cc:359] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.355899 25678 raft_consensus.cc:740] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.357242 25678 consensus_queue.cc:260] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.359232 25678 ts_tablet_manager.cc:1434] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:02:15.360153 25678 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.364905 12409 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:02:15.365391 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.068s	user 0.060s	sys 0.005s
I20260709 13:02:15.374675 25678 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.375881 25678 tablet_bootstrap.cc:492] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.376756 25678 ts_tablet_manager.cc:1403] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:02:15.379179 25678 raft_consensus.cc:359] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.379808 25678 raft_consensus.cc:740] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.380637 25678 consensus_queue.cc:260] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.383149 25678 ts_tablet_manager.cc:1434] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:02:15.384049 25678 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.399621 25678 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.401152 25678 tablet_bootstrap.cc:492] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.401916 25678 ts_tablet_manager.cc:1403] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 13:02:15.404814 25678 raft_consensus.cc:359] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:15.405572 25678 raft_consensus.cc:740] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.406306 25678 consensus_queue.cc:260] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
W20260709 13:02:15.407416 25403 consensus_peers.cc:597] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:02:15.408620 25678 ts_tablet_manager.cc:1434] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:02:15.409485 25678 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.422140 25678 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.423123 25678 tablet_bootstrap.cc:492] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.423839 25678 ts_tablet_manager.cc:1403] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:02:15.426191 25678 raft_consensus.cc:359] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.426779 25678 raft_consensus.cc:740] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.427464 25678 consensus_queue.cc:260] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.429936 25678 ts_tablet_manager.cc:1434] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:02:15.430675 25678 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.442224 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:35871
I20260709 13:02:15.442338 25743 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:35871 every 8 connection(s)
I20260709 13:02:15.444798 25678 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.445722 25678 tablet_bootstrap.cc:492] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.446491 25678 ts_tablet_manager.cc:1403] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:02:15.447360 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:15.455677 25678 raft_consensus.cc:359] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.456915 25678 raft_consensus.cc:740] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.457868 25678 consensus_queue.cc:260] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.460291 25678 ts_tablet_manager.cc:1434] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.004s
I20260709 13:02:15.461512 25678 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.490064 25678 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.491272 25678 tablet_bootstrap.cc:492] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.492084 25678 ts_tablet_manager.cc:1403] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.031s	user 0.023s	sys 0.004s
I20260709 13:02:15.495083 25678 raft_consensus.cc:359] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.495980 25678 raft_consensus.cc:740] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.496903 25678 consensus_queue.cc:260] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.499509 25678 ts_tablet_manager.cc:1434] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:02:15.500505 25678 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.501619 25438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
mode: GRACEFUL
new_leader_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:47114
I20260709 13:02:15.502494 25438 raft_consensus.cc:606] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Received request to transfer leadership to TS 7efd98fd5c384fa09947aac9da4b776b
I20260709 13:02:15.505101 25744 heartbeater.cc:344] Connected to a master server at 127.12.30.126:42253
I20260709 13:02:15.505554 25744 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:15.506683 25744 heartbeater.cc:507] Master 127.12.30.126:42253 requested a full tablet report, sending...
I20260709 13:02:15.513734 25260 ts_manager.cc:194] Re-registered known tserver with Master: e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871)
I20260709 13:02:15.520681 25678 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.521871 25678 tablet_bootstrap.cc:492] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.522656 25678 ts_tablet_manager.cc:1403] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 13:02:15.523638 25513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:15.524434 25513 raft_consensus.cc:606] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:15.525604 25678 raft_consensus.cc:359] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.526371 25260 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47148
I20260709 13:02:15.526396 25678 raft_consensus.cc:740] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.527396 25678 consensus_queue.cc:260] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.528508 25513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:15.529295 25513 raft_consensus.cc:606] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:15.529927 25678 ts_tablet_manager.cc:1434] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:02:15.530853 25678 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.531203 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:15.531729 25744 heartbeater.cc:499] Master 127.12.30.126:42253 was elected leader, sending a full tablet report...
I20260709 13:02:15.532218 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:15.533215 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:15.548231 25678 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.550048 25678 tablet_bootstrap.cc:492] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.550881 25678 ts_tablet_manager.cc:1403] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:02:15.554968 25678 raft_consensus.cc:359] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.555612 25678 raft_consensus.cc:740] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.556574 25678 consensus_queue.cc:260] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.559345 25678 ts_tablet_manager.cc:1434] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:02:15.561105 25678 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Bootstrap starting.
I20260709 13:02:15.578168 25678 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:02:15.578953 25678 tablet_bootstrap.cc:492] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Bootstrap complete.
I20260709 13:02:15.579615 25678 ts_tablet_manager.cc:1403] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.004s
I20260709 13:02:15.582166 25678 raft_consensus.cc:359] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.582873 25678 raft_consensus.cc:740] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e998827be8264d1d8681d5bfc1277e16, State: Initialized, Role: FOLLOWER
I20260709 13:02:15.583837 25678 consensus_queue.cc:260] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.586241 25678 ts_tablet_manager.cc:1434] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:02:15.675673 25679 raft_consensus.cc:993] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf: : Instructing follower 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:15.676313 25652 raft_consensus.cc:1081] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Signalling peer 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:15.678359 25513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:42700
I20260709 13:02:15.678884 25513 raft_consensus.cc:493] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:15.679155 25513 raft_consensus.cc:3060] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.685514 25513 raft_consensus.cc:515] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.688184 25513 leader_election.cc:290] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:15.688736 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:15.688808 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1caefa4abb3d435db0299d2ffadf77f5" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:15.689652 25717 raft_consensus.cc:3060] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.689740 25438 raft_consensus.cc:3055] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:15.690229 25438 raft_consensus.cc:740] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:15.691054 25438 consensus_queue.cc:260] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.692277 25438 raft_consensus.cc:3060] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.696985 25717 raft_consensus.cc:2468] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:15.698357 25477 leader_election.cc:304] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:15.699146 25747 raft_consensus.cc:2804] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:15.699646 25747 raft_consensus.cc:697] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:15.699687 25438 raft_consensus.cc:2468] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:15.700623 25747 consensus_queue.cc:237] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.709862 25260 catalog_manager.cc:5697] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 1 to 2, leader changed from 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 2 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:15.759555 25658 raft_consensus.cc:993] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:15.760300 25747 raft_consensus.cc:1081] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:15.762348 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:15.763197 25438 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:15.763574 25438 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.773555 25438 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.778185 25717 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.779795 25438 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:15.782492 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:15.783280 25513 raft_consensus.cc:3055] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:15.783641 25513 raft_consensus.cc:740] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:15.784699 25513 consensus_queue.cc:260] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.786227 25513 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.789450 25716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:15.790705 25716 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.791316 25717 pending_rounds.cc:85] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Aborting all ops after (but not including) 0
I20260709 13:02:15.792274 25717 pending_rounds.cc:107] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:02:15.792546 25717 raft_consensus.cc:2889] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:15.797564 25513 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 3.
I20260709 13:02:15.799664 25404 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:15.800705 25652 raft_consensus.cc:2804] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:15.801222 25716 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 3.
I20260709 13:02:15.801313 25652 raft_consensus.cc:697] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:15.803057 25652 consensus_queue.cc:237] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.812919 25262 catalog_manager.cc:5697] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 2 to 3, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 3 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:15.872218 25716 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:02:15.873716 25658 raft_consensus.cc:993] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:15.874320 25747 raft_consensus.cc:1081] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:15.876077 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:15.876662 25438 raft_consensus.cc:493] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:15.877041 25438 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.884771 25438 raft_consensus.cc:515] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.887303 25438 leader_election.cc:290] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:15.887872 25716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:15.888975 25716 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.888691 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:15.890388 25513 raft_consensus.cc:3055] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:15.890777 25513 raft_consensus.cc:740] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:15.893978 25513 consensus_queue.cc:260] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.895998 25513 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:15.901062 25716 raft_consensus.cc:2468] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:15.902545 25403 leader_election.cc:304] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:15.903403 25652 raft_consensus.cc:2804] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:15.903849 25652 raft_consensus.cc:697] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:15.904758 25652 consensus_queue.cc:237] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:15.905264 25513 raft_consensus.cc:2468] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:15.913954 25262 catalog_manager.cc:5697] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 1 to 2, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 2 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:16.132992 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:16.155501 25716 raft_consensus.cc:1275] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.157804 25771 consensus_queue.cc:1048] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:16.176589 25438 raft_consensus.cc:1275] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.178292 25747 consensus_queue.cc:1048] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:16.383136 25513 raft_consensus.cc:1275] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.384460 25776 consensus_queue.cc:1048] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:16.388075 25514 raft_consensus.cc:1275] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:16.389633 25652 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:16.396827 25716 raft_consensus.cc:1275] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:16.399683 25776 consensus_queue.cc:1048] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:16.413789 25717 raft_consensus.cc:1275] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:16.415625 25781 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:16.546500 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:16.556820 25438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
mode: GRACEFUL
new_leader_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:47114
I20260709 13:02:16.557688 25438 raft_consensus.cc:606] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Received request to transfer leadership to TS 7efd98fd5c384fa09947aac9da4b776b
I20260709 13:02:16.560901 25513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:16.561425 25513 raft_consensus.cc:606] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:16.564352 25438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
mode: GRACEFUL
new_leader_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:47114
I20260709 13:02:16.564910 25438 raft_consensus.cc:606] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Received request to transfer leadership to TS 7efd98fd5c384fa09947aac9da4b776b
I20260709 13:02:16.566382 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:16.566985 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:16.567829 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:16.763854 25652 raft_consensus.cc:993] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf: : Instructing follower 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:16.764410 25781 raft_consensus.cc:1081] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Signalling peer 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:16.766139 25513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:42700
I20260709 13:02:16.766812 25513 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:16.767114 25513 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.775195 25513 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:16.776852 25513 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 4 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:16.778156 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:16.778071 25716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:16.778821 25438 raft_consensus.cc:3055] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:16.778954 25716 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.779085 25438 raft_consensus.cc:740] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:16.780093 25438 consensus_queue.cc:260] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:16.781147 25438 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.786608 25716 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 4.
I20260709 13:02:16.787933 25438 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 4.
I20260709 13:02:16.788046 25477 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:16.789279 25747 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:16.789907 25747 raft_consensus.cc:697] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:16.791036 25747 consensus_queue.cc:237] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:16.799136 25262 catalog_manager.cc:5697] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 3 to 4, leader changed from 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 4 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:16.935129 25652 raft_consensus.cc:993] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf: : Instructing follower 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:16.935714 25781 raft_consensus.cc:1081] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Signalling peer 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:16.937357 25513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:42700
I20260709 13:02:16.937773 25513 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:16.938010 25513 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.945243 25513 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:16.946655 25513 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 4 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:16.947518 25716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:16.947778 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:16.948218 25716 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.948412 25438 raft_consensus.cc:3055] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:16.948825 25438 raft_consensus.cc:740] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:16.949755 25438 consensus_queue.cc:260] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:16.950837 25438 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:16.955144 25716 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 4.
I20260709 13:02:16.956737 25477 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:16.957263 25438 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 4.
I20260709 13:02:16.957530 25768 raft_consensus.cc:2804] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:16.958876 25768 raft_consensus.cc:697] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:16.959937 25768 consensus_queue.cc:237] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:16.968317 25262 catalog_manager.cc:5697] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 3 to 4, leader changed from 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 4 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:16.987181 25747 raft_consensus.cc:993] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:16.987658 25768 raft_consensus.cc:1081] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:16.989252 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:16.989779 25438 raft_consensus.cc:493] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:16.990106 25438 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:16.997311 25438 raft_consensus.cc:515] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:16.999169 25438 leader_election.cc:290] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:17.000515 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:17.000278 25716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:17.001299 25513 raft_consensus.cc:3055] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:17.001362 25716 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.001669 25513 raft_consensus.cc:740] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:17.002609 25513 consensus_queue.cc:260] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:17.003662 25513 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.008265 25716 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 3.
I20260709 13:02:17.009622 25403 leader_election.cc:304] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:17.010213 25513 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 3.
I20260709 13:02:17.010438 25781 raft_consensus.cc:2804] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:17.011049 25781 raft_consensus.cc:697] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:17.012559 25781 consensus_queue.cc:237] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:17.020867 25262 catalog_manager.cc:5697] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 2 to 3, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 3 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:17.135315 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:17.367070 25438 raft_consensus.cc:1275] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:17.368250 25716 raft_consensus.cc:1275] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:17.368782 25768 consensus_queue.cc:1048] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:17.369769 25747 consensus_queue.cc:1048] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:17.428750 25716 raft_consensus.cc:1275] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:17.430500 25768 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:17.440883 25438 raft_consensus.cc:1275] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:17.442672 25792 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:17.529697 25513 raft_consensus.cc:1275] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:17.531708 25781 consensus_queue.cc:1048] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:17.545037 25716 raft_consensus.cc:1275] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:17.547474 25800 consensus_queue.cc:1048] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:17.574494 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:17.585000 25438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
mode: GRACEFUL
new_leader_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:47114
I20260709 13:02:17.585685 25438 raft_consensus.cc:606] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Received request to transfer leadership to TS 7efd98fd5c384fa09947aac9da4b776b
I20260709 13:02:17.588749 25513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:17.589344 25513 raft_consensus.cc:606] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:17.592412 25513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:17.592945 25513 raft_consensus.cc:606] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:17.594141 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:17.594856 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:17.595685 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:17.663631 25795 raft_consensus.cc:993] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:17.664314 25792 raft_consensus.cc:1081] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:17.666474 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:17.667040 25438 raft_consensus.cc:493] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:17.667343 25438 raft_consensus.cc:3060] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.674108 25438 raft_consensus.cc:515] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:17.675885 25438 leader_election.cc:290] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:17.677126 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:17.676987 25716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e367fc7664aa486fbbb057c89d54508f" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:17.677866 25513 raft_consensus.cc:3055] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:17.678103 25716 raft_consensus.cc:3060] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.678188 25513 raft_consensus.cc:740] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:17.679245 25513 consensus_queue.cc:260] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:17.680390 25513 raft_consensus.cc:3060] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:17.686420 25513 raft_consensus.cc:2468] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:17.686713 25716 raft_consensus.cc:2468] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:17.687753 25404 leader_election.cc:304] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, 815ff4fecd7844c2bfb4864215f033cf; no voters: 
I20260709 13:02:17.688822 25800 raft_consensus.cc:2804] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:17.689296 25800 raft_consensus.cc:697] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:17.690359 25800 consensus_queue.cc:237] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } }
I20260709 13:02:17.699666 25262 catalog_manager.cc:5697] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 1 to 2, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 2 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:17.849049 25781 raft_consensus.cc:993] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf: : Instructing follower 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:17.849669 25781 raft_consensus.cc:1081] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Signalling peer 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:17.851588 25513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:42700
I20260709 13:02:17.852589 25513 raft_consensus.cc:493] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:17.852913 25513 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.860677 25513 raft_consensus.cc:515] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:17.862591 25513 leader_election.cc:290] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 3 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:17.863765 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:17.863646 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87c17d2fac24478924ba677e41fdc68" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:17.864755 25438 raft_consensus.cc:3055] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:17.864820 25717 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.865301 25438 raft_consensus.cc:740] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:17.866336 25438 consensus_queue.cc:260] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:17.867811 25438 raft_consensus.cc:3060] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:17.875253 25717 raft_consensus.cc:2468] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 3.
I20260709 13:02:17.876583 25438 raft_consensus.cc:2468] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 3.
I20260709 13:02:17.877663 25477 leader_election.cc:304] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:17.878679 25792 raft_consensus.cc:2804] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:17.879232 25792 raft_consensus.cc:697] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 3 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:17.880435 25792 consensus_queue.cc:237] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:17.888887 25262 catalog_manager.cc:5697] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 2 to 3, leader changed from 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 3 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:18.045156 25795 raft_consensus.cc:993] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:18.045846 25807 raft_consensus.cc:1081] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:18.047538 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:18.048167 25438 raft_consensus.cc:493] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:18.048555 25438 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.057029 25438 raft_consensus.cc:515] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.058991 25438 leader_election.cc:290] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 5 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:18.060124 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:18.060123 25513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd04fac15ca04d8db197a7b2f5b03d1c" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:18.061051 25717 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.061133 25513 raft_consensus.cc:3055] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:18.061524 25513 raft_consensus.cc:740] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:18.062368 25513 consensus_queue.cc:260] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.063827 25513 raft_consensus.cc:3060] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:18.070005 25717 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 5.
I20260709 13:02:18.072145 25403 leader_election.cc:304] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:18.072566 25513 raft_consensus.cc:2468] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 5.
I20260709 13:02:18.073583 25781 raft_consensus.cc:2804] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:18.074281 25781 raft_consensus.cc:697] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 5 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:18.075260 25781 consensus_queue.cc:237] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.082561 25262 catalog_manager.cc:5697] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 4 to 5, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 5 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:18.136404 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:18.196753 25513 raft_consensus.cc:1275] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:18.198086 25781 consensus_queue.cc:1048] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:18.208169 25717 raft_consensus.cc:1275] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:18.209939 25800 consensus_queue.cc:1048] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:18.270044 25438 raft_consensus.cc:1275] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:18.271374 25807 consensus_queue.cc:1048] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:18.280977 25717 raft_consensus.cc:1275] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:18.283406 25815 consensus_queue.cc:1048] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:18.489820 25717 raft_consensus.cc:1275] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:18.491711 25800 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:18.502060 25513 raft_consensus.cc:1275] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:18.503527 25800 consensus_queue.cc:1048] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:18.603295 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:18.611335 25438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
mode: GRACEFUL
new_leader_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:47114
I20260709 13:02:18.611824 25438 raft_consensus.cc:606] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Received request to transfer leadership to TS 7efd98fd5c384fa09947aac9da4b776b
I20260709 13:02:18.614646 25438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
mode: GRACEFUL
new_leader_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:47114
I20260709 13:02:18.615304 25438 raft_consensus.cc:606] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Received request to transfer leadership to TS 7efd98fd5c384fa09947aac9da4b776b
I20260709 13:02:18.618304 25514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:18.618907 25514 raft_consensus.cc:606] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:18.619280 25787 raft_consensus.cc:993] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf: : Instructing follower 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:18.619853 25800 raft_consensus.cc:1081] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Signalling peer 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:18.620374 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:18.620977 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:18.621572 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:18.621542 25514 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:42700
I20260709 13:02:18.622223 25514 raft_consensus.cc:493] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:18.622552 25514 raft_consensus.cc:3060] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.629523 25514 raft_consensus.cc:515] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.631469 25514 leader_election.cc:290] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:18.632390 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:18.632496 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56cfdecb47214faaa68fb5d2712780d7" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:18.633213 25717 raft_consensus.cc:3060] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.633268 25438 raft_consensus.cc:3055] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:18.633760 25438 raft_consensus.cc:740] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:18.634581 25438 consensus_queue.cc:260] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.635818 25438 raft_consensus.cc:3060] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.639247 25717 raft_consensus.cc:2468] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:18.640666 25477 leader_election.cc:304] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:18.641415 25438 raft_consensus.cc:2468] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:18.641449 25795 raft_consensus.cc:2804] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:18.642289 25795 raft_consensus.cc:697] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:18.643219 25795 consensus_queue.cc:237] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.650192 25262 catalog_manager.cc:5697] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 1 to 2, leader changed from 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 2 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:18.777689 25781 raft_consensus.cc:993] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf: : Instructing follower 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:18.778219 25800 raft_consensus.cc:1081] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Signalling peer 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:18.780160 25514 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:42700
I20260709 13:02:18.780788 25514 raft_consensus.cc:493] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:18.781200 25514 raft_consensus.cc:3060] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.787321 25514 raft_consensus.cc:515] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.789472 25514 leader_election.cc:290] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:18.790547 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:18.791209 25438 raft_consensus.cc:3055] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:18.791721 25438 raft_consensus.cc:740] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:18.791363 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39e51fa0d7af4f7ba2f0e1c4f1362152" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:18.792382 25717 raft_consensus.cc:3060] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.792629 25438 consensus_queue.cc:260] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.793987 25438 raft_consensus.cc:3060] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:18.798910 25717 raft_consensus.cc:2468] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:18.799104 25438 raft_consensus.cc:2468] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 2.
I20260709 13:02:18.800300 25477 leader_election.cc:304] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:18.801288 25795 raft_consensus.cc:2804] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:18.801914 25795 raft_consensus.cc:697] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:18.802697 25795 consensus_queue.cc:237] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:18.810678 25262 catalog_manager.cc:5697] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 1 to 2, leader changed from 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 2 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:19.072362 25792 raft_consensus.cc:993] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:19.072887 25792 raft_consensus.cc:1081] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:19.074515 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:19.074976 25438 raft_consensus.cc:493] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:19.075205 25438 raft_consensus.cc:3060] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.080540 25438 raft_consensus.cc:515] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.082175 25438 leader_election.cc:290] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:19.083151 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:19.083293 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c16a4a39cd54a9eb36628706ea642e3" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:19.084100 25514 raft_consensus.cc:3055] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:19.084102 25717 raft_consensus.cc:3060] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.084661 25514 raft_consensus.cc:740] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:19.085736 25514 consensus_queue.cc:260] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.087126 25514 raft_consensus.cc:3060] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.093442 25717 raft_consensus.cc:2468] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:19.095283 25403 leader_election.cc:304] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:19.095819 25514 raft_consensus.cc:2468] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:19.096225 25800 raft_consensus.cc:2804] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:19.097038 25800 raft_consensus.cc:697] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:19.098110 25800 consensus_queue.cc:237] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.105916 25262 catalog_manager.cc:5697] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 1 to 2, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 2 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:19.119333 25438 raft_consensus.cc:1275] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:19.120669 25792 consensus_queue.cc:1048] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.132882 25717 raft_consensus.cc:1275] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:19.134701 25792 consensus_queue.cc:1048] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:02:19.137547 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:19.306424 25717 raft_consensus.cc:1275] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:19.308527 25792 consensus_queue.cc:1048] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.312069 25438 raft_consensus.cc:1275] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:19.314116 25795 consensus_queue.cc:1048] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:19.578971 25717 raft_consensus.cc:1275] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:19.580585 25781 consensus_queue.cc:1048] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.595005 25514 raft_consensus.cc:1275] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:19.597198 25781 consensus_queue.cc:1048] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:19.628959 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:19.638775 25514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:19.639535 25514 raft_consensus.cc:606] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:19.643004 25514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
mode: GRACEFUL
new_leader_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:53920
I20260709 13:02:19.643615 25514 raft_consensus.cc:606] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Received request to transfer leadership to TS 815ff4fecd7844c2bfb4864215f033cf
I20260709 13:02:19.646886 25438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
mode: GRACEFUL
new_leader_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:47114
I20260709 13:02:19.647459 25438 raft_consensus.cc:606] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Received request to transfer leadership to TS 7efd98fd5c384fa09947aac9da4b776b
I20260709 13:02:19.648831 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:02:19.649770 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:19.651031 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:19.759357 25836 raft_consensus.cc:993] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:19.760010 25836 raft_consensus.cc:1081] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:19.761889 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:19.762405 25438 raft_consensus.cc:493] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:19.762681 25438 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.769938 25438 raft_consensus.cc:515] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.771627 25438 leader_election.cc:290] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 5 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:19.772886 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:19.773007 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cac1690929460f81c7dda80e26cf37" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:19.773640 25514 raft_consensus.cc:3055] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:19.773972 25514 raft_consensus.cc:740] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:19.774112 25717 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.774945 25514 consensus_queue.cc:260] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.776206 25514 raft_consensus.cc:3060] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:19.781492 25717 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 5.
I20260709 13:02:19.782964 25403 leader_election.cc:304] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:19.784178 25781 raft_consensus.cc:2804] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:19.784682 25514 raft_consensus.cc:2468] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 5.
I20260709 13:02:19.784752 25781 raft_consensus.cc:697] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 5 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:19.786101 25781 consensus_queue.cc:237] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.795835 25260 catalog_manager.cc:5697] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 4 to 5, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 5 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:19.812546 25792 raft_consensus.cc:993] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b: : Instructing follower 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:19.813086 25836 raft_consensus.cc:1081] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Signalling peer 815ff4fecd7844c2bfb4864215f033cf to start an election
I20260709 13:02:19.814960 25438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4"
dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
 from {username='slave'} at 127.0.0.1:37282
I20260709 13:02:19.815435 25438 raft_consensus.cc:493] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:19.815699 25438 raft_consensus.cc:3060] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.821782 25438 raft_consensus.cc:515] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.823413 25438 leader_election.cc:290] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67:41807)
I20260709 13:02:19.824597 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:19.824714 25514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e606040784cb3af6133607f421fb4" candidate_uuid: "815ff4fecd7844c2bfb4864215f033cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
I20260709 13:02:19.825278 25717 raft_consensus.cc:3060] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.825628 25514 raft_consensus.cc:3055] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:19.826108 25514 raft_consensus.cc:740] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:19.827136 25514 consensus_queue.cc:260] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.828704 25514 raft_consensus.cc:3060] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:19.833094 25717 raft_consensus.cc:2468] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:19.834995 25403 leader_election.cc:304] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815ff4fecd7844c2bfb4864215f033cf, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:19.836442 25781 raft_consensus.cc:2804] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:19.837302 25781 raft_consensus.cc:697] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Becoming Leader. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:19.837458 25514 raft_consensus.cc:2468] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815ff4fecd7844c2bfb4864215f033cf in term 2.
I20260709 13:02:19.838701 25781 consensus_queue.cc:237] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.848568 25260 catalog_manager.cc:5697] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf reported cstate change: term changed from 1 to 2, leader changed from 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67) to 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66). New cstate: current_term: 2 leader_uuid: "815ff4fecd7844c2bfb4864215f033cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:19.849702 25834 raft_consensus.cc:993] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf: : Instructing follower 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:19.850806 25781 raft_consensus.cc:1081] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Signalling peer 7efd98fd5c384fa09947aac9da4b776b to start an election
I20260709 13:02:19.853245 25514 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf"
dest_uuid: "7efd98fd5c384fa09947aac9da4b776b"
 from {username='slave'} at 127.0.0.1:42700
I20260709 13:02:19.853945 25514 raft_consensus.cc:493] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:19.854509 25514 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.861974 25514 raft_consensus.cc:515] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.863837 25514 leader_election.cc:290] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 4 election: Requested vote from peers e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871), 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721)
I20260709 13:02:19.864684 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "e998827be8264d1d8681d5bfc1277e16"
I20260709 13:02:19.864955 25438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0af08988d44a99a14043e9b39508cf" candidate_uuid: "7efd98fd5c384fa09947aac9da4b776b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "815ff4fecd7844c2bfb4864215f033cf"
I20260709 13:02:19.865451 25717 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.865609 25438 raft_consensus.cc:3055] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:19.866016 25438 raft_consensus.cc:740] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 815ff4fecd7844c2bfb4864215f033cf, State: Running, Role: LEADER
I20260709 13:02:19.867321 25438 consensus_queue.cc:260] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.868409 25438 raft_consensus.cc:3060] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:19.871721 25717 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 4.
I20260709 13:02:19.873600 25477 leader_election.cc:304] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7efd98fd5c384fa09947aac9da4b776b, e998827be8264d1d8681d5bfc1277e16; no voters: 
I20260709 13:02:19.874058 25438 raft_consensus.cc:2468] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7efd98fd5c384fa09947aac9da4b776b in term 4.
I20260709 13:02:19.874408 25836 raft_consensus.cc:2804] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:19.874960 25836 raft_consensus.cc:697] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Becoming Leader. State: Replica: 7efd98fd5c384fa09947aac9da4b776b, State: Running, Role: LEADER
I20260709 13:02:19.875850 25836 consensus_queue.cc:237] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } }
I20260709 13:02:19.883831 25260 catalog_manager.cc:5697] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b reported cstate change: term changed from 3 to 4, leader changed from 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66) to 7efd98fd5c384fa09947aac9da4b776b (127.12.30.67). New cstate: current_term: 4 leader_uuid: "7efd98fd5c384fa09947aac9da4b776b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 } health_report { overall_health: HEALTHY } } }
W20260709 13:02:20.139150 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:20.272787 25716 raft_consensus.cc:1275] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:20.274573 25781 consensus_queue.cc:1048] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:20.294185 25514 raft_consensus.cc:1275] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:20.296334 25847 consensus_queue.cc:1048] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:20.388075 25716 raft_consensus.cc:1275] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:20.390303 25847 consensus_queue.cc:1048] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:20.396922 25438 raft_consensus.cc:1275] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:20.405614 25514 raft_consensus.cc:1275] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Refusing update from remote peer 815ff4fecd7844c2bfb4864215f033cf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:02:20.408664 25834 consensus_queue.cc:1048] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7efd98fd5c384fa09947aac9da4b776b" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 41807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:20.409914 25792 consensus_queue.cc:1048] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "815ff4fecd7844c2bfb4864215f033cf" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:20.429929 25716 raft_consensus.cc:1275] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Refusing update from remote peer 7efd98fd5c384fa09947aac9da4b776b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:02:20.433836 25836 consensus_queue.cc:1048] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e998827be8264d1d8681d5bfc1277e16" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:20.659364 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:20.706971 12409 tablet_server.cc:179] TabletServer@127.12.30.65:35871 shutting down...
I20260709 13:02:20.744424 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:20.745417 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.746361 12409 raft_consensus.cc:2243] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.746893 12409 raft_consensus.cc:2272] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.749971 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.750715 12409 raft_consensus.cc:2243] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.751343 12409 raft_consensus.cc:2272] T fd04fac15ca04d8db197a7b2f5b03d1c P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.754729 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.755409 12409 raft_consensus.cc:2243] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.755846 12409 raft_consensus.cc:2272] T 56cfdecb47214faaa68fb5d2712780d7 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.758347 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.758988 12409 raft_consensus.cc:2243] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.759469 12409 raft_consensus.cc:2272] T 35cac1690929460f81c7dda80e26cf37 P e998827be8264d1d8681d5bfc1277e16 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.763557 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.764276 12409 raft_consensus.cc:2243] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.764703 12409 raft_consensus.cc:2272] T e367fc7664aa486fbbb057c89d54508f P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.767094 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.767639 12409 raft_consensus.cc:2243] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.768159 12409 raft_consensus.cc:2272] T 0d7e606040784cb3af6133607f421fb4 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.772393 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.772993 12409 raft_consensus.cc:2243] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.773445 12409 raft_consensus.cc:2272] T 1caefa4abb3d435db0299d2ffadf77f5 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.776268 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.776813 12409 raft_consensus.cc:2243] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.777231 12409 raft_consensus.cc:2272] T 7c16a4a39cd54a9eb36628706ea642e3 P e998827be8264d1d8681d5bfc1277e16 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.779656 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.780265 12409 raft_consensus.cc:2243] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.780730 12409 raft_consensus.cc:2272] T b87c17d2fac24478924ba677e41fdc68 P e998827be8264d1d8681d5bfc1277e16 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.783209 12409 tablet_replica.cc:333] stopping tablet replica
I20260709 13:02:20.783694 12409 raft_consensus.cc:2243] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.784269 12409 raft_consensus.cc:2272] T db0af08988d44a99a14043e9b39508cf P e998827be8264d1d8681d5bfc1277e16 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:20.803840 25403 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from unknown peer (error 108) [suppressed 8 similar messages]
I20260709 13:02:20.814167 12409 tablet_server.cc:196] TabletServer@127.12.30.65:35871 shutdown complete.
W20260709 13:02:20.814527 25403 consensus_peers.cc:597] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:20.816870 25477 consensus_peers.cc:597] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:20.827363 25403 consensus_peers.cc:597] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:20.828105 25403 consensus_peers.cc:597] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:20.855261 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
W20260709 13:02:20.894692 25479 consensus_peers.cc:597] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b -> Peer 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721): Couldn't send request to peer 815ff4fecd7844c2bfb4864215f033cf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:20.897770 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:20.899093 12409 tablet_replica.cc:333] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.900453 12409 raft_consensus.cc:2243] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.901062 12409 raft_consensus.cc:2272] T db0af08988d44a99a14043e9b39508cf P 815ff4fecd7844c2bfb4864215f033cf [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:20.903141 25477 consensus_peers.cc:597] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:20.905006 12409 tablet_replica.cc:333] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.905705 12409 raft_consensus.cc:2243] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.906137 12409 raft_consensus.cc:2272] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.908542 12409 tablet_replica.cc:333] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.909157 12409 raft_consensus.cc:2243] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.909636 12409 raft_consensus.cc:2272] T 1caefa4abb3d435db0299d2ffadf77f5 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.912055 12409 tablet_replica.cc:333] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.912693 12409 raft_consensus.cc:2243] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.913137 12409 raft_consensus.cc:2272] T b87c17d2fac24478924ba677e41fdc68 P 815ff4fecd7844c2bfb4864215f033cf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.916101 12409 tablet_replica.cc:333] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.916735 12409 raft_consensus.cc:2243] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:20.917626 12409 raft_consensus.cc:2272] T 7c16a4a39cd54a9eb36628706ea642e3 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.920084 12409 tablet_replica.cc:333] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.920715 12409 raft_consensus.cc:2243] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:02:20.921591 12409 raft_consensus.cc:2272] T fd04fac15ca04d8db197a7b2f5b03d1c P 815ff4fecd7844c2bfb4864215f033cf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.923853 12409 tablet_replica.cc:333] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.924554 12409 raft_consensus.cc:2243] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:02:20.925422 12409 raft_consensus.cc:2272] T 35cac1690929460f81c7dda80e26cf37 P 815ff4fecd7844c2bfb4864215f033cf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.929155 12409 tablet_replica.cc:333] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.929881 12409 raft_consensus.cc:2243] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:20.930783 12409 raft_consensus.cc:2272] T e367fc7664aa486fbbb057c89d54508f P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.933724 12409 tablet_replica.cc:333] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.934449 12409 raft_consensus.cc:2243] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:20.935335 12409 raft_consensus.cc:2272] T 0d7e606040784cb3af6133607f421fb4 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:20.938562 12409 tablet_replica.cc:333] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf: stopping tablet replica
I20260709 13:02:20.939186 12409 raft_consensus.cc:2243] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:20.939664 12409 raft_consensus.cc:2272] T 56cfdecb47214faaa68fb5d2712780d7 P 815ff4fecd7844c2bfb4864215f033cf [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:20.968030 25477 consensus_peers.cc:597] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:20.981606 25479 consensus_peers.cc:597] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b -> Peer 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721): Couldn't send request to peer 815ff4fecd7844c2bfb4864215f033cf. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:37721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:20.988098 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
W20260709 13:02:20.992069 25477 consensus_peers.cc:597] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b -> Peer e998827be8264d1d8681d5bfc1277e16 (127.12.30.65:35871): Couldn't send request to peer e998827be8264d1d8681d5bfc1277e16. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:21.029099 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
W20260709 13:02:21.044267 25479 consensus_peers.cc:597] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b -> Peer 815ff4fecd7844c2bfb4864215f033cf (127.12.30.66:37721): Couldn't send request to peer 815ff4fecd7844c2bfb4864215f033cf. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:37721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:21.089989 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:21.090879 12409 tablet_replica.cc:333] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.091640 12409 raft_consensus.cc:2243] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:21.092676 12409 raft_consensus.cc:2272] T 39e51fa0d7af4f7ba2f0e1c4f1362152 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.095402 12409 tablet_replica.cc:333] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.096310 12409 raft_consensus.cc:2243] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:02:21.097540 12409 raft_consensus.cc:2272] T db0af08988d44a99a14043e9b39508cf P 7efd98fd5c384fa09947aac9da4b776b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.101188 12409 tablet_replica.cc:333] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.101943 12409 raft_consensus.cc:2243] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:21.102988 12409 raft_consensus.cc:2272] T 1caefa4abb3d435db0299d2ffadf77f5 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.106448 12409 tablet_replica.cc:333] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.107272 12409 raft_consensus.cc:2243] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:21.107782 12409 raft_consensus.cc:2272] T 35cac1690929460f81c7dda80e26cf37 P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.111581 12409 tablet_replica.cc:333] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.112447 12409 raft_consensus.cc:2243] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:21.112977 12409 raft_consensus.cc:2272] T fd04fac15ca04d8db197a7b2f5b03d1c P 7efd98fd5c384fa09947aac9da4b776b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.115763 12409 tablet_replica.cc:333] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.116453 12409 raft_consensus.cc:2243] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:21.117385 12409 raft_consensus.cc:2272] T 56cfdecb47214faaa68fb5d2712780d7 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.120139 12409 tablet_replica.cc:333] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.120899 12409 raft_consensus.cc:2243] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:21.121413 12409 raft_consensus.cc:2272] T 0d7e606040784cb3af6133607f421fb4 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.125198 12409 tablet_replica.cc:333] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.126202 12409 raft_consensus.cc:2243] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:21.127178 12409 raft_consensus.cc:2272] T b87c17d2fac24478924ba677e41fdc68 P 7efd98fd5c384fa09947aac9da4b776b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.130146 12409 tablet_replica.cc:333] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.130842 12409 raft_consensus.cc:2243] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:21.131417 12409 raft_consensus.cc:2272] T 7c16a4a39cd54a9eb36628706ea642e3 P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.134577 12409 tablet_replica.cc:333] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b: stopping tablet replica
I20260709 13:02:21.135370 12409 raft_consensus.cc:2243] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:21.135978 12409 raft_consensus.cc:2272] T e367fc7664aa486fbbb057c89d54508f P 7efd98fd5c384fa09947aac9da4b776b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:21.140614 25311 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:02:21.177567 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
I20260709 13:02:21.212462 12409 master.cc:562] Master@127.12.30.126:42253 shutting down...
I20260709 13:02:21.242718 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:21.243336 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:21.243674 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2eabe16999d147e6a8d4ea80b3054552: stopping tablet replica
I20260709 13:02:21.265784 12409 master.cc:584] Master@127.12.30.126:42253 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13415 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:02:21.316669 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:44315
I20260709 13:02:21.317930 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:21.324433 25863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:21.324508 25864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:21.324915 25866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:21.325793 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:21.326921 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:21.327108 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:21.327226 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602141327208 us; error 0 us; skew 500 ppm
I20260709 13:02:21.327690 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:21.330551 12409 webserver.cc:533] Webserver started at http://127.12.30.126:38967/ using document root <none> and password file <none>
I20260709 13:02:21.331081 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:21.331243 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:21.331465 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:21.332711 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "0e98bcac70104497982156ca714db1a3"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.337751 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:02:21.341817 25871 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.342790 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:02:21.343130 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "0e98bcac70104497982156ca714db1a3"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.343410 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:21.379419 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:21.380925 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:21.432147 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:44315
I20260709 13:02:21.432260 25922 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:44315 every 8 connection(s)
I20260709 13:02:21.436429 25923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:21.449175 25923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3: Bootstrap starting.
I20260709 13:02:21.454298 25923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:21.458925 25923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3: No bootstrap required, opened a new log
I20260709 13:02:21.461257 25923 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e98bcac70104497982156ca714db1a3" member_type: VOTER }
I20260709 13:02:21.461905 25923 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:21.462137 25923 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e98bcac70104497982156ca714db1a3, State: Initialized, Role: FOLLOWER
I20260709 13:02:21.462687 25923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e98bcac70104497982156ca714db1a3" member_type: VOTER }
I20260709 13:02:21.463124 25923 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:02:21.463318 25923 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:02:21.463524 25923 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:21.469254 25923 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e98bcac70104497982156ca714db1a3" member_type: VOTER }
I20260709 13:02:21.469789 25923 leader_election.cc:304] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e98bcac70104497982156ca714db1a3; no voters: 
I20260709 13:02:21.470989 25923 leader_election.cc:290] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:02:21.471366 25926 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:21.472898 25926 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 1 LEADER]: Becoming Leader. State: Replica: 0e98bcac70104497982156ca714db1a3, State: Running, Role: LEADER
I20260709 13:02:21.473513 25926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e98bcac70104497982156ca714db1a3" member_type: VOTER }
I20260709 13:02:21.474079 25923 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:02:21.479101 25928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e98bcac70104497982156ca714db1a3. Latest consensus state: current_term: 1 leader_uuid: "0e98bcac70104497982156ca714db1a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e98bcac70104497982156ca714db1a3" member_type: VOTER } }
I20260709 13:02:21.480139 25928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:21.479004 25927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e98bcac70104497982156ca714db1a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e98bcac70104497982156ca714db1a3" member_type: VOTER } }
I20260709 13:02:21.481145 25927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:21.482633 25934 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:02:21.487529 25934 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:02:21.490594 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:02:21.497120 25934 catalog_manager.cc:1383] Generated new cluster ID: 6c5fc7034e4b46488cdd28c4bb2bf181
I20260709 13:02:21.497459 25934 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:02:21.527412 25934 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:02:21.528867 25934 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:02:21.543439 25934 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3: Generated new TSK 0
I20260709 13:02:21.544108 25934 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:02:21.558509 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:21.566783 25944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:21.568156 25945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:21.569854 12409 server_base.cc:1061] running on GCE node
W20260709 13:02:21.569908 25947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:21.571138 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:21.571364 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:21.571491 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602141571479 us; error 0 us; skew 500 ppm
I20260709 13:02:21.572050 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:21.574409 12409 webserver.cc:533] Webserver started at http://127.12.30.65:37091/ using document root <none> and password file <none>
I20260709 13:02:21.574891 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:21.575048 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:21.575254 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:21.576402 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "a5a518325a4e4daa8ad361de80bf53dc"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.581409 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:02:21.584956 25952 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.585837 12409 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:02:21.586115 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "a5a518325a4e4daa8ad361de80bf53dc"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.586376 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:21.609263 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:21.610509 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:21.612080 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:21.614643 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:21.614840 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.615156 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:21.615331 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.661329 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:33909
I20260709 13:02:21.661463 26014 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:33909 every 8 connection(s)
I20260709 13:02:21.666996 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:21.674499 26019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:21.674825 26020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:21.678615 26022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:21.679281 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:21.680692 26015 heartbeater.cc:344] Connected to a master server at 127.12.30.126:44315
I20260709 13:02:21.681051 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:02:21.681241 26015 heartbeater.cc:461] Registering TS with master...
W20260709 13:02:21.681375 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:21.681668 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602141681649 us; error 0 us; skew 500 ppm
I20260709 13:02:21.682320 26015 heartbeater.cc:507] Master 127.12.30.126:44315 requested a full tablet report, sending...
I20260709 13:02:21.682590 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:21.685067 25888 ts_manager.cc:194] Registered new tserver with Master: a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909)
I20260709 13:02:21.685678 12409 webserver.cc:533] Webserver started at http://127.12.30.66:42861/ using document root <none> and password file <none>
I20260709 13:02:21.686296 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:21.686529 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:21.686815 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:21.687211 25888 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43518
I20260709 13:02:21.688524 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "c564eb4b67bc41549d81bddc552f05a5"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.694423 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 13:02:21.698439 26027 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.699421 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:02:21.699687 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "c564eb4b67bc41549d81bddc552f05a5"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.700031 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:21.722683 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:21.723861 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:21.725387 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:21.727726 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:21.727906 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.728230 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:21.728404 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.772553 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:33129
I20260709 13:02:21.772653 26089 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:33129 every 8 connection(s)
I20260709 13:02:21.777594 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:21.784919 26093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:21.788393 26094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:21.788578 12409 server_base.cc:1061] running on GCE node
W20260709 13:02:21.790185 26096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:21.791589 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:21.791953 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:21.792395 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602141792373 us; error 0 us; skew 500 ppm
I20260709 13:02:21.793434 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:21.793431 26090 heartbeater.cc:344] Connected to a master server at 127.12.30.126:44315
I20260709 13:02:21.793999 26090 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:21.795215 26090 heartbeater.cc:507] Master 127.12.30.126:44315 requested a full tablet report, sending...
I20260709 13:02:21.796804 12409 webserver.cc:533] Webserver started at http://127.12.30.67:45365/ using document root <none> and password file <none>
I20260709 13:02:21.797618 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:21.797611 25888 ts_manager.cc:194] Registered new tserver with Master: c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66:33129)
I20260709 13:02:21.798166 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:21.798501 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:21.799760 25888 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43530
I20260709 13:02:21.799906 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "ca043e13cd9842c28ab38bc0d6de7372"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.805058 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:02:21.809422 26101 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.810427 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:02:21.810775 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "ca043e13cd9842c28ab38bc0d6de7372"
format_stamp: "Formatted at 2026-07-09 13:02:21 on dist-test-slave-mlt7"
I20260709 13:02:21.811209 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:21.834600 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:21.836056 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:21.837733 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:21.840396 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:21.840673 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.840986 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:21.841180 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:21.888695 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:32829
I20260709 13:02:21.888813 26163 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:32829 every 8 connection(s)
I20260709 13:02:21.905233 26164 heartbeater.cc:344] Connected to a master server at 127.12.30.126:44315
I20260709 13:02:21.905755 26164 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:21.906493 26164 heartbeater.cc:507] Master 127.12.30.126:44315 requested a full tablet report, sending...
I20260709 13:02:21.908497 25888 ts_manager.cc:194] Registered new tserver with Master: ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:21.908629 12409 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016747547s
I20260709 13:02:21.910024 25888 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43540
I20260709 13:02:21.936980 25888 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43546:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:21.939780 25888 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:21.988481 26055 tablet_service.cc:1511] Processing CreateTablet for tablet 14e20aecd5f0432b9129a6c0f42c28d1 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5f7bbe57ce604c7b90840d735d963d7f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:02:21.988482 26054 tablet_service.cc:1511] Processing CreateTablet for tablet 73f8dfc5cbda4f05bb7709d8559b8fe5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5f7bbe57ce604c7b90840d735d963d7f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:02:21.990074 26055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e20aecd5f0432b9129a6c0f42c28d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:21.990664 26054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f8dfc5cbda4f05bb7709d8559b8fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.000453 25980 tablet_service.cc:1511] Processing CreateTablet for tablet 14e20aecd5f0432b9129a6c0f42c28d1 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5f7bbe57ce604c7b90840d735d963d7f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:02:22.001588 25979 tablet_service.cc:1511] Processing CreateTablet for tablet 73f8dfc5cbda4f05bb7709d8559b8fe5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5f7bbe57ce604c7b90840d735d963d7f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:02:22.002297 25980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e20aecd5f0432b9129a6c0f42c28d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.003183 25979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f8dfc5cbda4f05bb7709d8559b8fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.020671 26129 tablet_service.cc:1511] Processing CreateTablet for tablet 14e20aecd5f0432b9129a6c0f42c28d1 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5f7bbe57ce604c7b90840d735d963d7f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:02:22.020882 26128 tablet_service.cc:1511] Processing CreateTablet for tablet 73f8dfc5cbda4f05bb7709d8559b8fe5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5f7bbe57ce604c7b90840d735d963d7f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:02:22.022418 26129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e20aecd5f0432b9129a6c0f42c28d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.024771 26128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f8dfc5cbda4f05bb7709d8559b8fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.038641 26179 tablet_bootstrap.cc:492] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5: Bootstrap starting.
I20260709 13:02:22.050948 26179 tablet_bootstrap.cc:654] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.053366 26178 tablet_bootstrap.cc:492] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc: Bootstrap starting.
I20260709 13:02:22.058852 26179 tablet_bootstrap.cc:492] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5: No bootstrap required, opened a new log
I20260709 13:02:22.059811 26179 ts_tablet_manager.cc:1403] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.005s
I20260709 13:02:22.061098 26178 tablet_bootstrap.cc:654] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.063583 26181 tablet_bootstrap.cc:492] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372: Bootstrap starting.
I20260709 13:02:22.063759 26179 raft_consensus.cc:359] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.064827 26179 raft_consensus.cc:385] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.065272 26179 raft_consensus.cc:740] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.066460 26179 consensus_queue.cc:260] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.071025 26178 tablet_bootstrap.cc:492] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc: No bootstrap required, opened a new log
I20260709 13:02:22.071604 26178 ts_tablet_manager.cc:1403] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 13:02:22.072523 26090 heartbeater.cc:499] Master 127.12.30.126:44315 was elected leader, sending a full tablet report...
I20260709 13:02:22.073254 26179 ts_tablet_manager.cc:1434] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:02:22.074332 26179 tablet_bootstrap.cc:492] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5: Bootstrap starting.
I20260709 13:02:22.074829 26181 tablet_bootstrap.cc:654] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.075450 26178 raft_consensus.cc:359] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.076370 26178 raft_consensus.cc:385] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.076989 26178 raft_consensus.cc:740] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.077745 26178 consensus_queue.cc:260] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.083246 26179 tablet_bootstrap.cc:654] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.096210 26015 heartbeater.cc:499] Master 127.12.30.126:44315 was elected leader, sending a full tablet report...
I20260709 13:02:22.096818 26178 ts_tablet_manager.cc:1434] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.011s
I20260709 13:02:22.098130 26178 tablet_bootstrap.cc:492] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc: Bootstrap starting.
I20260709 13:02:22.099041 26181 tablet_bootstrap.cc:492] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372: No bootstrap required, opened a new log
I20260709 13:02:22.099635 26181 ts_tablet_manager.cc:1403] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.016s
I20260709 13:02:22.100016 26179 tablet_bootstrap.cc:492] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5: No bootstrap required, opened a new log
I20260709 13:02:22.100564 26179 ts_tablet_manager.cc:1403] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5: Time spent bootstrapping tablet: real 0.026s	user 0.021s	sys 0.001s
I20260709 13:02:22.102706 26181 raft_consensus.cc:359] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.103535 26181 raft_consensus.cc:385] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.104012 26181 raft_consensus.cc:740] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca043e13cd9842c28ab38bc0d6de7372, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.103554 26179 raft_consensus.cc:359] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.104514 26179 raft_consensus.cc:385] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.104835 26179 raft_consensus.cc:740] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.105131 26178 tablet_bootstrap.cc:654] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.105911 26179 consensus_queue.cc:260] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.105012 26181 consensus_queue.cc:260] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.116928 26179 ts_tablet_manager.cc:1434] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:02:22.119120 26164 heartbeater.cc:499] Master 127.12.30.126:44315 was elected leader, sending a full tablet report...
I20260709 13:02:22.119796 26181 ts_tablet_manager.cc:1434] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 13:02:22.122354 26181 tablet_bootstrap.cc:492] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372: Bootstrap starting.
I20260709 13:02:22.124138 26178 tablet_bootstrap.cc:492] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc: No bootstrap required, opened a new log
I20260709 13:02:22.124715 26178 ts_tablet_manager.cc:1403] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.011s
I20260709 13:02:22.127707 26178 raft_consensus.cc:359] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.128648 26178 raft_consensus.cc:385] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.128958 26178 raft_consensus.cc:740] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.129165 26181 tablet_bootstrap.cc:654] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.129802 26178 consensus_queue.cc:260] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.131875 26178 ts_tablet_manager.cc:1434] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:02:22.138484 26181 tablet_bootstrap.cc:492] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372: No bootstrap required, opened a new log
I20260709 13:02:22.139072 26181 ts_tablet_manager.cc:1403] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.002s
I20260709 13:02:22.141537 26181 raft_consensus.cc:359] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.142185 26181 raft_consensus.cc:385] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.142452 26181 raft_consensus.cc:740] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca043e13cd9842c28ab38bc0d6de7372, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.143368 26181 consensus_queue.cc:260] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.146114 26181 ts_tablet_manager.cc:1434] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:22.284726 26183 raft_consensus.cc:493] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.285586 26183 raft_consensus.cc:515] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.288695 26183 leader_election.cc:290] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:22.301923 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc" is_pre_election: true
I20260709 13:02:22.302829 25990 raft_consensus.cc:2468] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 0.
I20260709 13:02:22.303654 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372" is_pre_election: true
I20260709 13:02:22.304136 26029 leader_election.cc:304] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5a518325a4e4daa8ad361de80bf53dc, c564eb4b67bc41549d81bddc552f05a5; no voters: 
I20260709 13:02:22.304597 26139 raft_consensus.cc:2468] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 0.
I20260709 13:02:22.305226 26183 raft_consensus.cc:2804] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:22.305586 26183 raft_consensus.cc:493] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.305883 26183 raft_consensus.cc:3060] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.314913 26183 raft_consensus.cc:515] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.317582 26183 leader_election.cc:290] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 election: Requested vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:22.318343 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
I20260709 13:02:22.319162 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
I20260709 13:02:22.319173 25990 raft_consensus.cc:3060] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.319852 26139 raft_consensus.cc:3060] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.326736 26139 raft_consensus.cc:2468] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 1.
I20260709 13:02:22.327216 25990 raft_consensus.cc:2468] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 1.
I20260709 13:02:22.328099 26029 leader_election.cc:304] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c564eb4b67bc41549d81bddc552f05a5, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:22.329149 26183 raft_consensus.cc:2804] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:22.330658 26183 raft_consensus.cc:697] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Becoming Leader. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Running, Role: LEADER
I20260709 13:02:22.331830 26183 consensus_queue.cc:237] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.342715 25887 catalog_manager.cc:5697] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "c564eb4b67bc41549d81bddc552f05a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.431874 26183 raft_consensus.cc:493] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.432641 26183 raft_consensus.cc:515] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.434806 26183 leader_election.cc:290] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:22.436254 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f8dfc5cbda4f05bb7709d8559b8fe5" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc" is_pre_election: true
I20260709 13:02:22.436730 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f8dfc5cbda4f05bb7709d8559b8fe5" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372" is_pre_election: true
I20260709 13:02:22.436982 25990 raft_consensus.cc:2468] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 0.
I20260709 13:02:22.437845 26139 raft_consensus.cc:2468] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 0.
I20260709 13:02:22.438629 26029 leader_election.cc:304] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5a518325a4e4daa8ad361de80bf53dc, c564eb4b67bc41549d81bddc552f05a5; no voters: 
I20260709 13:02:22.439541 26183 raft_consensus.cc:2804] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:22.439909 26183 raft_consensus.cc:493] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.440263 26183 raft_consensus.cc:3060] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.447724 26183 raft_consensus.cc:515] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.449990 26183 leader_election.cc:290] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 election: Requested vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:22.452139 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f8dfc5cbda4f05bb7709d8559b8fe5" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
I20260709 13:02:22.451858 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f8dfc5cbda4f05bb7709d8559b8fe5" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
I20260709 13:02:22.452770 26139 raft_consensus.cc:3060] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.452848 25990 raft_consensus.cc:3060] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.461895 26139 raft_consensus.cc:2468] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 1.
I20260709 13:02:22.461895 25990 raft_consensus.cc:2468] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 1.
I20260709 13:02:22.463542 26029 leader_election.cc:304] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c564eb4b67bc41549d81bddc552f05a5, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:22.464375 26183 raft_consensus.cc:2804] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:22.464846 26183 raft_consensus.cc:697] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Becoming Leader. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Running, Role: LEADER
I20260709 13:02:22.465771 26183 consensus_queue.cc:237] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.474012 25887 catalog_manager.cc:5697] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "c564eb4b67bc41549d81bddc552f05a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:22.486605 25936 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:22.628762 25887 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43548:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:22.631765 25887 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:22.661949 25980 tablet_service.cc:1511] Processing CreateTablet for tablet b851918cdd314632a40e94a964142729 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=40c8909a5d5f4b8e8960efa54de146b3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:02:22.663044 26055 tablet_service.cc:1511] Processing CreateTablet for tablet b851918cdd314632a40e94a964142729 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=40c8909a5d5f4b8e8960efa54de146b3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:02:22.663640 25980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b851918cdd314632a40e94a964142729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.664620 26055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b851918cdd314632a40e94a964142729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.665092 26128 tablet_service.cc:1511] Processing CreateTablet for tablet b851918cdd314632a40e94a964142729 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=40c8909a5d5f4b8e8960efa54de146b3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:02:22.666404 26128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b851918cdd314632a40e94a964142729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.666726 25979 tablet_service.cc:1511] Processing CreateTablet for tablet afa679c262414b07a5f95a7f50407b99 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=40c8909a5d5f4b8e8960efa54de146b3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:02:22.668015 25979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa679c262414b07a5f95a7f50407b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.686129 26054 tablet_service.cc:1511] Processing CreateTablet for tablet afa679c262414b07a5f95a7f50407b99 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=40c8909a5d5f4b8e8960efa54de146b3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:02:22.688402 26054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa679c262414b07a5f95a7f50407b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.700664 26202 tablet_bootstrap.cc:492] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5: Bootstrap starting.
I20260709 13:02:22.700969 26201 tablet_bootstrap.cc:492] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: Bootstrap starting.
I20260709 13:02:22.704187 26128 tablet_service.cc:1511] Processing CreateTablet for tablet afa679c262414b07a5f95a7f50407b99 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=40c8909a5d5f4b8e8960efa54de146b3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:02:22.705861 26128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa679c262414b07a5f95a7f50407b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:22.708272 26202 tablet_bootstrap.cc:654] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.716360 26201 tablet_bootstrap.cc:654] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.724931 26183 consensus_queue.cc:1048] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:22.729302 26200 tablet_bootstrap.cc:492] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc: Bootstrap starting.
I20260709 13:02:22.731714 26202 tablet_bootstrap.cc:492] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5: No bootstrap required, opened a new log
I20260709 13:02:22.735037 26201 tablet_bootstrap.cc:492] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: No bootstrap required, opened a new log
I20260709 13:02:22.735771 26201 ts_tablet_manager.cc:1403] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.003s
I20260709 13:02:22.735107 26202 ts_tablet_manager.cc:1403] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.007s
I20260709 13:02:22.740352 26201 raft_consensus.cc:359] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.741313 26200 tablet_bootstrap.cc:654] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.741374 26201 raft_consensus.cc:385] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.744603 26201 raft_consensus.cc:740] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca043e13cd9842c28ab38bc0d6de7372, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.744005 26202 raft_consensus.cc:359] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.745944 26201 consensus_queue.cc:260] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.746840 26202 raft_consensus.cc:385] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.747267 26202 raft_consensus.cc:740] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.748251 26202 consensus_queue.cc:260] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.752794 26202 ts_tablet_manager.cc:1434] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:02:22.767355 26201 ts_tablet_manager.cc:1434] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.009s
I20260709 13:02:22.769901 26201 tablet_bootstrap.cc:492] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372: Bootstrap starting.
I20260709 13:02:22.780155 26200 tablet_bootstrap.cc:492] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc: No bootstrap required, opened a new log
I20260709 13:02:22.780866 26200 ts_tablet_manager.cc:1403] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.008s
I20260709 13:02:22.782390 26201 tablet_bootstrap.cc:654] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.794587 26200 raft_consensus.cc:359] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.796654 26202 tablet_bootstrap.cc:492] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5: Bootstrap starting.
I20260709 13:02:22.795819 26200 raft_consensus.cc:385] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.801328 26191 consensus_queue.cc:1048] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:22.802366 26200 raft_consensus.cc:740] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.809110 26202 tablet_bootstrap.cc:654] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.817787 26201 tablet_bootstrap.cc:492] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372: No bootstrap required, opened a new log
I20260709 13:02:22.818889 26201 ts_tablet_manager.cc:1403] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372: Time spent bootstrapping tablet: real 0.049s	user 0.020s	sys 0.021s
I20260709 13:02:22.811074 26200 consensus_queue.cc:260] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.822971 26201 raft_consensus.cc:359] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.823836 26201 raft_consensus.cc:385] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.824178 26201 raft_consensus.cc:740] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca043e13cd9842c28ab38bc0d6de7372, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.824267 26202 tablet_bootstrap.cc:492] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5: No bootstrap required, opened a new log
I20260709 13:02:22.825001 26202 ts_tablet_manager.cc:1403] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.008s
I20260709 13:02:22.825235 26201 consensus_queue.cc:260] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.826099 26200 ts_tablet_manager.cc:1434] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc: Time spent starting tablet: real 0.045s	user 0.011s	sys 0.000s
I20260709 13:02:22.827373 26200 tablet_bootstrap.cc:492] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc: Bootstrap starting.
I20260709 13:02:22.828428 26201 ts_tablet_manager.cc:1434] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:02:22.828698 26202 raft_consensus.cc:359] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.830950 26202 raft_consensus.cc:385] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.831290 26202 raft_consensus.cc:740] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.832919 26202 consensus_queue.cc:260] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.834961 26191 raft_consensus.cc:493] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.835526 26191 raft_consensus.cc:515] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.840250 26202 ts_tablet_manager.cc:1434] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.008s
I20260709 13:02:22.844182 26191 leader_election.cc:290] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:22.845757 26200 tablet_bootstrap.cc:654] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:22.845832 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc" is_pre_election: true
I20260709 13:02:22.846519 26138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372" is_pre_election: true
I20260709 13:02:22.847366 26138 raft_consensus.cc:2468] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 0.
W20260709 13:02:22.847530 26029 leader_election.cc:343] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:22.850886 26029 leader_election.cc:304] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c564eb4b67bc41549d81bddc552f05a5, ca043e13cd9842c28ab38bc0d6de7372; no voters: a5a518325a4e4daa8ad361de80bf53dc
I20260709 13:02:22.851992 26191 raft_consensus.cc:2804] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:22.852506 26191 raft_consensus.cc:493] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.852972 26191 raft_consensus.cc:3060] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.864933 26191 raft_consensus.cc:515] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.865418 26200 tablet_bootstrap.cc:492] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc: No bootstrap required, opened a new log
I20260709 13:02:22.866195 26200 ts_tablet_manager.cc:1403] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.001s
I20260709 13:02:22.867233 26191 leader_election.cc:290] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 election: Requested vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:22.868449 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
I20260709 13:02:22.869151 26139 raft_consensus.cc:3060] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.869293 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99" candidate_uuid: "c564eb4b67bc41549d81bddc552f05a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
I20260709 13:02:22.869192 26200 raft_consensus.cc:359] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.870270 26200 raft_consensus.cc:385] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:22.870544 26200 raft_consensus.cc:740] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Initialized, Role: FOLLOWER
I20260709 13:02:22.871292 26200 consensus_queue.cc:260] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.872890 25990 raft_consensus.cc:3060] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:22.875515 26200 ts_tablet_manager.cc:1434] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:02:22.879565 26139 raft_consensus.cc:2468] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 1.
I20260709 13:02:22.881031 26029 leader_election.cc:304] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c564eb4b67bc41549d81bddc552f05a5, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:22.882149 26191 raft_consensus.cc:2804] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:22.882854 26191 raft_consensus.cc:697] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Becoming Leader. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Running, Role: LEADER
I20260709 13:02:22.883224 25990 raft_consensus.cc:2468] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c564eb4b67bc41549d81bddc552f05a5 in term 1.
I20260709 13:02:22.883709 26191 consensus_queue.cc:237] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.893893 25885 catalog_manager.cc:5697] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "c564eb4b67bc41549d81bddc552f05a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:22.964634 26206 raft_consensus.cc:493] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.965412 26206 raft_consensus.cc:515] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:22.968019 26206 leader_election.cc:290] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66:33129)
I20260709 13:02:22.988376 26191 consensus_queue.cc:1048] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:23.000218 25989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "ca043e13cd9842c28ab38bc0d6de7372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc" is_pre_election: true
I20260709 13:02:23.001729 25989 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca043e13cd9842c28ab38bc0d6de7372 in term 0.
I20260709 13:02:23.003017 26065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "ca043e13cd9842c28ab38bc0d6de7372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c564eb4b67bc41549d81bddc552f05a5" is_pre_election: true
I20260709 13:02:23.003670 26103 leader_election.cc:304] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5a518325a4e4daa8ad361de80bf53dc, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:23.003965 26065 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca043e13cd9842c28ab38bc0d6de7372 in term 0.
I20260709 13:02:23.004837 26206 raft_consensus.cc:2804] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:23.005255 26206 raft_consensus.cc:493] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.005616 26206 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:23.014478 26222 consensus_queue.cc:1048] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:23.021055 26206 raft_consensus.cc:515] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.025437 26206 leader_election.cc:290] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [CANDIDATE]: Term 1 election: Requested vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66:33129)
I20260709 13:02:23.025799 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "ca043e13cd9842c28ab38bc0d6de7372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
I20260709 13:02:23.026780 25990 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:23.026396 26065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "ca043e13cd9842c28ab38bc0d6de7372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c564eb4b67bc41549d81bddc552f05a5"
I20260709 13:02:23.027456 26065 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:23.042160 25990 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca043e13cd9842c28ab38bc0d6de7372 in term 1.
I20260709 13:02:23.044018 26103 leader_election.cc:304] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5a518325a4e4daa8ad361de80bf53dc, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:23.047639 26065 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca043e13cd9842c28ab38bc0d6de7372 in term 1.
I20260709 13:02:23.048193 26206 raft_consensus.cc:2804] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:23.049710 26206 raft_consensus.cc:697] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 LEADER]: Becoming Leader. State: Replica: ca043e13cd9842c28ab38bc0d6de7372, State: Running, Role: LEADER
I20260709 13:02:23.050587 26206 consensus_queue.cc:237] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.061012 25886 catalog_manager.cc:5697] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "ca043e13cd9842c28ab38bc0d6de7372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:23.139528 26065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1"
dest_uuid: "c564eb4b67bc41549d81bddc552f05a5"
mode: GRACEFUL
new_leader_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
 from {username='slave'} at 127.0.0.1:50342
I20260709 13:02:23.140317 26065 raft_consensus.cc:606] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Received request to transfer leadership to TS a5a518325a4e4daa8ad361de80bf53dc
I20260709 13:02:23.168978 26139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b851918cdd314632a40e94a964142729"
dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
mode: GRACEFUL
new_leader_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
 from {username='slave'} at 127.0.0.1:38992
I20260709 13:02:23.169734 26139 raft_consensus.cc:606] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 LEADER]: Received request to transfer leadership to TS a5a518325a4e4daa8ad361de80bf53dc
I20260709 13:02:23.177040 26065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99"
dest_uuid: "c564eb4b67bc41549d81bddc552f05a5"
mode: GRACEFUL
new_leader_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
 from {username='slave'} at 127.0.0.1:50342
I20260709 13:02:23.177788 26065 raft_consensus.cc:606] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Received request to transfer leadership to TS a5a518325a4e4daa8ad361de80bf53dc
I20260709 13:02:23.223614 26222 raft_consensus.cc:993] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5: : Instructing follower a5a518325a4e4daa8ad361de80bf53dc to start an election
I20260709 13:02:23.224198 26222 raft_consensus.cc:1081] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Signalling peer a5a518325a4e4daa8ad361de80bf53dc to start an election
I20260709 13:02:23.225979 25990 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1"
dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
 from {username='slave'} at 127.0.0.1:34222
I20260709 13:02:23.226584 25990 raft_consensus.cc:493] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:23.226928 25990 raft_consensus.cc:3060] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.234683 25990 raft_consensus.cc:515] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.241215 25990 leader_election.cc:290] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [CANDIDATE]: Term 2 election: Requested vote from peers c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66:33129), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:23.265700 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1" candidate_uuid: "a5a518325a4e4daa8ad361de80bf53dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
I20260709 13:02:23.266443 26139 raft_consensus.cc:3060] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.267202 26065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e20aecd5f0432b9129a6c0f42c28d1" candidate_uuid: "a5a518325a4e4daa8ad361de80bf53dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c564eb4b67bc41549d81bddc552f05a5"
I20260709 13:02:23.267980 26065 raft_consensus.cc:3055] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:23.268373 26065 raft_consensus.cc:740] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Running, Role: LEADER
I20260709 13:02:23.269272 26065 consensus_queue.cc:260] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.273433 26065 raft_consensus.cc:3060] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.278090 26139 raft_consensus.cc:2468] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a518325a4e4daa8ad361de80bf53dc in term 2.
I20260709 13:02:23.280011 25954 leader_election.cc:304] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5a518325a4e4daa8ad361de80bf53dc, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:23.281288 26205 raft_consensus.cc:2804] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:23.283450 26065 raft_consensus.cc:2468] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a518325a4e4daa8ad361de80bf53dc in term 2.
I20260709 13:02:23.283644 26205 raft_consensus.cc:697] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Becoming Leader. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Running, Role: LEADER
I20260709 13:02:23.285895 26205 consensus_queue.cc:237] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.293931 25886 catalog_manager.cc:5697] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc reported cstate change: term changed from 1 to 2, leader changed from c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66) to a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65). New cstate: current_term: 2 leader_uuid: "a5a518325a4e4daa8ad361de80bf53dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:23.322253 26222 consensus_queue.cc:1048] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:23.336126 26222 consensus_queue.cc:1048] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:23.346515 26191 raft_consensus.cc:993] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5: : Instructing follower a5a518325a4e4daa8ad361de80bf53dc to start an election
I20260709 13:02:23.347065 26191 raft_consensus.cc:1081] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Signalling peer a5a518325a4e4daa8ad361de80bf53dc to start an election
I20260709 13:02:23.349026 25990 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99"
dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
 from {username='slave'} at 127.0.0.1:34222
I20260709 13:02:23.349615 25990 raft_consensus.cc:493] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:23.350101 25990 raft_consensus.cc:3060] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.356531 25990 raft_consensus.cc:515] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.359472 25990 leader_election.cc:290] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [CANDIDATE]: Term 2 election: Requested vote from peers c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66:33129), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:23.359649 26065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99" candidate_uuid: "a5a518325a4e4daa8ad361de80bf53dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c564eb4b67bc41549d81bddc552f05a5"
I20260709 13:02:23.360217 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa679c262414b07a5f95a7f50407b99" candidate_uuid: "a5a518325a4e4daa8ad361de80bf53dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
I20260709 13:02:23.360467 26065 raft_consensus.cc:3055] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:23.360877 26065 raft_consensus.cc:740] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c564eb4b67bc41549d81bddc552f05a5, State: Running, Role: LEADER
I20260709 13:02:23.360984 26139 raft_consensus.cc:3060] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.361842 26065 consensus_queue.cc:260] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.363190 26065 raft_consensus.cc:3060] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.369035 26139 raft_consensus.cc:2468] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a518325a4e4daa8ad361de80bf53dc in term 2.
I20260709 13:02:23.370340 25954 leader_election.cc:304] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5a518325a4e4daa8ad361de80bf53dc, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:23.371054 26065 raft_consensus.cc:2468] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a518325a4e4daa8ad361de80bf53dc in term 2.
I20260709 13:02:23.371250 26205 raft_consensus.cc:2804] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:23.371881 26205 raft_consensus.cc:697] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Becoming Leader. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Running, Role: LEADER
I20260709 13:02:23.372874 26205 consensus_queue.cc:237] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.380949 25888 catalog_manager.cc:5697] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc reported cstate change: term changed from 1 to 2, leader changed from c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66) to a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65). New cstate: current_term: 2 leader_uuid: "a5a518325a4e4daa8ad361de80bf53dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:23.597493 26206 consensus_queue.cc:1048] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:23.610905 26206 consensus_queue.cc:1048] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:23.622220 26235 raft_consensus.cc:993] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: : Instructing follower a5a518325a4e4daa8ad361de80bf53dc to start an election
I20260709 13:02:23.622754 26206 raft_consensus.cc:1081] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 LEADER]: Signalling peer a5a518325a4e4daa8ad361de80bf53dc to start an election
I20260709 13:02:23.624472 25990 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b851918cdd314632a40e94a964142729"
dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
 from {username='slave'} at 127.0.0.1:34234
I20260709 13:02:23.625339 25990 raft_consensus.cc:493] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:23.625741 25990 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.631366 25990 raft_consensus.cc:515] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.633615 25990 leader_election.cc:290] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [CANDIDATE]: Term 2 election: Requested vote from peers c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66:33129), ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67:32829)
I20260709 13:02:23.634413 26065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "a5a518325a4e4daa8ad361de80bf53dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c564eb4b67bc41549d81bddc552f05a5"
I20260709 13:02:23.634948 26139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "a5a518325a4e4daa8ad361de80bf53dc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
I20260709 13:02:23.635190 26065 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.635555 26139 raft_consensus.cc:3055] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:23.635957 26139 raft_consensus.cc:740] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca043e13cd9842c28ab38bc0d6de7372, State: Running, Role: LEADER
I20260709 13:02:23.636847 26139 consensus_queue.cc:260] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.638207 26139 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:23.641603 26065 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a518325a4e4daa8ad361de80bf53dc in term 2.
I20260709 13:02:23.642765 25953 leader_election.cc:304] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5a518325a4e4daa8ad361de80bf53dc, c564eb4b67bc41549d81bddc552f05a5; no voters: 
I20260709 13:02:23.643626 26205 raft_consensus.cc:2804] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:23.644024 26139 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5a518325a4e4daa8ad361de80bf53dc in term 2.
I20260709 13:02:23.644295 26205 raft_consensus.cc:697] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Becoming Leader. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Running, Role: LEADER
I20260709 13:02:23.645447 26205 consensus_queue.cc:237] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:23.652575 25888 catalog_manager.cc:5697] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc reported cstate change: term changed from 1 to 2, leader changed from ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67) to a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65). New cstate: current_term: 2 leader_uuid: "a5a518325a4e4daa8ad361de80bf53dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:23.686352 12409 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:02:23.688059 12409 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:02:23.688548 12409 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:02:23.703863 25990 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b851918cdd314632a40e94a964142729"
dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
mode: GRACEFUL
new_leader_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
 from {username='slave'} at 127.0.0.1:34238
I20260709 13:02:23.704789 25990 raft_consensus.cc:606] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Received request to transfer leadership to TS ca043e13cd9842c28ab38bc0d6de7372
I20260709 13:02:23.716666 12409 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:02:23.741791 26139 raft_consensus.cc:1275] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Refusing update from remote peer a5a518325a4e4daa8ad361de80bf53dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:23.751591 26205 consensus_queue.cc:1048] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:23.794142 26065 raft_consensus.cc:1275] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Refusing update from remote peer a5a518325a4e4daa8ad361de80bf53dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:23.796190 26205 consensus_queue.cc:1048] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:23.804503 26139 raft_consensus.cc:1275] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Refusing update from remote peer a5a518325a4e4daa8ad361de80bf53dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:23.806142 26205 consensus_queue.cc:1048] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:23.817670 26065 raft_consensus.cc:1275] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Refusing update from remote peer a5a518325a4e4daa8ad361de80bf53dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:23.821651 26240 consensus_queue.cc:1048] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:24.174562 26139 raft_consensus.cc:1275] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Refusing update from remote peer a5a518325a4e4daa8ad361de80bf53dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:24.176007 26240 consensus_queue.cc:1048] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:24.182559 26230 raft_consensus.cc:993] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc: : Instructing follower ca043e13cd9842c28ab38bc0d6de7372 to start an election
I20260709 13:02:24.183158 26205 raft_consensus.cc:1081] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Signalling peer ca043e13cd9842c28ab38bc0d6de7372 to start an election
I20260709 13:02:24.188024 26065 raft_consensus.cc:1275] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Refusing update from remote peer a5a518325a4e4daa8ad361de80bf53dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:24.189071 26139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b851918cdd314632a40e94a964142729"
dest_uuid: "ca043e13cd9842c28ab38bc0d6de7372"
 from {username='slave'} at 127.0.0.1:38994
I20260709 13:02:24.189832 26139 raft_consensus.cc:493] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:24.190361 26139 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.190299 26230 consensus_queue.cc:1048] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:24.201118 26139 raft_consensus.cc:515] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:24.205276 25990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "ca043e13cd9842c28ab38bc0d6de7372" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a5a518325a4e4daa8ad361de80bf53dc"
I20260709 13:02:24.205955 26139 leader_election.cc:290] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [CANDIDATE]: Term 3 election: Requested vote from peers a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65:33909), c564eb4b67bc41549d81bddc552f05a5 (127.12.30.66:33129)
I20260709 13:02:24.206126 25990 raft_consensus.cc:3055] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:24.206331 26065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b851918cdd314632a40e94a964142729" candidate_uuid: "ca043e13cd9842c28ab38bc0d6de7372" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c564eb4b67bc41549d81bddc552f05a5"
I20260709 13:02:24.206652 25990 raft_consensus.cc:740] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a5a518325a4e4daa8ad361de80bf53dc, State: Running, Role: LEADER
I20260709 13:02:24.207273 26065 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.207739 25990 consensus_queue.cc:260] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:24.210026 25990 raft_consensus.cc:3060] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:24.217509 26065 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca043e13cd9842c28ab38bc0d6de7372 in term 3.
I20260709 13:02:24.219512 26102 leader_election.cc:304] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c564eb4b67bc41549d81bddc552f05a5, ca043e13cd9842c28ab38bc0d6de7372; no voters: 
I20260709 13:02:24.219749 25990 raft_consensus.cc:2468] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca043e13cd9842c28ab38bc0d6de7372 in term 3.
I20260709 13:02:24.220894 26206 raft_consensus.cc:2804] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:24.223158 26206 raft_consensus.cc:697] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 3 LEADER]: Becoming Leader. State: Replica: ca043e13cd9842c28ab38bc0d6de7372, State: Running, Role: LEADER
I20260709 13:02:24.224040 26206 consensus_queue.cc:237] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } }
I20260709 13:02:24.232538 25885 catalog_manager.cc:5697] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 reported cstate change: term changed from 2 to 3, leader changed from a5a518325a4e4daa8ad361de80bf53dc (127.12.30.65) to ca043e13cd9842c28ab38bc0d6de7372 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "ca043e13cd9842c28ab38bc0d6de7372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5a518325a4e4daa8ad361de80bf53dc" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c564eb4b67bc41549d81bddc552f05a5" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 33129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca043e13cd9842c28ab38bc0d6de7372" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 32829 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:24.418582 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
I20260709 13:02:24.446656 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:24.447404 12409 tablet_replica.cc:333] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc: stopping tablet replica
I20260709 13:02:24.448129 12409 raft_consensus.cc:2243] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:24.449007 12409 raft_consensus.cc:2272] T afa679c262414b07a5f95a7f50407b99 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.451764 12409 tablet_replica.cc:333] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc: stopping tablet replica
I20260709 13:02:24.453049 12409 raft_consensus.cc:2243] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.453577 12409 raft_consensus.cc:2272] T b851918cdd314632a40e94a964142729 P a5a518325a4e4daa8ad361de80bf53dc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.457178 12409 tablet_replica.cc:333] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc: stopping tablet replica
I20260709 13:02:24.457795 12409 raft_consensus.cc:2243] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.458248 12409 raft_consensus.cc:2272] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P a5a518325a4e4daa8ad361de80bf53dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.460837 12409 tablet_replica.cc:333] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc: stopping tablet replica
I20260709 13:02:24.461313 12409 raft_consensus.cc:2243] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:24.462167 12409 raft_consensus.cc:2272] T 14e20aecd5f0432b9129a6c0f42c28d1 P a5a518325a4e4daa8ad361de80bf53dc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.492298 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
I20260709 13:02:24.509402 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
I20260709 13:02:24.535969 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:24.536767 12409 tablet_replica.cc:333] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5: stopping tablet replica
I20260709 13:02:24.537431 12409 raft_consensus.cc:2243] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.537983 12409 raft_consensus.cc:2272] T afa679c262414b07a5f95a7f50407b99 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.540879 12409 tablet_replica.cc:333] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5: stopping tablet replica
I20260709 13:02:24.541590 12409 raft_consensus.cc:2243] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.542083 12409 raft_consensus.cc:2272] T b851918cdd314632a40e94a964142729 P c564eb4b67bc41549d81bddc552f05a5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.545102 12409 tablet_replica.cc:333] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5: stopping tablet replica
I20260709 13:02:24.545706 12409 raft_consensus.cc:2243] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:24.546572 12409 raft_consensus.cc:2272] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P c564eb4b67bc41549d81bddc552f05a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.548743 12409 tablet_replica.cc:333] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5: stopping tablet replica
I20260709 13:02:24.549285 12409 raft_consensus.cc:2243] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.549751 12409 raft_consensus.cc:2272] T 14e20aecd5f0432b9129a6c0f42c28d1 P c564eb4b67bc41549d81bddc552f05a5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.575376 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
I20260709 13:02:24.589499 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
I20260709 13:02:24.613982 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:24.615160 12409 tablet_replica.cc:333] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372: stopping tablet replica
I20260709 13:02:24.616047 12409 raft_consensus.cc:2243] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.616667 12409 raft_consensus.cc:2272] T afa679c262414b07a5f95a7f50407b99 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.620462 12409 tablet_replica.cc:333] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: stopping tablet replica
I20260709 13:02:24.621321 12409 raft_consensus.cc:2243] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:24.622206 12409 pending_rounds.cc:70] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: Trying to abort 1 pending ops.
I20260709 13:02:24.622453 12409 pending_rounds.cc:77] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305634382751784960 op_type: NO_OP noop_request { }
I20260709 13:02:24.622808 12409 raft_consensus.cc:2889] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:02:24.623128 12409 raft_consensus.cc:2272] T b851918cdd314632a40e94a964142729 P ca043e13cd9842c28ab38bc0d6de7372 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.625926 12409 tablet_replica.cc:333] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372: stopping tablet replica
I20260709 13:02:24.626555 12409 raft_consensus.cc:2243] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.627094 12409 raft_consensus.cc:2272] T 73f8dfc5cbda4f05bb7709d8559b8fe5 P ca043e13cd9842c28ab38bc0d6de7372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.629426 12409 tablet_replica.cc:333] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372: stopping tablet replica
I20260709 13:02:24.629952 12409 raft_consensus.cc:2243] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:24.630427 12409 raft_consensus.cc:2272] T 14e20aecd5f0432b9129a6c0f42c28d1 P ca043e13cd9842c28ab38bc0d6de7372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.656173 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
I20260709 13:02:24.672976 12409 master.cc:562] Master@127.12.30.126:44315 shutting down...
I20260709 13:02:24.694298 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:24.695086 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:24.695468 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0e98bcac70104497982156ca714db1a3: stopping tablet replica
I20260709 13:02:24.717818 12409 master.cc:584] Master@127.12.30.126:44315 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3439 ms)
[----------] 6 tests from LeaderRebalancerTest (417589 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:02:24.757934 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:42727
I20260709 13:02:24.759469 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:24.765703 26246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:24.766494 26247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:24.767810 26249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:24.769263 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:24.770412 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:24.770677 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:24.770882 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602144770862 us; error 0 us; skew 500 ppm
I20260709 13:02:24.771644 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:24.774417 12409 webserver.cc:533] Webserver started at http://127.12.30.126:33831/ using document root <none> and password file <none>
I20260709 13:02:24.775045 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:24.775272 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:24.775580 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:24.776888 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "1a749a25ad9c41bda919d20db95b9c62"
format_stamp: "Formatted at 2026-07-09 13:02:24 on dist-test-slave-mlt7"
I20260709 13:02:24.782043 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.002s
I20260709 13:02:24.786166 26254 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:24.787153 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:02:24.787498 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "1a749a25ad9c41bda919d20db95b9c62"
format_stamp: "Formatted at 2026-07-09 13:02:24 on dist-test-slave-mlt7"
I20260709 13:02:24.787796 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:24.800449 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:24.801542 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:24.843066 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:42727
I20260709 13:02:24.843225 26305 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:42727 every 8 connection(s)
I20260709 13:02:24.847455 26306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:24.862004 26306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62: Bootstrap starting.
I20260709 13:02:24.867666 26306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:24.873034 26306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62: No bootstrap required, opened a new log
I20260709 13:02:24.875633 26306 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a749a25ad9c41bda919d20db95b9c62" member_type: VOTER }
I20260709 13:02:24.876142 26306 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:24.876405 26306 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a749a25ad9c41bda919d20db95b9c62, State: Initialized, Role: FOLLOWER
I20260709 13:02:24.877189 26306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a749a25ad9c41bda919d20db95b9c62" member_type: VOTER }
I20260709 13:02:24.878000 26306 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:02:24.878310 26306 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:02:24.878638 26306 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:24.885021 26306 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a749a25ad9c41bda919d20db95b9c62" member_type: VOTER }
I20260709 13:02:24.885684 26306 leader_election.cc:304] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a749a25ad9c41bda919d20db95b9c62; no voters: 
I20260709 13:02:24.887068 26306 leader_election.cc:290] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:02:24.887554 26309 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:24.889278 26309 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 1 LEADER]: Becoming Leader. State: Replica: 1a749a25ad9c41bda919d20db95b9c62, State: Running, Role: LEADER
I20260709 13:02:24.890082 26309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a749a25ad9c41bda919d20db95b9c62" member_type: VOTER }
I20260709 13:02:24.890610 26306 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:02:24.893537 26310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a749a25ad9c41bda919d20db95b9c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a749a25ad9c41bda919d20db95b9c62" member_type: VOTER } }
I20260709 13:02:24.893857 26311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a749a25ad9c41bda919d20db95b9c62. Latest consensus state: current_term: 1 leader_uuid: "1a749a25ad9c41bda919d20db95b9c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a749a25ad9c41bda919d20db95b9c62" member_type: VOTER } }
I20260709 13:02:24.894595 26311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:24.894380 26310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:24.903571 26315 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:02:24.911130 26315 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:02:24.914301 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:02:24.923648 26315 catalog_manager.cc:1383] Generated new cluster ID: af4ba73c5db54d44bca119af4ea4d1d8
I20260709 13:02:24.924067 26315 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:02:24.959465 26315 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:02:24.960935 26315 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:02:24.976495 26315 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62: Generated new TSK 0
I20260709 13:02:24.977138 26315 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:02:24.982508 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:24.989861 26327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:24.991087 26328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:24.994191 26330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:24.997223 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:24.998358 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:24.998584 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:24.998761 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602144998743 us; error 0 us; skew 500 ppm
I20260709 13:02:24.999325 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:25.002239 12409 webserver.cc:533] Webserver started at http://127.12.30.65:33501/ using document root <none> and password file <none>
I20260709 13:02:25.002843 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:25.003064 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:25.003362 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:25.005040 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "1d387e18cd424760b5c49569a29c26a6"
format_stamp: "Formatted at 2026-07-09 13:02:24 on dist-test-slave-mlt7"
I20260709 13:02:25.010756 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:02:25.015066 26335 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.016391 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:02:25.016734 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "1d387e18cd424760b5c49569a29c26a6"
format_stamp: "Formatted at 2026-07-09 13:02:24 on dist-test-slave-mlt7"
I20260709 13:02:25.017050 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:25.046845 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:25.048525 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:25.050340 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:25.054605 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:25.055060 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.055611 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:25.055877 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.109035 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:36139
I20260709 13:02:25.109123 26397 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:36139 every 8 connection(s)
I20260709 13:02:25.115525 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:25.124581 26402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:25.125658 26403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:25.130671 26405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:25.131160 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:25.133481 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:25.133780 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:25.134017 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602145133996 us; error 0 us; skew 500 ppm
I20260709 13:02:25.134738 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:25.134953 26398 heartbeater.cc:344] Connected to a master server at 127.12.30.126:42727
I20260709 13:02:25.135968 26398 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:25.137228 26398 heartbeater.cc:507] Master 127.12.30.126:42727 requested a full tablet report, sending...
I20260709 13:02:25.139328 12409 webserver.cc:533] Webserver started at http://127.12.30.66:40615/ using document root <none> and password file <none>
I20260709 13:02:25.140130 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:25.140360 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:25.140654 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:25.140772 26271 ts_manager.cc:194] Registered new tserver with Master: 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139)
I20260709 13:02:25.142134 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "07f704db827b4614991c7f6e49ae5207"
format_stamp: "Formatted at 2026-07-09 13:02:25 on dist-test-slave-mlt7"
I20260709 13:02:25.144034 26271 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35034
I20260709 13:02:25.148283 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:02:25.152746 26410 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.153864 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:02:25.154261 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "07f704db827b4614991c7f6e49ae5207"
format_stamp: "Formatted at 2026-07-09 13:02:25 on dist-test-slave-mlt7"
I20260709 13:02:25.154596 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:25.166291 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:25.167510 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:25.169224 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:25.171821 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:25.172155 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:02:25.172430 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:25.172602 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.231067 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:34883
I20260709 13:02:25.231187 26472 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:34883 every 8 connection(s)
I20260709 13:02:25.236364 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:25.244247 26476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:25.245018 26477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:25.250663 12409 server_base.cc:1061] running on GCE node
W20260709 13:02:25.252308 26479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:25.253159 26473 heartbeater.cc:344] Connected to a master server at 127.12.30.126:42727
I20260709 13:02:25.253532 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:02:25.253674 26473 heartbeater.cc:461] Registering TS with master...
W20260709 13:02:25.253865 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:25.254201 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602145254188 us; error 0 us; skew 500 ppm
I20260709 13:02:25.254841 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:25.255052 26473 heartbeater.cc:507] Master 127.12.30.126:42727 requested a full tablet report, sending...
I20260709 13:02:25.258076 12409 webserver.cc:533] Webserver started at http://127.12.30.67:39851/ using document root <none> and password file <none>
I20260709 13:02:25.258162 26271 ts_manager.cc:194] Registered new tserver with Master: 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:25.259037 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:25.259388 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:25.259725 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:25.260190 26271 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35046
I20260709 13:02:25.261157 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "efa1b588222545e18dff850a02f8ec37"
format_stamp: "Formatted at 2026-07-09 13:02:25 on dist-test-slave-mlt7"
I20260709 13:02:25.266902 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:02:25.270835 26484 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.272099 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:02:25.272568 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "efa1b588222545e18dff850a02f8ec37"
format_stamp: "Formatted at 2026-07-09 13:02:25 on dist-test-slave-mlt7"
I20260709 13:02:25.272943 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:25.311246 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:25.312440 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:25.313858 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:25.316457 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:25.316700 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.316989 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:25.317144 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:25.362627 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:44275
I20260709 13:02:25.362751 26546 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:44275 every 8 connection(s)
I20260709 13:02:25.377848 26547 heartbeater.cc:344] Connected to a master server at 127.12.30.126:42727
I20260709 13:02:25.378428 26547 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:25.379243 26547 heartbeater.cc:507] Master 127.12.30.126:42727 requested a full tablet report, sending...
I20260709 13:02:25.381357 26271 ts_manager.cc:194] Registered new tserver with Master: efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:25.382068 12409 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016218584s
I20260709 13:02:25.382881 26271 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35048
I20260709 13:02:25.409564 26271 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35050:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:25.412259 26271 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:25.478324 26363 tablet_service.cc:1511] Processing CreateTablet for tablet 903a21f3964a4311a2109b046548c563 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:25.479166 26362 tablet_service.cc:1511] Processing CreateTablet for tablet 6201671c09434607ae6a2fc6e005421e (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:25.479779 26363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903a21f3964a4311a2109b046548c563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.480629 26362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6201671c09434607ae6a2fc6e005421e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.485404 26361 tablet_service.cc:1511] Processing CreateTablet for tablet 2d70bd89a6f7403c9fa79bf2ed68a6c3 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:25.487080 26361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d70bd89a6f7403c9fa79bf2ed68a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.488402 26512 tablet_service.cc:1511] Processing CreateTablet for tablet 903a21f3964a4311a2109b046548c563 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:25.489897 26512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903a21f3964a4311a2109b046548c563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.495471 26360 tablet_service.cc:1511] Processing CreateTablet for tablet 09955d4a1d6843a6a2c528d4b4263374 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:25.497143 26360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09955d4a1d6843a6a2c528d4b4263374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.500324 26511 tablet_service.cc:1511] Processing CreateTablet for tablet 6201671c09434607ae6a2fc6e005421e (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:25.502344 26511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6201671c09434607ae6a2fc6e005421e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.504648 26359 tablet_service.cc:1511] Processing CreateTablet for tablet ed59cec85356493cae6a495fbfb2d404 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:25.506167 26359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed59cec85356493cae6a495fbfb2d404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.509791 26510 tablet_service.cc:1511] Processing CreateTablet for tablet 2d70bd89a6f7403c9fa79bf2ed68a6c3 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:25.510723 26358 tablet_service.cc:1511] Processing CreateTablet for tablet 86e2049c522743d984904f37d2bff5cb (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:25.511931 26510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d70bd89a6f7403c9fa79bf2ed68a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.512233 26358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e2049c522743d984904f37d2bff5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.516079 26509 tablet_service.cc:1511] Processing CreateTablet for tablet 09955d4a1d6843a6a2c528d4b4263374 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:25.517733 26509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09955d4a1d6843a6a2c528d4b4263374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.520103 26508 tablet_service.cc:1511] Processing CreateTablet for tablet ed59cec85356493cae6a495fbfb2d404 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:25.521580 26508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed59cec85356493cae6a495fbfb2d404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.533586 26507 tablet_service.cc:1511] Processing CreateTablet for tablet 86e2049c522743d984904f37d2bff5cb (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:25.535177 26507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e2049c522743d984904f37d2bff5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.534955 26357 tablet_service.cc:1511] Processing CreateTablet for tablet d349544786e74ca88edd37fc4a36bd90 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:25.536450 26357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d349544786e74ca88edd37fc4a36bd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.540433 26437 tablet_service.cc:1511] Processing CreateTablet for tablet 6201671c09434607ae6a2fc6e005421e (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:25.540759 26436 tablet_service.cc:1511] Processing CreateTablet for tablet 2d70bd89a6f7403c9fa79bf2ed68a6c3 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:25.541985 26437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6201671c09434607ae6a2fc6e005421e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.541915 26438 tablet_service.cc:1511] Processing CreateTablet for tablet 903a21f3964a4311a2109b046548c563 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:25.542544 26435 tablet_service.cc:1511] Processing CreateTablet for tablet 09955d4a1d6843a6a2c528d4b4263374 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:25.544909 26438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903a21f3964a4311a2109b046548c563. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.544987 26434 tablet_service.cc:1511] Processing CreateTablet for tablet ed59cec85356493cae6a495fbfb2d404 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:25.546473 26434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed59cec85356493cae6a495fbfb2d404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.549021 26433 tablet_service.cc:1511] Processing CreateTablet for tablet 86e2049c522743d984904f37d2bff5cb (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:25.550446 26433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e2049c522743d984904f37d2bff5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.552490 26435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09955d4a1d6843a6a2c528d4b4263374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.556514 26436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d70bd89a6f7403c9fa79bf2ed68a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.572839 26432 tablet_service.cc:1511] Processing CreateTablet for tablet d349544786e74ca88edd37fc4a36bd90 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:25.574544 26432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d349544786e74ca88edd37fc4a36bd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.582515 26506 tablet_service.cc:1511] Processing CreateTablet for tablet d349544786e74ca88edd37fc4a36bd90 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:25.584406 26506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d349544786e74ca88edd37fc4a36bd90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.584713 26360 tablet_service.cc:1511] Processing CreateTablet for tablet bb4a8280554641faab8f7a597754ebf1 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:25.586088 26360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4a8280554641faab8f7a597754ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.599426 26431 tablet_service.cc:1511] Processing CreateTablet for tablet bb4a8280554641faab8f7a597754ebf1 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:25.600999 26431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4a8280554641faab8f7a597754ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.600924 26505 tablet_service.cc:1511] Processing CreateTablet for tablet bb4a8280554641faab8f7a597754ebf1 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:25.602664 26505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4a8280554641faab8f7a597754ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.609508 26359 tablet_service.cc:1511] Processing CreateTablet for tablet 2c2f11d94f7f45f8982152cdfa656466 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:25.610859 26359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2f11d94f7f45f8982152cdfa656466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.615303 26562 tablet_bootstrap.cc:492] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:25.632539 26435 tablet_service.cc:1511] Processing CreateTablet for tablet 2c2f11d94f7f45f8982152cdfa656466 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:25.634153 26435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2f11d94f7f45f8982152cdfa656466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.634047 26504 tablet_service.cc:1511] Processing CreateTablet for tablet 2c2f11d94f7f45f8982152cdfa656466 (DEFAULT_TABLE table=test-workload [id=4d0e10d11b4f42d5b2eca143545c97c4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:25.636279 26504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2f11d94f7f45f8982152cdfa656466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:25.653080 26563 tablet_bootstrap.cc:492] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:25.661808 26563 tablet_bootstrap.cc:654] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.693372 26562 tablet_bootstrap.cc:654] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.697377 26564 tablet_bootstrap.cc:492] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:25.702042 26563 tablet_bootstrap.cc:492] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:25.702625 26563 ts_tablet_manager.cc:1403] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.022s
I20260709 13:02:25.706595 26562 tablet_bootstrap.cc:492] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:25.706670 26563 raft_consensus.cc:359] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.707337 26562 ts_tablet_manager.cc:1403] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.004s
I20260709 13:02:25.707664 26563 raft_consensus.cc:385] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.707831 26564 tablet_bootstrap.cc:654] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.708137 26563 raft_consensus.cc:740] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.709131 26563 consensus_queue.cc:260] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.710793 26562 raft_consensus.cc:359] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.711428 26473 heartbeater.cc:499] Master 127.12.30.126:42727 was elected leader, sending a full tablet report...
I20260709 13:02:25.711685 26562 raft_consensus.cc:385] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.711889 26563 ts_tablet_manager.cc:1434] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:02:25.712065 26562 raft_consensus.cc:740] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.713107 26563 tablet_bootstrap.cc:492] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:25.712950 26562 consensus_queue.cc:260] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.719499 26563 tablet_bootstrap.cc:654] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.724115 26564 tablet_bootstrap.cc:492] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:25.724988 26564 ts_tablet_manager.cc:1403] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.004s
I20260709 13:02:25.729840 26562 ts_tablet_manager.cc:1434] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.014s
I20260709 13:02:25.730269 26398 heartbeater.cc:499] Master 127.12.30.126:42727 was elected leader, sending a full tablet report...
I20260709 13:02:25.731093 26562 tablet_bootstrap.cc:492] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:25.738240 26562 tablet_bootstrap.cc:654] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.746515 26563 tablet_bootstrap.cc:492] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:25.747077 26563 ts_tablet_manager.cc:1403] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.012s
I20260709 13:02:25.729905 26564 raft_consensus.cc:359] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.749673 26564 raft_consensus.cc:385] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.750335 26564 raft_consensus.cc:740] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.750358 26563 raft_consensus.cc:359] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.751420 26563 raft_consensus.cc:385] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.751773 26563 raft_consensus.cc:740] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.753069 26563 consensus_queue.cc:260] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.758962 26564 consensus_queue.cc:260] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.762497 26563 ts_tablet_manager.cc:1434] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:02:25.763731 26563 tablet_bootstrap.cc:492] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:25.764837 26562 tablet_bootstrap.cc:492] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:25.765444 26562 ts_tablet_manager.cc:1403] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.020s
I20260709 13:02:25.769107 26547 heartbeater.cc:499] Master 127.12.30.126:42727 was elected leader, sending a full tablet report...
I20260709 13:02:25.769342 26563 tablet_bootstrap.cc:654] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.769637 26564 ts_tablet_manager.cc:1434] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.044s	user 0.011s	sys 0.000s
I20260709 13:02:25.770727 26564 tablet_bootstrap.cc:492] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:25.775086 26562 raft_consensus.cc:359] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.776396 26562 raft_consensus.cc:385] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.776749 26562 raft_consensus.cc:740] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.780419 26564 tablet_bootstrap.cc:654] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.781912 26562 consensus_queue.cc:260] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.783164 26563 tablet_bootstrap.cc:492] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:25.783739 26563 ts_tablet_manager.cc:1403] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:02:25.784551 26562 ts_tablet_manager.cc:1434] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 13:02:25.785943 26562 tablet_bootstrap.cc:492] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:25.787082 26563 raft_consensus.cc:359] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.788723 26563 raft_consensus.cc:385] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.789062 26563 raft_consensus.cc:740] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.790274 26563 consensus_queue.cc:260] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.795711 26562 tablet_bootstrap.cc:654] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.796200 26572 raft_consensus.cc:493] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.796818 26572 raft_consensus.cc:515] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.798725 26563 ts_tablet_manager.cc:1434] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:02:25.800508 26572 leader_election.cc:290] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:25.801344 26563 tablet_bootstrap.cc:492] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:25.802739 26564 tablet_bootstrap.cc:492] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:25.805655 26564 ts_tablet_manager.cc:1403] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.000s
I20260709 13:02:25.810861 26564 raft_consensus.cc:359] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.811656 26564 raft_consensus.cc:385] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.816354 26564 raft_consensus.cc:740] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.818499 26564 consensus_queue.cc:260] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.820194 26563 tablet_bootstrap.cc:654] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.825214 26564 ts_tablet_manager.cc:1434] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.000s
I20260709 13:02:25.826484 26564 tablet_bootstrap.cc:492] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:25.827126 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
W20260709 13:02:25.829073 26485 leader_election.cc:343] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:25.826608 26562 tablet_bootstrap.cc:492] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:25.829937 26562 ts_tablet_manager.cc:1403] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.019s
I20260709 13:02:25.833541 26562 raft_consensus.cc:359] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:25.834336 26562 raft_consensus.cc:385] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.834656 26562 raft_consensus.cc:740] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.836059 26564 tablet_bootstrap.cc:654] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.835602 26562 consensus_queue.cc:260] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:25.838594 26562 ts_tablet_manager.cc:1434] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:25.839821 26562 tablet_bootstrap.cc:492] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:25.846973 26562 tablet_bootstrap.cc:654] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.855405 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:25.856480 26448 raft_consensus.cc:2468] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:25.857149 26563 tablet_bootstrap.cc:492] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:25.858053 26563 ts_tablet_manager.cc:1403] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.014s
I20260709 13:02:25.858641 26486 leader_election.cc:304] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:25.859957 26572 raft_consensus.cc:2804] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:25.860602 26572 raft_consensus.cc:493] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.860951 26572 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:25.862622 26564 tablet_bootstrap.cc:492] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:25.863054 26563 raft_consensus.cc:359] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.863945 26563 raft_consensus.cc:385] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.864259 26563 raft_consensus.cc:740] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.863513 26564 ts_tablet_manager.cc:1403] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.012s
I20260709 13:02:25.865151 26563 consensus_queue.cc:260] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.867554 26563 ts_tablet_manager.cc:1434] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:02:25.868008 26564 raft_consensus.cc:359] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.868937 26564 raft_consensus.cc:385] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.869390 26564 raft_consensus.cc:740] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.868937 26563 tablet_bootstrap.cc:492] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:25.870317 26564 consensus_queue.cc:260] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.873399 26572 raft_consensus.cc:515] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.877918 26563 tablet_bootstrap.cc:654] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.880131 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
W20260709 13:02:25.882273 26485 leader_election.cc:343] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:25.877954 26562 tablet_bootstrap.cc:492] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:25.884109 26562 ts_tablet_manager.cc:1403] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.004s
I20260709 13:02:25.888374 26564 ts_tablet_manager.cc:1434] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.013s
I20260709 13:02:25.888500 26572 leader_election.cc:290] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:25.890153 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:25.890856 26448 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:25.892273 26562 raft_consensus.cc:359] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.893489 26562 raft_consensus.cc:385] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.894016 26562 raft_consensus.cc:740] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.895242 26562 consensus_queue.cc:260] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.900713 26448 raft_consensus.cc:2468] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:25.902341 26486 leader_election.cc:304] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:25.903781 26564 tablet_bootstrap.cc:492] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:25.905936 26580 raft_consensus.cc:2804] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:25.908305 26580 raft_consensus.cc:697] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
W20260709 13:02:25.909794 26319 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:25.909850 26580 consensus_queue.cc:237] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.910974 26562 ts_tablet_manager.cc:1434] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:02:25.912350 26562 tablet_bootstrap.cc:492] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:25.912747 26564 tablet_bootstrap.cc:654] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.920065 26568 raft_consensus.cc:493] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.921047 26568 raft_consensus.cc:515] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:25.922214 26563 tablet_bootstrap.cc:492] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:25.923029 26563 ts_tablet_manager.cc:1403] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.054s	user 0.020s	sys 0.016s
I20260709 13:02:25.924342 26562 tablet_bootstrap.cc:654] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.926438 26564 tablet_bootstrap.cc:492] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:25.927110 26564 ts_tablet_manager.cc:1403] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.007s
I20260709 13:02:25.927091 26563 raft_consensus.cc:359] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.928225 26563 raft_consensus.cc:385] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.928575 26563 raft_consensus.cc:740] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.929818 26563 consensus_queue.cc:260] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.934990 26564 raft_consensus.cc:359] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.935876 26564 raft_consensus.cc:385] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.935410 26271 catalog_manager.cc:5697] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:25.936142 26568 leader_election.cc:290] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:25.936206 26564 raft_consensus.cc:740] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.937386 26564 consensus_queue.cc:260] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.939839 26564 ts_tablet_manager.cc:1434] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:02:25.942356 26564 tablet_bootstrap.cc:492] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:25.944198 26580 raft_consensus.cc:493] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.944585 26563 ts_tablet_manager.cc:1434] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.004s
I20260709 13:02:25.944880 26580 raft_consensus.cc:515] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.948458 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:25.949373 26563 tablet_bootstrap.cc:492] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:25.949651 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
W20260709 13:02:25.949977 26486 leader_election.cc:343] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:25.951053 26485 leader_election.cc:343] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:25.950781 26580 leader_election.cc:290] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:25.951573 26485 leader_election.cc:304] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:25.952497 26580 raft_consensus.cc:2749] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:25.959051 26562 tablet_bootstrap.cc:492] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:25.959817 26562 ts_tablet_manager.cc:1403] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.008s
I20260709 13:02:25.964365 26562 raft_consensus.cc:359] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.965094 26564 tablet_bootstrap.cc:654] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.965420 26562 raft_consensus.cc:385] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.965819 26562 raft_consensus.cc:740] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.966925 26562 consensus_queue.cc:260] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.965898 26563 tablet_bootstrap.cc:654] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.970482 26562 ts_tablet_manager.cc:1434] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:02:25.972055 26562 tablet_bootstrap.cc:492] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:25.981612 26562 tablet_bootstrap.cc:654] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:25.983858 26563 tablet_bootstrap.cc:492] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:25.984488 26563 ts_tablet_manager.cc:1403] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.005s
I20260709 13:02:25.984819 26568 raft_consensus.cc:493] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.985368 26568 raft_consensus.cc:515] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:25.990648 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a21f3964a4311a2109b046548c563" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:25.993916 26568 leader_election.cc:290] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:25.994211 26563 raft_consensus.cc:359] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:25.995008 26563 raft_consensus.cc:385] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:25.995376 26563 raft_consensus.cc:740] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:25.996347 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:25.997925 26563 consensus_queue.cc:260] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:26.000633 26564 tablet_bootstrap.cc:492] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:26.001538 26564 ts_tablet_manager.cc:1403] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.059s	user 0.021s	sys 0.022s
I20260709 13:02:26.003628 26563 ts_tablet_manager.cc:1434] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 13:02:26.005221 26567 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.005796 26567 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.006395 26564 raft_consensus.cc:359] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.007850 26564 raft_consensus.cc:385] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.008451 26564 raft_consensus.cc:740] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.009531 26568 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.009577 26564 consensus_queue.cc:260] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.010223 26568 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.013434 26568 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.016707 26564 ts_tablet_manager.cc:1434] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:02:26.017793 26567 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.018487 26564 tablet_bootstrap.cc:492] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:26.026396 26564 tablet_bootstrap.cc:654] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.026402 26563 tablet_bootstrap.cc:492] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
W20260709 13:02:25.997932 26338 leader_election.cc:343] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:26.035280 26338 leader_election.cc:343] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.037889 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.038311 26562 tablet_bootstrap.cc:492] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:26.039201 26562 ts_tablet_manager.cc:1403] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.067s	user 0.026s	sys 0.017s
I20260709 13:02:26.048355 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a21f3964a4311a2109b046548c563" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.049360 26448 raft_consensus.cc:2468] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:26.051668 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.052009 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.052980 26447 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:26.054085 26337 leader_election.cc:304] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.055501 26568 raft_consensus.cc:2804] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.055878 26568 raft_consensus.cc:493] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.056236 26568 raft_consensus.cc:3060] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.057111 26567 raft_consensus.cc:493] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.057683 26567 raft_consensus.cc:515] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.059903 26567 leader_election.cc:290] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.059408 26562 raft_consensus.cc:359] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.060689 26562 raft_consensus.cc:385] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.061122 26562 raft_consensus.cc:740] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.061650 26563 tablet_bootstrap.cc:654] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:26.040376 26338 leader_election.cc:343] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.067858 26521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.067711 26562 consensus_queue.cc:260] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.069262 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.070984 26568 raft_consensus.cc:515] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
W20260709 13:02:26.071201 26413 leader_election.cc:343] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:26.072376 26413 leader_election.cc:343] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.073114 26568 leader_election.cc:290] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:26.073956 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a21f3964a4311a2109b046548c563" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.075455 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.076574 26373 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:26.077997 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.078897 26564 tablet_bootstrap.cc:492] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:26.079632 26564 ts_tablet_manager.cc:1403] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.061s	user 0.020s	sys 0.024s
I20260709 13:02:26.080278 26563 tablet_bootstrap.cc:492] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:26.080994 26563 ts_tablet_manager.cc:1403] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.008s
W20260709 13:02:26.080302 26338 leader_election.cc:343] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.082433 26411 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.083534 26567 raft_consensus.cc:2804] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.083973 26567 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.084475 26567 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.083742 26564 raft_consensus.cc:359] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:26.084987 26564 raft_consensus.cc:385] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.085450 26564 raft_consensus.cc:740] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.084714 26563 raft_consensus.cc:359] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.068354 26337 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.086622 26568 raft_consensus.cc:2804] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:02:26.086931 26337 leader_election.cc:343] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.087036 26568 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.087530 26337 leader_election.cc:304] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6; no voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.087960 26562 ts_tablet_manager.cc:1434] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.048s	user 0.018s	sys 0.003s
I20260709 13:02:26.087724 26568 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.089313 26562 tablet_bootstrap.cc:492] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
W20260709 13:02:26.089885 26411 leader_election.cc:343] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.090397 26411 leader_election.cc:304] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207; no voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.091208 26563 raft_consensus.cc:385] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.092204 26594 raft_consensus.cc:2749] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.092480 26563 raft_consensus.cc:740] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.093317 26563 consensus_queue.cc:260] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.094017 26567 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.096848 26567 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.097412 26567 raft_consensus.cc:2749] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.097558 26564 consensus_queue.cc:260] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:26.099021 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.100917 26580 raft_consensus.cc:493] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.101471 26580 raft_consensus.cc:515] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.102705 26564 ts_tablet_manager.cc:1434] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:02:26.103639 26580 leader_election.cc:290] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:26.105713 26564 tablet_bootstrap.cc:492] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:26.108716 26568 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.109301 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:26.110246 26562 tablet_bootstrap.cc:654] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.110634 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a21f3964a4311a2109b046548c563" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.112685 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.113067 26448 raft_consensus.cc:3060] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.113464 26447 raft_consensus.cc:2468] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:26.112732 26372 raft_consensus.cc:2393] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07f704db827b4614991c7f6e49ae5207 in current term 1: Already voted for candidate 1d387e18cd424760b5c49569a29c26a6 in this term.
I20260709 13:02:26.114696 26568 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.115425 26446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.115844 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.116381 26446 raft_consensus.cc:2393] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1d387e18cd424760b5c49569a29c26a6 in current term 1: Already voted for candidate 07f704db827b4614991c7f6e49ae5207 in this term.
W20260709 13:02:26.117355 26338 leader_election.cc:343] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.118244 26337 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6; no voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.111516 26563 ts_tablet_manager.cc:1434] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.002s
I20260709 13:02:26.119741 26568 raft_consensus.cc:2749] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.120827 26563 tablet_bootstrap.cc:492] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:26.124392 26564 tablet_bootstrap.cc:654] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:26.124852 26413 leader_election.cc:343] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.125399 26413 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207; no voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.127095 26448 raft_consensus.cc:2468] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:26.131753 26337 leader_election.cc:304] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.133668 26568 raft_consensus.cc:2804] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:26.134444 26568 raft_consensus.cc:697] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:26.135682 26568 consensus_queue.cc:237] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:26.137012 26596 raft_consensus.cc:2749] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.127517 26563 tablet_bootstrap.cc:654] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.128381 26486 leader_election.cc:304] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:26.138726 26580 raft_consensus.cc:2804] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.139125 26580 raft_consensus.cc:493] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.139467 26580 raft_consensus.cc:3060] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.127027 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.144797 26372 raft_consensus.cc:2468] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:26.145707 26562 tablet_bootstrap.cc:492] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:26.146550 26562 ts_tablet_manager.cc:1403] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.001s
I20260709 13:02:26.151657 26562 raft_consensus.cc:359] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.152774 26562 raft_consensus.cc:385] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.153182 26562 raft_consensus.cc:740] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.154345 26562 consensus_queue.cc:260] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.162539 26562 ts_tablet_manager.cc:1434] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:02:26.162693 26270 catalog_manager.cc:5697] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:26.164311 26562 tablet_bootstrap.cc:492] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:26.165176 26596 raft_consensus.cc:493] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.165740 26596 raft_consensus.cc:515] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.167960 26572 raft_consensus.cc:493] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.168641 26572 raft_consensus.cc:515] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.170821 26572 leader_election.cc:290] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:26.173372 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
W20260709 13:02:26.175073 26486 leader_election.cc:343] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.175772 26562 tablet_bootstrap.cc:654] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.176138 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.177258 26372 raft_consensus.cc:2468] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:26.178460 26485 leader_election.cc:304] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:26.180153 26572 raft_consensus.cc:2804] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.180676 26572 raft_consensus.cc:493] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.182698 26572 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.185063 26580 raft_consensus.cc:515] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.189013 26596 leader_election.cc:290] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.189651 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.191361 26522 raft_consensus.cc:2468] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:26.191449 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:26.192796 26373 raft_consensus.cc:3060] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.193485 26413 leader_election.cc:304] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:26.194845 26596 raft_consensus.cc:2804] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.195225 26596 raft_consensus.cc:493] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.196383 26568 raft_consensus.cc:493] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.196950 26568 raft_consensus.cc:515] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.199824 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.200558 26448 raft_consensus.cc:3060] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.200618 26568 leader_election.cc:290] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.195545 26596 raft_consensus.cc:3060] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.208267 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.210225 26372 raft_consensus.cc:2468] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:26.212304 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.213701 26568 raft_consensus.cc:493] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.214314 26568 raft_consensus.cc:515] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.207089 26580 leader_election.cc:290] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:26.220422 26568 leader_election.cc:290] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.222157 26596 raft_consensus.cc:515] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
W20260709 13:02:26.256042 26338 leader_election.cc:343] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.260098 26446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.262566 26446 raft_consensus.cc:2468] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:26.265492 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.271724 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.273638 26522 raft_consensus.cc:2468] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:26.271724 26337 leader_election.cc:304] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.275887 26338 leader_election.cc:304] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:26.277129 26562 tablet_bootstrap.cc:492] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:26.278407 26562 ts_tablet_manager.cc:1403] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.115s	user 0.024s	sys 0.020s
I20260709 13:02:26.281358 26564 tablet_bootstrap.cc:492] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:26.282714 26563 tablet_bootstrap.cc:492] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:26.283002 26564 ts_tablet_manager.cc:1403] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.178s	user 0.044s	sys 0.045s
I20260709 13:02:26.288998 26580 raft_consensus.cc:493] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.289247 26594 raft_consensus.cc:2804] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.290122 26594 raft_consensus.cc:493] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.291008 26594 raft_consensus.cc:3060] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.290563 26580 raft_consensus.cc:515] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } }
I20260709 13:02:26.292989 26568 raft_consensus.cc:493] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.289731 26563 ts_tablet_manager.cc:1403] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.169s	user 0.013s	sys 0.006s
I20260709 13:02:26.295997 26448 raft_consensus.cc:2468] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:26.297161 26447 raft_consensus.cc:2393] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1d387e18cd424760b5c49569a29c26a6 in current term 1: Already voted for candidate 07f704db827b4614991c7f6e49ae5207 in this term.
I20260709 13:02:26.297910 26564 raft_consensus.cc:359] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.298611 26564 raft_consensus.cc:385] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.298836 26564 raft_consensus.cc:740] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.221926 26572 raft_consensus.cc:515] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.300644 26486 leader_election.cc:304] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:26.300886 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a21f3964a4311a2109b046548c563" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.294507 26568 raft_consensus.cc:515] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.303989 26572 leader_election.cc:290] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:26.308199 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:26.310433 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.309463 26372 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.312076 26568 leader_election.cc:290] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.312235 26572 raft_consensus.cc:2804] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:26.316278 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a21f3964a4311a2109b046548c563" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.310542 26580 leader_election.cc:290] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
W20260709 13:02:26.313603 26337 leader_election.cc:343] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.314033 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.318955 26447 raft_consensus.cc:2393] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa1b588222545e18dff850a02f8ec37 in current term 1: Already voted for candidate 1d387e18cd424760b5c49569a29c26a6 in this term.
W20260709 13:02:26.320708 26486 leader_election.cc:343] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.321378 26373 raft_consensus.cc:2468] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:26.321632 26486 leader_election.cc:304] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:26.316834 26572 raft_consensus.cc:697] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:26.316311 26568 raft_consensus.cc:2804] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.323834 26563 raft_consensus.cc:359] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.326447 26568 raft_consensus.cc:493] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.327615 26563 raft_consensus.cc:385] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.327793 26568 raft_consensus.cc:3060] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.325728 26564 consensus_queue.cc:260] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.327980 26563 raft_consensus.cc:740] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.328354 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.329259 26522 raft_consensus.cc:2468] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:26.329003 26563 consensus_queue.cc:260] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.326555 26580 raft_consensus.cc:3060] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.332405 26563 ts_tablet_manager.cc:1434] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.001s
I20260709 13:02:26.334300 26563 tablet_bootstrap.cc:492] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:26.336519 26564 ts_tablet_manager.cc:1434] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.006s
I20260709 13:02:26.338543 26564 tablet_bootstrap.cc:492] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:26.339843 26568 raft_consensus.cc:515] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.340847 26562 raft_consensus.cc:359] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.341735 26562 raft_consensus.cc:385] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.341991 26562 raft_consensus.cc:740] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.342135 26338 leader_election.cc:304] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:26.342510 26568 leader_election.cc:290] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.343235 26568 raft_consensus.cc:2804] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.342948 26562 consensus_queue.cc:260] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.343331 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.343628 26568 raft_consensus.cc:493] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.344080 26568 raft_consensus.cc:3060] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.344328 26447 raft_consensus.cc:2393] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1d387e18cd424760b5c49569a29c26a6 in current term 1: Already voted for candidate 07f704db827b4614991c7f6e49ae5207 in this term.
I20260709 13:02:26.345904 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.346671 26562 ts_tablet_manager.cc:1434] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.068s	user 0.003s	sys 0.007s
I20260709 13:02:26.347069 26563 tablet_bootstrap.cc:654] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.361471 26564 tablet_bootstrap.cc:654] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.361536 26522 raft_consensus.cc:3060] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.362355 26580 raft_consensus.cc:2749] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.325079 26594 raft_consensus.cc:515] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.350580 26572 consensus_queue.cc:237] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.365743 26594 leader_election.cc:290] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.352384 26596 leader_election.cc:290] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.353569 26372 raft_consensus.cc:2468] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:26.353454 26568 raft_consensus.cc:515] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.367641 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.370100 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.354457 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:26.371230 26447 raft_consensus.cc:3060] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.371611 26373 raft_consensus.cc:2393] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07f704db827b4614991c7f6e49ae5207 in current term 1: Already voted for candidate 1d387e18cd424760b5c49569a29c26a6 in this term.
I20260709 13:02:26.370306 26568 leader_election.cc:290] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
W20260709 13:02:26.370411 26338 leader_election.cc:343] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.374449 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.375514 26520 raft_consensus.cc:3060] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.360026 26521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.380616 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.381693 26448 raft_consensus.cc:3060] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.382683 26485 leader_election.cc:304] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:26.381263 26413 leader_election.cc:304] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207; no voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.389485 26596 raft_consensus.cc:2749] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.390610 26604 raft_consensus.cc:2804] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:26.400966 26520 raft_consensus.cc:2468] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:26.402063 26447 raft_consensus.cc:2468] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:26.402632 26338 leader_election.cc:304] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:26.403656 26594 raft_consensus.cc:2804] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:26.404202 26594 raft_consensus.cc:697] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:26.404378 26337 leader_election.cc:304] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.405418 26572 consensus_queue.cc:1048] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:26.405486 26594 consensus_queue.cc:237] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.406025 26568 raft_consensus.cc:2804] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:26.407451 26568 raft_consensus.cc:697] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:26.408531 26568 consensus_queue.cc:237] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.411259 26563 tablet_bootstrap.cc:492] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:26.411806 26563 ts_tablet_manager.cc:1403] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.078s	user 0.016s	sys 0.032s
I20260709 13:02:26.416672 26562 tablet_bootstrap.cc:492] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:26.420652 26563 raft_consensus.cc:359] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.422868 26563 raft_consensus.cc:385] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.423153 26563 raft_consensus.cc:740] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.423611 26564 tablet_bootstrap.cc:492] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:26.424325 26564 ts_tablet_manager.cc:1403] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.086s	user 0.016s	sys 0.001s
I20260709 13:02:26.425406 26563 consensus_queue.cc:260] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.425984 26448 raft_consensus.cc:2468] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:26.435400 26563 ts_tablet_manager.cc:1434] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.003s
I20260709 13:02:26.436856 26562 tablet_bootstrap.cc:654] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.437832 26522 raft_consensus.cc:2468] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:26.441159 26270 catalog_manager.cc:5697] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.443485 26268 catalog_manager.cc:5697] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:26.439906 26564 raft_consensus.cc:359] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.446856 26564 raft_consensus.cc:385] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.447247 26564 raft_consensus.cc:740] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.449344 26338 leader_election.cc:304] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:26.433115 26604 raft_consensus.cc:697] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:26.452211 26594 raft_consensus.cc:2804] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:26.452817 26594 raft_consensus.cc:697] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:26.453806 26604 consensus_queue.cc:237] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.453995 26594 consensus_queue.cc:237] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.458381 26572 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.462581 26572 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.465294 26572 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:26.464507 26564 consensus_queue.cc:260] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.466533 26562 tablet_bootstrap.cc:492] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:26.467031 26562 ts_tablet_manager.cc:1403] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.008s
I20260709 13:02:26.469904 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.471418 26372 raft_consensus.cc:2393] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa1b588222545e18dff850a02f8ec37 in current term 1: Already voted for candidate 1d387e18cd424760b5c49569a29c26a6 in this term.
I20260709 13:02:26.471208 26562 raft_consensus.cc:359] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.472295 26562 raft_consensus.cc:385] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.472677 26562 raft_consensus.cc:740] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.469964 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.474023 26447 raft_consensus.cc:2393] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa1b588222545e18dff850a02f8ec37 in current term 1: Already voted for candidate 07f704db827b4614991c7f6e49ae5207 in this term.
I20260709 13:02:26.475607 26486 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:26.476679 26572 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.476558 26562 consensus_queue.cc:260] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.483558 26562 ts_tablet_manager.cc:1434] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 13:02:26.487978 26564 ts_tablet_manager.cc:1434] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 13:02:26.492580 26372 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.493043 26564 tablet_bootstrap.cc:492] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:26.502259 26572 raft_consensus.cc:2749] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.505203 26564 tablet_bootstrap.cc:654] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:26.507560 26594 raft_consensus.cc:493] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.508176 26594 raft_consensus.cc:515] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.510030 26607 consensus_queue.cc:1048] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:26.511497 26594 leader_election.cc:290] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.505890 26270 catalog_manager.cc:5697] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:26.512707 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.513327 26270 catalog_manager.cc:5697] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:26.514676 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:26.515491 26447 raft_consensus.cc:2468] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
W20260709 13:02:26.516775 26338 leader_election.cc:343] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:26.523872 26337 leader_election.cc:304] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.526072 26594 raft_consensus.cc:2804] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:26.526556 26594 raft_consensus.cc:493] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.526955 26594 raft_consensus.cc:3060] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.528918 26268 catalog_manager.cc:5697] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.531553 26564 tablet_bootstrap.cc:492] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:26.532156 26564 ts_tablet_manager.cc:1403] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.008s
I20260709 13:02:26.535145 26564 raft_consensus.cc:359] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.535802 26564 raft_consensus.cc:385] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:26.536096 26564 raft_consensus.cc:740] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:26.539712 26564 consensus_queue.cc:260] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.542075 26564 ts_tablet_manager.cc:1434] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:02:26.542984 26594 raft_consensus.cc:515] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.549072 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:26.549860 26447 raft_consensus.cc:3060] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.553030 26594 leader_election.cc:290] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.554092 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:26.554729 26522 raft_consensus.cc:3060] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.562844 26447 raft_consensus.cc:2468] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:26.564657 26522 raft_consensus.cc:2468] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:26.565032 26337 leader_election.cc:304] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:26.565976 26594 raft_consensus.cc:2804] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:26.566473 26594 raft_consensus.cc:697] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:26.567346 26594 consensus_queue.cc:237] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.575456 26268 catalog_manager.cc:5697] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:26.632436 26594 consensus_queue.cc:1048] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:26.647570 26603 consensus_queue.cc:1048] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:26.807605 26596 raft_consensus.cc:493] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.808235 26596 raft_consensus.cc:515] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:26.810250 26596 leader_election.cc:290] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:26.811564 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:26.811430 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:26.812702 26372 raft_consensus.cc:2393] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07f704db827b4614991c7f6e49ae5207 in current term 1: Already voted for candidate efa1b588222545e18dff850a02f8ec37 in this term.
I20260709 13:02:26.813932 26411 leader_election.cc:304] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207; no voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:26.814698 26596 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.821302 26596 raft_consensus.cc:2749] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:26.848688 26603 consensus_queue.cc:1048] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:26.861423 26522 raft_consensus.cc:3060] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:26.864518 26603 consensus_queue.cc:1048] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:26.867594 26572 consensus_queue.cc:1048] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:26.877550 26603 consensus_queue.cc:1048] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:26.896327 26604 consensus_queue.cc:1048] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:26.900274 26627 consensus_queue.cc:1048] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:02:26.925165 26319 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:26.980489 26604 consensus_queue.cc:1048] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:26.986164 26604 consensus_queue.cc:1048] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:27.001557 26594 consensus_queue.cc:1048] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:27.011765 26603 consensus_queue.cc:1048] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:27.029260 26594 consensus_queue.cc:1048] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:27.037048 26594 consensus_queue.cc:1048] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:27.747422 26640 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.748039 26640 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:27.750119 26640 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:27.751072 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:27.751132 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:27.751751 26372 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 1.
I20260709 13:02:27.752036 26522 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 1.
I20260709 13:02:27.753449 26413 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:27.754530 26640 raft_consensus.cc:2804] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:27.754885 26640 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.755210 26640 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:27.761651 26640 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:27.763454 26640 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:27.764423 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:27.764647 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:27.765269 26522 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:27.765204 26372 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:27.772413 26522 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:27.773555 26413 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:27.773770 26372 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:27.774464 26640 raft_consensus.cc:2804] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:27.775663 26640 raft_consensus.cc:697] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:27.776542 26640 consensus_queue.cc:237] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:27.785449 26269 catalog_manager.cc:5697] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 0 to 2, leader changed from <none> to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.161280 26522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
mode: GRACEFUL
new_leader_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52158
I20260709 13:02:28.161931 26522 raft_consensus.cc:606] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Received request to transfer leadership to TS 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:28.167547 26522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
mode: GRACEFUL
new_leader_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52158
I20260709 13:02:28.168190 26522 raft_consensus.cc:606] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Received request to transfer leadership to TS 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:28.183202 26448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
mode: GRACEFUL
new_leader_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:02:28.183812 26448 raft_consensus.cc:606] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Received request to transfer leadership to TS 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:28.200559 26372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52424
I20260709 13:02:28.201125 26372 raft_consensus.cc:606] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:28.207829 26372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52424
I20260709 13:02:28.208475 26372 raft_consensus.cc:606] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:28.215286 26372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52424
I20260709 13:02:28.216322 26372 raft_consensus.cc:606] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:28.222880 26522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52158
I20260709 13:02:28.223887 26522 raft_consensus.cc:606] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:28.226884 26522 raft_consensus.cc:1275] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:02:28.228170 26640 consensus_queue.cc:1048] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:28.230057 26372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:52424
I20260709 13:02:28.230976 26372 raft_consensus.cc:606] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Received request to transfer leadership to TS efa1b588222545e18dff850a02f8ec37
I20260709 13:02:28.243157 26372 raft_consensus.cc:1275] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:02:28.244840 26640 consensus_queue.cc:1048] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:28.257412 26641 raft_consensus.cc:993] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207: : Instructing follower 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:28.257937 26640 raft_consensus.cc:1081] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Signalling peer 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:28.259804 26372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52416
I20260709 13:02:28.260468 26372 raft_consensus.cc:493] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.260784 26372 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.266258 26372 raft_consensus.cc:515] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.268033 26372 leader_election.cc:290] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 3 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:28.269235 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:28.269869 26448 raft_consensus.cc:3055] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:28.270040 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09955d4a1d6843a6a2c528d4b4263374" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:28.270252 26448 raft_consensus.cc:740] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:28.270812 26520 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.271308 26448 consensus_queue.cc:260] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.272735 26448 raft_consensus.cc:3060] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:28.277246 26520 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 3.
I20260709 13:02:28.278474 26338 leader_election.cc:304] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:28.278784 26448 raft_consensus.cc:2468] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 3.
I20260709 13:02:28.279388 26568 raft_consensus.cc:2804] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:28.279910 26568 raft_consensus.cc:697] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 3 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:28.280820 26568 consensus_queue.cc:237] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.288461 26271 catalog_manager.cc:5697] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 2 to 3, leader changed from 07f704db827b4614991c7f6e49ae5207 (127.12.30.66) to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 3 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.320799 26656 raft_consensus.cc:993] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.321353 26568 raft_consensus.cc:1081] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.323137 26448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:39012
I20260709 13:02:28.323767 26448 raft_consensus.cc:493] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.324172 26448 raft_consensus.cc:3060] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.329969 26448 raft_consensus.cc:515] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.331992 26448 leader_election.cc:290] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:28.332863 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:28.333089 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d349544786e74ca88edd37fc4a36bd90" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:28.333570 26372 raft_consensus.cc:3055] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:28.333732 26522 raft_consensus.cc:3060] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.333997 26372 raft_consensus.cc:740] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:28.334933 26372 consensus_queue.cc:260] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.336123 26372 raft_consensus.cc:3060] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.340150 26522 raft_consensus.cc:2468] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.341511 26413 leader_election.cc:304] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:28.342301 26640 raft_consensus.cc:2804] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.342600 26372 raft_consensus.cc:2468] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.342829 26640 raft_consensus.cc:697] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:28.344154 26640 consensus_queue.cc:237] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.352576 26271 catalog_manager.cc:5697] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.450403 26608 raft_consensus.cc:993] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37: : Instructing follower 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:28.450992 26658 raft_consensus.cc:1081] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Signalling peer 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:28.452760 26372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52408
I20260709 13:02:28.453327 26372 raft_consensus.cc:493] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.453658 26372 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.459829 26372 raft_consensus.cc:515] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.462059 26372 leader_election.cc:290] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:28.462888 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:28.463162 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:28.463624 26448 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.463781 26522 raft_consensus.cc:3055] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:28.464224 26522 raft_consensus.cc:740] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:28.464902 26522 consensus_queue.cc:260] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.466166 26522 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.470698 26448 raft_consensus.cc:2468] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:28.472077 26337 leader_election.cc:304] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:28.472657 26522 raft_consensus.cc:2468] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:28.473074 26568 raft_consensus.cc:2804] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.473644 26568 raft_consensus.cc:697] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:28.474625 26568 consensus_queue.cc:237] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.482681 26271 catalog_manager.cc:5697] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 1 to 2, leader changed from efa1b588222545e18dff850a02f8ec37 (127.12.30.67) to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.534857 26656 raft_consensus.cc:993] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.535336 26656 raft_consensus.cc:1081] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.537003 26448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:39012
I20260709 13:02:28.537547 26448 raft_consensus.cc:493] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.537855 26448 raft_consensus.cc:3060] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.543210 26448 raft_consensus.cc:515] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.545092 26448 leader_election.cc:290] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:28.545990 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:28.546305 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e2049c522743d984904f37d2bff5cb" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:28.546738 26372 raft_consensus.cc:3055] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:28.546926 26522 raft_consensus.cc:3060] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.547155 26372 raft_consensus.cc:740] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:28.548161 26372 consensus_queue.cc:260] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.549173 26372 raft_consensus.cc:3060] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.552724 26522 raft_consensus.cc:2468] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.553913 26413 leader_election.cc:304] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:28.554672 26640 raft_consensus.cc:2804] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.554934 26372 raft_consensus.cc:2468] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.555110 26640 raft_consensus.cc:697] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:28.556682 26640 consensus_queue.cc:237] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.564786 26271 catalog_manager.cc:5697] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.586007 26658 raft_consensus.cc:993] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37: : Instructing follower 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:28.586385 26661 raft_consensus.cc:993] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.586632 26608 raft_consensus.cc:1081] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Signalling peer 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:28.586984 26658 raft_consensus.cc:1081] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.588459 26372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52408
I20260709 13:02:28.588613 26448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:38998
I20260709 13:02:28.589250 26372 raft_consensus.cc:493] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.589646 26448 raft_consensus.cc:493] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.589725 26372 raft_consensus.cc:3060] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.590047 26448 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.595992 26448 raft_consensus.cc:515] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.596190 26372 raft_consensus.cc:515] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.598399 26448 leader_election.cc:290] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:28.599079 26372 leader_election.cc:290] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:28.599272 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:28.600127 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:28.600576 26373 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.600482 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d70bd89a6f7403c9fa79bf2ed68a6c3" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:28.600538 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:28.600713 26520 raft_consensus.cc:3055] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:28.601681 26448 raft_consensus.cc:3060] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.601737 26522 raft_consensus.cc:3055] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:28.602542 26520 raft_consensus.cc:740] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:28.602941 26522 raft_consensus.cc:740] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:28.603739 26520 consensus_queue.cc:260] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.604329 26522 consensus_queue.cc:260] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.605456 26520 raft_consensus.cc:3060] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.605944 26522 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.610394 26373 raft_consensus.cc:2468] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.610608 26448 raft_consensus.cc:2468] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:28.611709 26411 leader_election.cc:304] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:28.612987 26640 raft_consensus.cc:2804] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.613055 26337 leader_election.cc:304] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:28.613533 26640 raft_consensus.cc:697] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:28.614118 26656 raft_consensus.cc:2804] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.614130 26520 raft_consensus.cc:2468] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:28.614768 26656 raft_consensus.cc:697] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:28.614653 26522 raft_consensus.cc:2468] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.614775 26640 consensus_queue.cc:237] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.616828 26656 consensus_queue.cc:237] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.625224 26271 catalog_manager.cc:5697] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from efa1b588222545e18dff850a02f8ec37 (127.12.30.67) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.625689 26269 catalog_manager.cc:5697] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 1 to 2, leader changed from efa1b588222545e18dff850a02f8ec37 (127.12.30.67) to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.630987 26568 raft_consensus.cc:993] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:28.631718 26656 raft_consensus.cc:1081] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Signalling peer efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:28.633503 26520 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:52148
I20260709 13:02:28.634092 26520 raft_consensus.cc:493] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.634502 26520 raft_consensus.cc:3060] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.641458 26520 raft_consensus.cc:515] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.643780 26520 leader_election.cc:290] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:28.644706 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:28.645372 26373 raft_consensus.cc:3055] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:28.645375 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2f11d94f7f45f8982152cdfa656466" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:28.646035 26373 raft_consensus.cc:740] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:28.646287 26448 raft_consensus.cc:3060] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.647070 26373 consensus_queue.cc:260] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.648233 26373 raft_consensus.cc:3060] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.652431 26448 raft_consensus.cc:2468] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 2.
I20260709 13:02:28.653882 26373 raft_consensus.cc:2468] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 2.
I20260709 13:02:28.654003 26486 leader_election.cc:304] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:28.654922 26658 raft_consensus.cc:2804] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.655383 26658 raft_consensus.cc:697] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 2 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:28.656231 26658 consensus_queue.cc:237] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.665135 26269 catalog_manager.cc:5697] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 1 to 2, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 2 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.723901 26520 raft_consensus.cc:1275] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:28.725301 26656 consensus_queue.cc:1048] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:28.734565 26448 raft_consensus.cc:1275] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:28.736541 26656 consensus_queue.cc:1048] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:28.757802 26568 raft_consensus.cc:993] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.758328 26568 raft_consensus.cc:1081] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:28.759903 26448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:39012
I20260709 13:02:28.760447 26448 raft_consensus.cc:493] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:28.760772 26448 raft_consensus.cc:3060] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.767074 26448 raft_consensus.cc:515] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.768786 26448 leader_election.cc:290] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:28.769904 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:28.769922 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed59cec85356493cae6a495fbfb2d404" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:28.770541 26373 raft_consensus.cc:3055] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:28.770675 26520 raft_consensus.cc:3060] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.770838 26373 raft_consensus.cc:740] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:28.771720 26373 consensus_queue.cc:260] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.772816 26373 raft_consensus.cc:3060] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:28.776798 26520 raft_consensus.cc:2468] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.778473 26413 leader_election.cc:304] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:28.778746 26373 raft_consensus.cc:2468] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:28.779413 26640 raft_consensus.cc:2804] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:28.779959 26640 raft_consensus.cc:697] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:28.780745 26640 consensus_queue.cc:237] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:28.788129 26271 catalog_manager.cc:5697] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:28.856807 26448 raft_consensus.cc:1275] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:28.858203 26568 consensus_queue.cc:1048] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:28.860342 26520 raft_consensus.cc:1275] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:28.862349 26640 consensus_queue.cc:1048] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:28.870076 26522 raft_consensus.cc:1275] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:28.872931 26568 consensus_queue.cc:1048] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:28.886351 26373 raft_consensus.cc:1275] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:28.893013 26640 consensus_queue.cc:1048] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:29.062847 26520 raft_consensus.cc:1275] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.064453 26640 consensus_queue.cc:1048] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.068471 26373 raft_consensus.cc:1275] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.070370 26640 consensus_queue.cc:1048] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.078797 26372 raft_consensus.cc:1275] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.082947 26640 consensus_queue.cc:1048] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.085168 26522 raft_consensus.cc:1275] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.087146 26672 consensus_queue.cc:1048] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.128114 26448 raft_consensus.cc:1275] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.129680 26568 consensus_queue.cc:1048] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.139143 26522 raft_consensus.cc:1275] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.140368 26373 raft_consensus.cc:1275] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.141136 26665 consensus_queue.cc:1048] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.142030 26658 consensus_queue.cc:1048] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.153587 26448 raft_consensus.cc:1275] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.155725 26658 consensus_queue.cc:1048] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.367614 26522 raft_consensus.cc:1275] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.369088 26640 consensus_queue.cc:1048] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:29.379119 26373 raft_consensus.cc:1275] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:29.381129 26640 consensus_queue.cc:1048] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:31.268893 26271 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35062:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:31.271754 26271 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:31.335218 26360 tablet_service.cc:1511] Processing CreateTablet for tablet 6b61a7f8106b49608bcbe252db84ba1b (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:31.336539 26434 tablet_service.cc:1511] Processing CreateTablet for tablet 6b61a7f8106b49608bcbe252db84ba1b (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:31.337193 26360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b61a7f8106b49608bcbe252db84ba1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.340556 26359 tablet_service.cc:1511] Processing CreateTablet for tablet b14217c326114ae4b16d142c842aab60 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:31.338195 26434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b61a7f8106b49608bcbe252db84ba1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.342171 26359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b14217c326114ae4b16d142c842aab60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.338721 26504 tablet_service.cc:1511] Processing CreateTablet for tablet 6b61a7f8106b49608bcbe252db84ba1b (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:31.345103 26504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b61a7f8106b49608bcbe252db84ba1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.347941 26437 tablet_service.cc:1511] Processing CreateTablet for tablet b14217c326114ae4b16d142c842aab60 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:31.348770 26506 tablet_service.cc:1511] Processing CreateTablet for tablet b14217c326114ae4b16d142c842aab60 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:31.350208 26437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b14217c326114ae4b16d142c842aab60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.351441 26506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b14217c326114ae4b16d142c842aab60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.358817 26362 tablet_service.cc:1511] Processing CreateTablet for tablet 6f1e63296a834ce8b40da0ca2db516af (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:31.360891 26362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f1e63296a834ce8b40da0ca2db516af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.361032 26436 tablet_service.cc:1511] Processing CreateTablet for tablet 6f1e63296a834ce8b40da0ca2db516af (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:31.363071 26436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f1e63296a834ce8b40da0ca2db516af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.379889 26509 tablet_service.cc:1511] Processing CreateTablet for tablet 6f1e63296a834ce8b40da0ca2db516af (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:31.382069 26509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f1e63296a834ce8b40da0ca2db516af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.383181 26361 tablet_service.cc:1511] Processing CreateTablet for tablet b66b819e22154b18ae674dd737f94846 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:31.384960 26361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b66b819e22154b18ae674dd737f94846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.412696 26435 tablet_service.cc:1511] Processing CreateTablet for tablet b66b819e22154b18ae674dd737f94846 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:31.414546 26435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b66b819e22154b18ae674dd737f94846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.415078 26698 tablet_bootstrap.cc:492] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.420415 26357 tablet_service.cc:1511] Processing CreateTablet for tablet 8279219bcb70418081b545fd56e267aa (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:31.422824 26357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8279219bcb70418081b545fd56e267aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.426086 26432 tablet_service.cc:1511] Processing CreateTablet for tablet 8279219bcb70418081b545fd56e267aa (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:31.427824 26432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8279219bcb70418081b545fd56e267aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.428844 26506 tablet_service.cc:1511] Processing CreateTablet for tablet b66b819e22154b18ae674dd737f94846 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:31.430300 26506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b66b819e22154b18ae674dd737f94846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.447486 26504 tablet_service.cc:1511] Processing CreateTablet for tablet 8279219bcb70418081b545fd56e267aa (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:31.449092 26504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8279219bcb70418081b545fd56e267aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.448170 26362 tablet_service.cc:1511] Processing CreateTablet for tablet 4cf9803ebb9d44b7addf794d5683468c (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:31.451201 26362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf9803ebb9d44b7addf794d5683468c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.456713 26698 tablet_bootstrap.cc:654] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.462402 26699 tablet_bootstrap.cc:492] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:31.465631 26700 tablet_bootstrap.cc:492] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:31.469094 26437 tablet_service.cc:1511] Processing CreateTablet for tablet 4cf9803ebb9d44b7addf794d5683468c (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:31.472481 26434 tablet_service.cc:1511] Processing CreateTablet for tablet 35e5c724781f4a038e42c66c91fd0a6b (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:31.480013 26508 tablet_service.cc:1511] Processing CreateTablet for tablet 35e5c724781f4a038e42c66c91fd0a6b (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:31.481280 26434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e5c724781f4a038e42c66c91fd0a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.481606 26508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e5c724781f4a038e42c66c91fd0a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.476394 26360 tablet_service.cc:1511] Processing CreateTablet for tablet 000f44e8c7b641bcbf1a129b4ed9d49e (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:31.476686 26359 tablet_service.cc:1511] Processing CreateTablet for tablet 35e5c724781f4a038e42c66c91fd0a6b (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:31.484817 26363 tablet_service.cc:1511] Processing CreateTablet for tablet 4b22d31181384271a22afd9139db8879 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:31.485212 26505 tablet_service.cc:1511] Processing CreateTablet for tablet 000f44e8c7b641bcbf1a129b4ed9d49e (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:31.477505 26436 tablet_service.cc:1511] Processing CreateTablet for tablet 000f44e8c7b641bcbf1a129b4ed9d49e (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:31.486600 26505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000f44e8c7b641bcbf1a129b4ed9d49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.486747 26436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000f44e8c7b641bcbf1a129b4ed9d49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.484879 26360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000f44e8c7b641bcbf1a129b4ed9d49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.492676 26437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf9803ebb9d44b7addf794d5683468c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.479538 26509 tablet_service.cc:1511] Processing CreateTablet for tablet 4cf9803ebb9d44b7addf794d5683468c (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:31.493019 26359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e5c724781f4a038e42c66c91fd0a6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.494012 26509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cf9803ebb9d44b7addf794d5683468c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.496474 26363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b22d31181384271a22afd9139db8879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.518618 26699 tablet_bootstrap.cc:654] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.523314 26438 tablet_service.cc:1511] Processing CreateTablet for tablet 4b22d31181384271a22afd9139db8879 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:31.525327 26700 tablet_bootstrap.cc:654] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.531782 26506 tablet_service.cc:1511] Processing CreateTablet for tablet 4b22d31181384271a22afd9139db8879 (DEFAULT_TABLE table=soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:31.537043 26438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b22d31181384271a22afd9139db8879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.537068 26506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b22d31181384271a22afd9139db8879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:31.558939 26698 tablet_bootstrap.cc:492] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:31.559612 26698 ts_tablet_manager.cc:1403] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.145s	user 0.009s	sys 0.007s
I20260709 13:02:31.568643 26700 tablet_bootstrap.cc:492] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:31.569480 26700 ts_tablet_manager.cc:1403] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.104s	user 0.007s	sys 0.012s
I20260709 13:02:31.573015 26700 raft_consensus.cc:359] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.579692 26700 raft_consensus.cc:385] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.580157 26700 raft_consensus.cc:740] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.578786 26698 raft_consensus.cc:359] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.581055 26698 raft_consensus.cc:385] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.581430 26698 raft_consensus.cc:740] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.582301 26698 consensus_queue.cc:260] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.586297 26698 ts_tablet_manager.cc:1434] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.002s
I20260709 13:02:31.588515 26698 tablet_bootstrap.cc:492] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.595642 26698 tablet_bootstrap.cc:654] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.586863 26700 consensus_queue.cc:260] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.597747 26700 ts_tablet_manager.cc:1434] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:02:31.598482 26699 tablet_bootstrap.cc:492] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:31.598831 26700 tablet_bootstrap.cc:492] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:31.599009 26699 ts_tablet_manager.cc:1403] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.137s	user 0.000s	sys 0.013s
I20260709 13:02:31.604529 26699 raft_consensus.cc:359] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.605683 26699 raft_consensus.cc:385] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.606132 26699 raft_consensus.cc:740] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.608110 26700 tablet_bootstrap.cc:654] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.608460 26699 consensus_queue.cc:260] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.610489 26698 tablet_bootstrap.cc:492] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:31.611159 26698 ts_tablet_manager.cc:1403] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.006s
I20260709 13:02:31.615232 26698 raft_consensus.cc:359] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.616417 26698 raft_consensus.cc:385] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.616463 26699 ts_tablet_manager.cc:1434] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 13:02:31.616998 26698 raft_consensus.cc:740] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.618315 26699 tablet_bootstrap.cc:492] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:31.618090 26698 consensus_queue.cc:260] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.619999 26700 tablet_bootstrap.cc:492] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:31.620597 26700 ts_tablet_manager.cc:1403] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.016s
I20260709 13:02:31.624311 26700 raft_consensus.cc:359] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.625140 26700 raft_consensus.cc:385] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.625511 26700 raft_consensus.cc:740] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.625801 26699 tablet_bootstrap.cc:654] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.626791 26698 ts_tablet_manager.cc:1434] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:02:31.626384 26700 consensus_queue.cc:260] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.627907 26698 tablet_bootstrap.cc:492] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.629004 26700 ts_tablet_manager.cc:1434] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:02:31.632606 26700 tablet_bootstrap.cc:492] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:31.641428 26700 tablet_bootstrap.cc:654] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.642760 26698 tablet_bootstrap.cc:654] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.643270 26699 tablet_bootstrap.cc:492] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:31.644342 26699 ts_tablet_manager.cc:1403] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.012s
I20260709 13:02:31.651532 26699 raft_consensus.cc:359] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.652943 26699 raft_consensus.cc:385] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.653470 26699 raft_consensus.cc:740] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.654384 26699 consensus_queue.cc:260] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.656243 26700 tablet_bootstrap.cc:492] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:31.656314 26698 tablet_bootstrap.cc:492] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:31.657074 26699 ts_tablet_manager.cc:1434] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:02:31.657222 26700 ts_tablet_manager.cc:1403] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.008s
I20260709 13:02:31.657267 26698 ts_tablet_manager.cc:1403] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.007s
I20260709 13:02:31.658429 26699 tablet_bootstrap.cc:492] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:31.661798 26700 raft_consensus.cc:359] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.662014 26698 raft_consensus.cc:359] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.662770 26698 raft_consensus.cc:385] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.662796 26700 raft_consensus.cc:385] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.663316 26698 raft_consensus.cc:740] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.663404 26700 raft_consensus.cc:740] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.664294 26698 consensus_queue.cc:260] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.664565 26700 consensus_queue.cc:260] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.666286 26699 tablet_bootstrap.cc:654] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.667318 26698 ts_tablet_manager.cc:1434] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:02:31.668509 26698 tablet_bootstrap.cc:492] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.672389 26640 raft_consensus.cc:493] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.672981 26640 raft_consensus.cc:515] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.675487 26640 leader_election.cc:290] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:31.676013 26698 tablet_bootstrap.cc:654] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.677480 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14217c326114ae4b16d142c842aab60" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:31.678217 26372 raft_consensus.cc:2468] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:31.680881 26411 leader_election.cc:304] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:31.676574 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14217c326114ae4b16d142c842aab60" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:31.681739 26640 raft_consensus.cc:2804] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:31.682272 26640 raft_consensus.cc:493] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.682341 26522 raft_consensus.cc:2468] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:31.682793 26640 raft_consensus.cc:3060] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.681072 26699 tablet_bootstrap.cc:492] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:31.683563 26699 ts_tablet_manager.cc:1403] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.006s
I20260709 13:02:31.686858 26699 raft_consensus.cc:359] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.687599 26699 raft_consensus.cc:385] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.687948 26699 raft_consensus.cc:740] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.687150 26700 ts_tablet_manager.cc:1434] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 13:02:31.689325 26700 tablet_bootstrap.cc:492] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:31.688869 26699 consensus_queue.cc:260] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.691562 26699 ts_tablet_manager.cc:1434] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:31.692073 26640 raft_consensus.cc:515] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.692699 26699 tablet_bootstrap.cc:492] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:31.695245 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14217c326114ae4b16d142c842aab60" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:31.695878 26372 raft_consensus.cc:3060] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.700676 26699 tablet_bootstrap.cc:654] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.707055 26700 tablet_bootstrap.cc:654] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.709257 26687 raft_consensus.cc:493] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.709865 26687 raft_consensus.cc:515] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.710160 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14217c326114ae4b16d142c842aab60" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:31.711105 26520 raft_consensus.cc:3060] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.713414 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:31.714131 26448 raft_consensus.cc:2468] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:31.714190 26699 tablet_bootstrap.cc:492] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:31.714877 26372 raft_consensus.cc:2468] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 1.
I20260709 13:02:31.715345 26699 ts_tablet_manager.cc:1403] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:02:31.715377 26640 leader_election.cc:290] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:31.714262 26698 tablet_bootstrap.cc:492] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:31.715966 26486 leader_election.cc:304] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:31.716777 26698 ts_tablet_manager.cc:1403] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.000s
I20260709 13:02:31.717921 26687 leader_election.cc:290] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:31.718559 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:31.719201 26411 leader_election.cc:304] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:31.719369 26373 raft_consensus.cc:2468] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:31.719169 26699 raft_consensus.cc:359] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.720794 26699 raft_consensus.cc:385] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.721273 26699 raft_consensus.cc:740] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.718561 26687 raft_consensus.cc:2804] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:31.721755 26687 raft_consensus.cc:493] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.722146 26687 raft_consensus.cc:3060] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.726053 26640 raft_consensus.cc:2804] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:31.726853 26640 raft_consensus.cc:697] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:31.727475 26698 raft_consensus.cc:359] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.727510 26707 raft_consensus.cc:493] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.732172 26698 raft_consensus.cc:385] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.732600 26698 raft_consensus.cc:740] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.732332 26707 raft_consensus.cc:515] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.732858 26640 consensus_queue.cc:237] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.733464 26698 consensus_queue.cc:260] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.734797 26707 leader_election.cc:290] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:31.736122 26698 ts_tablet_manager.cc:1434] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:02:31.736478 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:31.737142 26522 raft_consensus.cc:2468] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:31.738343 26413 leader_election.cc:304] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:31.739768 26698 tablet_bootstrap.cc:492] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.741501 26568 raft_consensus.cc:493] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.742013 26568 raft_consensus.cc:515] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.745419 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:31.745594 26687 raft_consensus.cc:515] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.746718 26448 raft_consensus.cc:2468] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:31.748931 26337 leader_election.cc:304] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:31.749547 26687 leader_election.cc:290] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:31.752493 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:31.753473 26448 raft_consensus.cc:3060] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.758549 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:31.760056 26568 leader_election.cc:290] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:31.760739 26568 raft_consensus.cc:2804] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:31.761132 26568 raft_consensus.cc:493] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.761596 26568 raft_consensus.cc:3060] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.759734 26522 raft_consensus.cc:2468] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:31.772276 26700 tablet_bootstrap.cc:492] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:31.773434 26700 ts_tablet_manager.cc:1403] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.019s
I20260709 13:02:31.778429 26700 raft_consensus.cc:359] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.731025 26520 raft_consensus.cc:2468] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 1.
I20260709 13:02:31.727388 26699 consensus_queue.cc:260] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.779621 26700 raft_consensus.cc:385] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.780540 26700 raft_consensus.cc:740] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.780789 26707 raft_consensus.cc:493] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.781380 26707 raft_consensus.cc:515] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.781548 26700 consensus_queue.cc:260] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.785234 26700 ts_tablet_manager.cc:1434] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:02:31.786568 26700 tablet_bootstrap.cc:492] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:31.786746 26707 leader_election.cc:290] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:31.788185 26707 raft_consensus.cc:2804] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:31.788908 26698 tablet_bootstrap.cc:654] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.789979 26707 raft_consensus.cc:493] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.790841 26707 raft_consensus.cc:3060] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.791419 26448 raft_consensus.cc:2468] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:31.793128 26486 leader_election.cc:304] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:31.794111 26687 raft_consensus.cc:2804] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:31.794761 26687 raft_consensus.cc:697] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:31.795998 26687 consensus_queue.cc:237] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.798480 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:31.798904 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:31.799849 26371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:31.800741 26371 raft_consensus.cc:3060] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.803857 26698 tablet_bootstrap.cc:492] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:31.809286 26698 ts_tablet_manager.cc:1403] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.070s	user 0.017s	sys 0.025s
I20260709 13:02:31.810977 26371 raft_consensus.cc:2468] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
W20260709 13:02:31.812776 26411 leader_election.cc:343] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:31.813023 26698 raft_consensus.cc:359] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.813977 26698 raft_consensus.cc:385] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.795847 26699 ts_tablet_manager.cc:1434] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.080s	user 0.009s	sys 0.005s
I20260709 13:02:31.817536 26568 raft_consensus.cc:515] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
W20260709 13:02:31.817919 26413 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.30.67:44275, user_credentials={real_user=slave}} blocked reactor thread for 79734.5us
I20260709 13:02:31.818843 26373 raft_consensus.cc:2393] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07f704db827b4614991c7f6e49ae5207 in current term 1: Already voted for candidate 1d387e18cd424760b5c49569a29c26a6 in this term.
I20260709 13:02:31.818887 26699 tablet_bootstrap.cc:492] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:31.819592 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:31.817503 26707 raft_consensus.cc:515] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.821310 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:31.822059 26448 raft_consensus.cc:2393] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1d387e18cd424760b5c49569a29c26a6 in current term 1: Already voted for candidate 07f704db827b4614991c7f6e49ae5207 in this term.
I20260709 13:02:31.822484 26568 leader_election.cc:290] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:31.823956 26707 leader_election.cc:290] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:31.824231 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
W20260709 13:02:31.824468 26413 leader_election.cc:343] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:31.824952 26413 leader_election.cc:304] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207; no voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:31.824944 26522 raft_consensus.cc:3060] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:31.827035 26699 tablet_bootstrap.cc:654] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.827062 26521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:31.814335 26698 raft_consensus.cc:740] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.828467 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b61a7f8106b49608bcbe252db84ba1b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:31.817442 26700 tablet_bootstrap.cc:654] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.828557 26698 consensus_queue.cc:260] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.830224 26373 raft_consensus.cc:2393] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07f704db827b4614991c7f6e49ae5207 in current term 1: Already voted for candidate 1d387e18cd424760b5c49569a29c26a6 in this term.
I20260709 13:02:31.831403 26411 leader_election.cc:304] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207; no voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:31.831830 26698 ts_tablet_manager.cc:1434] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:02:31.832623 26715 raft_consensus.cc:2749] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:31.833033 26698 tablet_bootstrap.cc:492] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.829124 26707 raft_consensus.cc:2749] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:31.814910 26269 catalog_manager.cc:5697] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:31.837647 26271 catalog_manager.cc:5697] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:31.842070 26698 tablet_bootstrap.cc:654] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.849606 26522 raft_consensus.cc:2468] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:31.854588 26338 leader_election.cc:304] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:31.857095 26716 raft_consensus.cc:2804] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:31.857777 26716 raft_consensus.cc:697] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:31.858933 26716 consensus_queue.cc:237] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.890105 26698 tablet_bootstrap.cc:492] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:31.890648 26698 ts_tablet_manager.cc:1403] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.058s	user 0.031s	sys 0.013s
I20260709 13:02:31.890981 26699 tablet_bootstrap.cc:492] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:31.891661 26699 ts_tablet_manager.cc:1403] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.006s
I20260709 13:02:31.891839 26700 tablet_bootstrap.cc:492] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:31.892593 26700 ts_tablet_manager.cc:1403] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.106s	user 0.027s	sys 0.036s
I20260709 13:02:31.894805 26698 raft_consensus.cc:359] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.896064 26698 raft_consensus.cc:385] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.896443 26698 raft_consensus.cc:740] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.895524 26699 raft_consensus.cc:359] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.897229 26699 raft_consensus.cc:385] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.897740 26699 raft_consensus.cc:740] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.898244 26270 catalog_manager.cc:5697] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:31.898754 26699 consensus_queue.cc:260] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.901443 26699 ts_tablet_manager.cc:1434] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:02:31.901682 26700 raft_consensus.cc:359] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.902453 26700 raft_consensus.cc:385] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.902573 26699 tablet_bootstrap.cc:492] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:31.902807 26700 raft_consensus.cc:740] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.897286 26698 consensus_queue.cc:260] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.903736 26700 consensus_queue.cc:260] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.905841 26698 ts_tablet_manager.cc:1434] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:02:31.906517 26700 ts_tablet_manager.cc:1434] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:02:31.907629 26700 tablet_bootstrap.cc:492] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:31.907083 26698 tablet_bootstrap.cc:492] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.915730 26700 tablet_bootstrap.cc:654] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.919485 26698 tablet_bootstrap.cc:654] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.919983 26699 tablet_bootstrap.cc:654] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.930895 26700 tablet_bootstrap.cc:492] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:31.931442 26700 ts_tablet_manager.cc:1403] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:02:31.936558 26700 raft_consensus.cc:359] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.947669 26700 raft_consensus.cc:385] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.948068 26700 raft_consensus.cc:740] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.949308 26699 tablet_bootstrap.cc:492] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:31.949033 26700 consensus_queue.cc:260] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.949904 26699 ts_tablet_manager.cc:1403] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.048s	user 0.021s	sys 0.016s
I20260709 13:02:31.951653 26700 ts_tablet_manager.cc:1434] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:02:31.953263 26699 raft_consensus.cc:359] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.954068 26699 raft_consensus.cc:385] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.954387 26699 raft_consensus.cc:740] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.955304 26698 tablet_bootstrap.cc:492] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:31.955111 26699 consensus_queue.cc:260] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.954079 26700 tablet_bootstrap.cc:492] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:31.956521 26698 ts_tablet_manager.cc:1403] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.013s
I20260709 13:02:31.958725 26699 ts_tablet_manager.cc:1434] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:02:31.960002 26699 tablet_bootstrap.cc:492] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:31.964082 26700 tablet_bootstrap.cc:654] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.963817 26698 raft_consensus.cc:359] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.965008 26698 raft_consensus.cc:385] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.968246 26698 raft_consensus.cc:740] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.969422 26698 consensus_queue.cc:260] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.972295 26698 ts_tablet_manager.cc:1434] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:02:31.972505 26699 tablet_bootstrap.cc:654] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.974550 26698 tablet_bootstrap.cc:492] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:31.985386 26700 tablet_bootstrap.cc:492] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:31.986033 26700 ts_tablet_manager.cc:1403] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.006s
W20260709 13:02:31.988791 26319 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:31.994588 26699 tablet_bootstrap.cc:492] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:31.995160 26699 ts_tablet_manager.cc:1403] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.015s
I20260709 13:02:31.996359 26698 tablet_bootstrap.cc:654] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:31.998021 26699 raft_consensus.cc:359] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:31.998672 26699 raft_consensus.cc:385] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:31.998932 26699 raft_consensus.cc:740] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:31.999621 26699 consensus_queue.cc:260] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.000543 26700 raft_consensus.cc:359] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.001410 26700 raft_consensus.cc:385] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:32.001715 26700 raft_consensus.cc:740] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:32.002084 26698 tablet_bootstrap.cc:492] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:32.001998 26699 ts_tablet_manager.cc:1434] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:32.002743 26698 ts_tablet_manager.cc:1403] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.008s
I20260709 13:02:32.002629 26700 consensus_queue.cc:260] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.003608 26699 tablet_bootstrap.cc:492] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:32.005689 26700 ts_tablet_manager.cc:1434] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:02:32.006170 26698 raft_consensus.cc:359] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.006858 26698 raft_consensus.cc:385] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:32.007030 26700 tablet_bootstrap.cc:492] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:32.007130 26698 raft_consensus.cc:740] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:32.008098 26698 consensus_queue.cc:260] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.017163 26699 tablet_bootstrap.cc:654] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:32.014952 26700 tablet_bootstrap.cc:654] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:32.027992 26698 ts_tablet_manager.cc:1434] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:02:32.029035 26698 tablet_bootstrap.cc:492] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: Bootstrap starting.
I20260709 13:02:32.029677 26699 tablet_bootstrap.cc:492] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:32.030042 26716 raft_consensus.cc:493] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.030221 26699 ts_tablet_manager.cc:1403] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.003s
I20260709 13:02:32.030591 26716 raft_consensus.cc:515] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.034392 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8279219bcb70418081b545fd56e267aa" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:32.034560 26699 raft_consensus.cc:359] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.035159 26522 raft_consensus.cc:2468] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:32.035604 26699 raft_consensus.cc:385] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:32.036018 26699 raft_consensus.cc:740] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:32.037019 26699 consensus_queue.cc:260] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.037511 26338 leader_election.cc:304] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.037882 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8279219bcb70418081b545fd56e267aa" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:32.038620 26568 raft_consensus.cc:2804] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:32.038652 26448 raft_consensus.cc:2468] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:32.039253 26568 raft_consensus.cc:493] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.039669 26699 ts_tablet_manager.cc:1434] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:02:32.042811 26700 tablet_bootstrap.cc:492] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:32.039844 26568 raft_consensus.cc:3060] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.044967 26700 ts_tablet_manager.cc:1403] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.005s
I20260709 13:02:32.047216 26698 tablet_bootstrap.cc:654] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:32.048780 26700 raft_consensus.cc:359] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.050203 26700 raft_consensus.cc:385] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:32.050472 26700 raft_consensus.cc:740] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:32.051262 26700 consensus_queue.cc:260] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.044075 26716 leader_election.cc:290] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.057708 26568 raft_consensus.cc:515] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.043807 26699 tablet_bootstrap.cc:492] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207: Bootstrap starting.
I20260709 13:02:32.064091 26700 ts_tablet_manager.cc:1434] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:02:32.065408 26700 tablet_bootstrap.cc:492] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: Bootstrap starting.
I20260709 13:02:32.068394 26698 tablet_bootstrap.cc:492] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: No bootstrap required, opened a new log
I20260709 13:02:32.068431 26699 tablet_bootstrap.cc:654] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:32.069175 26698 ts_tablet_manager.cc:1403] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.008s
I20260709 13:02:32.072448 26698 raft_consensus.cc:359] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.073261 26698 raft_consensus.cc:385] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:32.073534 26698 raft_consensus.cc:740] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Initialized, Role: FOLLOWER
I20260709 13:02:32.074786 26700 tablet_bootstrap.cc:654] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:32.075230 26698 consensus_queue.cc:260] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.078441 26698 ts_tablet_manager.cc:1434] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:02:32.081991 26699 tablet_bootstrap.cc:492] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207: No bootstrap required, opened a new log
I20260709 13:02:32.082444 26700 tablet_bootstrap.cc:492] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: No bootstrap required, opened a new log
I20260709 13:02:32.082576 26699 ts_tablet_manager.cc:1403] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.008s
I20260709 13:02:32.082967 26700 ts_tablet_manager.cc:1403] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.004s
I20260709 13:02:32.083815 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8279219bcb70418081b545fd56e267aa" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.084002 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8279219bcb70418081b545fd56e267aa" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.084435 26448 raft_consensus.cc:3060] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.084573 26522 raft_consensus.cc:3060] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.085558 26699 raft_consensus.cc:359] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.086263 26699 raft_consensus.cc:385] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:32.086062 26700 raft_consensus.cc:359] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.086601 26699 raft_consensus.cc:740] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Initialized, Role: FOLLOWER
I20260709 13:02:32.086820 26700 raft_consensus.cc:385] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:32.087328 26699 consensus_queue.cc:260] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.087144 26700 raft_consensus.cc:740] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Initialized, Role: FOLLOWER
I20260709 13:02:32.087997 26687 raft_consensus.cc:493] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.088557 26687 raft_consensus.cc:515] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.089700 26699 ts_tablet_manager.cc:1434] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 13:02:32.090700 26687 leader_election.cc:290] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:32.091977 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:32.092221 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:32.092705 26447 raft_consensus.cc:2468] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:32.092908 26373 raft_consensus.cc:2468] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:32.093695 26522 raft_consensus.cc:2468] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:32.094319 26485 leader_election.cc:304] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.094846 26338 leader_election.cc:304] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.095144 26687 raft_consensus.cc:2804] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:32.094947 26700 consensus_queue.cc:260] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.095572 26687 raft_consensus.cc:493] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.095975 26568 leader_election.cc:290] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.095748 26738 raft_consensus.cc:2804] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:32.096974 26738 raft_consensus.cc:697] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:32.096035 26687 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.097730 26700 ts_tablet_manager.cc:1434] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.005s
I20260709 13:02:32.097939 26738 consensus_queue.cc:237] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.106729 26687 raft_consensus.cc:515] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.097770 26448 raft_consensus.cc:2468] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:32.110826 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.111404 26373 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.117735 26271 catalog_manager.cc:5697] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.119241 26687 leader_election.cc:290] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:32.120927 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.121563 26448 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.121711 26373 raft_consensus.cc:2468] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:32.122922 26485 leader_election.cc:304] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.123693 26687 raft_consensus.cc:2804] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:32.124289 26687 raft_consensus.cc:697] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:32.125175 26687 consensus_queue.cc:237] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.129021 26448 raft_consensus.cc:2468] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:32.136261 26271 catalog_manager.cc:5697] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:32.181777 26707 raft_consensus.cc:493] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.182384 26707 raft_consensus.cc:515] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.183432 26715 consensus_queue.cc:1048] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.185026 26707 leader_election.cc:290] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.185415 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:32.185757 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:32.186352 26373 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:32.186441 26522 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:32.187543 26411 leader_election.cc:304] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:32.188864 26687 raft_consensus.cc:493] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.189674 26687 raft_consensus.cc:515] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.192055 26687 leader_election.cc:290] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:32.193405 26715 raft_consensus.cc:2804] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:32.193941 26715 raft_consensus.cc:493] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.193996 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b22d31181384271a22afd9139db8879" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:32.194424 26715 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.194741 26448 raft_consensus.cc:2468] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:32.196228 26486 leader_election.cc:304] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.197449 26707 consensus_queue.cc:1048] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.199206 26720 consensus_queue.cc:1048] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.197975 26687 raft_consensus.cc:2804] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:32.200337 26687 raft_consensus.cc:493] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.202502 26687 raft_consensus.cc:3060] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.193428 26373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b22d31181384271a22afd9139db8879" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:32.203509 26373 raft_consensus.cc:2468] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:32.215852 26720 raft_consensus.cc:493] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.216778 26720 raft_consensus.cc:515] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.218791 26707 raft_consensus.cc:493] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.219820 26707 raft_consensus.cc:515] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.221182 26720 leader_election.cc:290] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:32.222151 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:32.223162 26372 raft_consensus.cc:2468] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:32.223330 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:32.223695 26707 leader_election.cc:290] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.224251 26448 raft_consensus.cc:2468] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 0.
I20260709 13:02:32.224403 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:32.225267 26522 raft_consensus.cc:2468] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:32.225646 26486 leader_election.cc:304] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.226799 26720 raft_consensus.cc:2804] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:32.227219 26720 raft_consensus.cc:493] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.227061 26687 raft_consensus.cc:515] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.227617 26720 raft_consensus.cc:3060] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.232163 26413 leader_election.cc:304] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.234283 26707 raft_consensus.cc:2804] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:32.236896 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6" is_pre_election: true
I20260709 13:02:32.236771 26370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b22d31181384271a22afd9139db8879" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.237849 26372 raft_consensus.cc:2468] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 0.
I20260709 13:02:32.232239 26715 raft_consensus.cc:515] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.238078 26370 raft_consensus.cc:3060] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.236856 26707 raft_consensus.cc:493] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.242084 26707 raft_consensus.cc:3060] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.250607 26370 raft_consensus.cc:2468] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:32.254801 26485 leader_election.cc:304] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.255867 26713 raft_consensus.cc:2804] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:32.260272 26738 raft_consensus.cc:493] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.260864 26738 raft_consensus.cc:515] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.263332 26738 leader_election.cc:290] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.264156 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207" is_pre_election: true
I20260709 13:02:32.264845 26448 raft_consensus.cc:2468] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:32.266181 26720 raft_consensus.cc:515] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.267558 26371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.268404 26371 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.269001 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37" is_pre_election: true
I20260709 13:02:32.269114 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.269743 26522 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.270103 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.259775 26707 raft_consensus.cc:515] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.270903 26720 leader_election.cc:290] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:32.271656 26337 leader_election.cc:304] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:32.272460 26448 raft_consensus.cc:2393] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate efa1b588222545e18dff850a02f8ec37 in current term 1: Already voted for candidate 07f704db827b4614991c7f6e49ae5207 in this term.
I20260709 13:02:32.273094 26715 leader_election.cc:290] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.274911 26738 raft_consensus.cc:2804] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:32.275473 26738 raft_consensus.cc:493] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.275841 26738 raft_consensus.cc:3060] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.277933 26522 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 1.
I20260709 13:02:32.279224 26413 leader_election.cc:304] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.280371 26715 raft_consensus.cc:2804] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:32.281610 26715 raft_consensus.cc:697] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:32.281701 26687 leader_election.cc:290] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:32.282636 26370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.282884 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b22d31181384271a22afd9139db8879" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.283420 26370 raft_consensus.cc:3060] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.282655 26715 consensus_queue.cc:237] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.269717 26520 raft_consensus.cc:2468] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 0.
I20260709 13:02:32.283598 26448 raft_consensus.cc:3060] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.288720 26713 raft_consensus.cc:697] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:32.290160 26713 consensus_queue.cc:237] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.297866 26568 consensus_queue.cc:1048] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.298658 26707 leader_election.cc:290] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.298882 26448 raft_consensus.cc:2468] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:32.300964 26738 raft_consensus.cc:515] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.302348 26371 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 1.
I20260709 13:02:32.302489 26687 consensus_queue.cc:1048] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.304559 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.305179 26520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.311407 26738 leader_election.cc:290] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.310984 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.312016 26521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.322887 26520 raft_consensus.cc:2393] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 07f704db827b4614991c7f6e49ae5207 in current term 1: Already voted for candidate efa1b588222545e18dff850a02f8ec37 in this term.
I20260709 13:02:32.323043 26521 raft_consensus.cc:3060] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.324513 26413 leader_election.cc:304] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207; no voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37
I20260709 13:02:32.326045 26707 raft_consensus.cc:2749] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:32.317831 26271 catalog_manager.cc:5697] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:32.326880 26448 raft_consensus.cc:3060] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:32.312791 26370 raft_consensus.cc:2468] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 1.
I20260709 13:02:32.332864 26485 leader_election.cc:304] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:32.334730 26751 raft_consensus.cc:2804] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:32.335438 26751 raft_consensus.cc:697] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:32.340603 26448 raft_consensus.cc:2468] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:32.341962 26337 leader_election.cc:304] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:32.342870 26738 raft_consensus.cc:2804] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:32.343336 26738 raft_consensus.cc:697] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:32.344173 26738 consensus_queue.cc:237] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.336341 26751 consensus_queue.cc:237] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.336227 26521 raft_consensus.cc:2468] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 1.
I20260709 13:02:32.347991 26268 catalog_manager.cc:5697] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 1 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.374806 26271 catalog_manager.cc:5697] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.376678 26738 consensus_queue.cc:1048] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.381807 26268 catalog_manager.cc:5697] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 1 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:32.524106 26738 consensus_queue.cc:1048] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.525274 26447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b14217c326114ae4b16d142c842aab60"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
mode: GRACEFUL
new_leader_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:02:32.526260 26447 raft_consensus.cc:606] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Received request to transfer leadership to TS 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:32.534922 26522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
mode: GRACEFUL
new_leader_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52158
I20260709 13:02:32.535578 26522 raft_consensus.cc:606] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Received request to transfer leadership to TS 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:32.538053 26738 consensus_queue.cc:1048] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.546615 26448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b66b819e22154b18ae674dd737f94846"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
mode: GRACEFUL
new_leader_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:02:32.547394 26448 raft_consensus.cc:606] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Received request to transfer leadership to TS 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:32.555639 26370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8279219bcb70418081b545fd56e267aa"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52424
I20260709 13:02:32.556378 26370 raft_consensus.cc:606] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:32.563817 26522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52158
I20260709 13:02:32.564571 26522 raft_consensus.cc:606] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:32.571291 26370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52424
I20260709 13:02:32.572311 26370 raft_consensus.cc:606] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:32.579442 26522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
mode: GRACEFUL
new_leader_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:52158
I20260709 13:02:32.580274 26522 raft_consensus.cc:606] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Received request to transfer leadership to TS 07f704db827b4614991c7f6e49ae5207
I20260709 13:02:32.656636 26751 consensus_queue.cc:1048] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.665067 26687 raft_consensus.cc:993] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.665665 26755 raft_consensus.cc:1081] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.670053 26755 consensus_queue.cc:1048] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.671162 26448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:38998
I20260709 13:02:32.671936 26448 raft_consensus.cc:493] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:32.672346 26448 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.675961 26754 consensus_queue.cc:1048] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.681131 26448 raft_consensus.cc:515] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.684278 26371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.685516 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.686225 26522 raft_consensus.cc:3055] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.686535 26522 raft_consensus.cc:740] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:32.687321 26448 leader_election.cc:290] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.687350 26522 consensus_queue.cc:260] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.688810 26522 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.693982 26715 raft_consensus.cc:993] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: : Instructing follower 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:32.694572 26715 raft_consensus.cc:1081] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Signalling peer 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:32.696655 26371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b14217c326114ae4b16d142c842aab60"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52416
I20260709 13:02:32.697288 26371 raft_consensus.cc:493] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:32.697701 26371 raft_consensus.cc:3060] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.698305 26522 raft_consensus.cc:2468] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:32.699671 26413 leader_election.cc:304] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:32.700589 26640 raft_consensus.cc:2804] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:32.701064 26640 raft_consensus.cc:697] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:32.701972 26640 consensus_queue.cc:237] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.703840 26713 consensus_queue.cc:1048] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.715991 26371 raft_consensus.cc:515] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.720736 26371 leader_election.cc:290] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.720906 26268 catalog_manager.cc:5697] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from efa1b588222545e18dff850a02f8ec37 (127.12.30.67) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.722476 26448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14217c326114ae4b16d142c842aab60" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.723219 26448 raft_consensus.cc:3055] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.723315 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b14217c326114ae4b16d142c842aab60" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.723654 26448 raft_consensus.cc:740] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:32.724006 26522 raft_consensus.cc:3060] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.724665 26448 consensus_queue.cc:260] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.726006 26448 raft_consensus.cc:3060] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.729683 26755 consensus_queue.cc:1048] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.733878 26448 raft_consensus.cc:2468] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:32.736011 26337 leader_election.cc:304] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:32.737020 26738 raft_consensus.cc:2804] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:32.737632 26738 raft_consensus.cc:697] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:32.738531 26738 consensus_queue.cc:237] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.741554 26754 raft_consensus.cc:993] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.742178 26687 raft_consensus.cc:1081] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.742493 26713 raft_consensus.cc:993] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: : Instructing follower 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:32.742965 26713 raft_consensus.cc:1081] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Signalling peer 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:32.744851 26371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52408
I20260709 13:02:32.745446 26371 raft_consensus.cc:493] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:32.745842 26371 raft_consensus.cc:3060] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.748968 26447 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:38998
I20260709 13:02:32.749119 26754 consensus_queue.cc:1048] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.749747 26447 raft_consensus.cc:493] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:32.750206 26447 raft_consensus.cc:3060] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.756268 26371 raft_consensus.cc:515] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.760460 26522 raft_consensus.cc:2468] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:32.763317 26447 raft_consensus.cc:515] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.764528 26448 raft_consensus.cc:1240] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Rejecting Update request from peer efa1b588222545e18dff850a02f8ec37 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:32.771891 26687 consensus_queue.cc:1059] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.773528 26446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.774320 26446 raft_consensus.cc:3060] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.775252 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.775935 26522 raft_consensus.cc:3055] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.776136 26447 leader_election.cc:290] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.776304 26522 raft_consensus.cc:740] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:32.778429 26371 leader_election.cc:290] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.778487 26522 consensus_queue.cc:260] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.780973 26522 raft_consensus.cc:3060] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.781282 26521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1e63296a834ce8b40da0ca2db516af" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.783171 26521 raft_consensus.cc:3055] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.783572 26521 raft_consensus.cc:740] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:32.784976 26371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cf9803ebb9d44b7addf794d5683468c" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.785868 26568 consensus_queue.cc:1048] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:32.786808 26446 raft_consensus.cc:2468] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:32.784744 26521 consensus_queue.cc:260] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.788446 26521 raft_consensus.cc:3060] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.794699 26337 leader_election.cc:304] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:32.796118 26568 raft_consensus.cc:2804] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:32.796967 26568 raft_consensus.cc:697] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:32.791807 26522 raft_consensus.cc:2468] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:32.797968 26568 consensus_queue.cc:237] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.799991 26413 leader_election.cc:304] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 1d387e18cd424760b5c49569a29c26a6
I20260709 13:02:32.800969 26707 raft_consensus.cc:2804] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:32.801678 26707 raft_consensus.cc:697] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:32.802672 26707 consensus_queue.cc:237] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.804528 26270 catalog_manager.cc:5697] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 1 to 2, leader changed from 07f704db827b4614991c7f6e49ae5207 (127.12.30.66) to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.809448 26739 raft_consensus.cc:993] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.812549 26521 raft_consensus.cc:2468] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:32.811403 26716 raft_consensus.cc:1081] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.818042 26446 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:39012
I20260709 13:02:32.818709 26446 raft_consensus.cc:493] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:32.819226 26446 raft_consensus.cc:3060] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.829600 26738 consensus_queue.cc:1048] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.832281 26446 raft_consensus.cc:515] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.836090 26446 leader_election.cc:290] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.836222 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.837329 26372 raft_consensus.cc:3055] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.838050 26372 raft_consensus.cc:740] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:32.840435 26522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e5c724781f4a038e42c66c91fd0a6b" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.842933 26270 catalog_manager.cc:5697] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 1 to 2, leader changed from efa1b588222545e18dff850a02f8ec37 (127.12.30.67) to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.845405 26372 consensus_queue.cc:260] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.846786 26372 raft_consensus.cc:3060] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.857813 26372 raft_consensus.cc:2468] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:32.859668 26411 leader_election.cc:304] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: efa1b588222545e18dff850a02f8ec37
I20260709 13:02:32.860733 26707 raft_consensus.cc:2804] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:32.861305 26707 raft_consensus.cc:697] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:32.861977 26268 catalog_manager.cc:5697] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from efa1b588222545e18dff850a02f8ec37 (127.12.30.67) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.862259 26707 consensus_queue.cc:237] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.885075 26707 consensus_queue.cc:1048] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.885110 26271 catalog_manager.cc:5697] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.901561 26707 consensus_queue.cc:1048] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:32.909117 26640 raft_consensus.cc:993] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: : Instructing follower 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:32.909644 26640 raft_consensus.cc:1081] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Signalling peer 1d387e18cd424760b5c49569a29c26a6 to start an election
I20260709 13:02:32.916433 26372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b66b819e22154b18ae674dd737f94846"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
 from {username='slave'} at 127.0.0.1:52416
I20260709 13:02:32.917182 26372 raft_consensus.cc:493] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:32.917624 26372 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.932390 26372 raft_consensus.cc:515] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.936581 26372 leader_election.cc:290] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Requested vote from peers 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.937781 26446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:32.938002 26521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "1d387e18cd424760b5c49569a29c26a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.938645 26446 raft_consensus.cc:3055] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.939090 26446 raft_consensus.cc:740] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:32.938723 26521 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.940075 26446 consensus_queue.cc:260] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.941627 26446 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.950181 26521 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:32.951588 26338 leader_election.cc:304] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:32.951719 26446 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d387e18cd424760b5c49569a29c26a6 in term 2.
I20260709 13:02:32.952514 26738 raft_consensus.cc:2804] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:32.953029 26738 raft_consensus.cc:697] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Becoming Leader. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:32.953907 26738 consensus_queue.cc:237] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.965973 26738 raft_consensus.cc:993] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.966607 26739 raft_consensus.cc:1081] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Signalling peer 07f704db827b4614991c7f6e49ae5207 to start an election
I20260709 13:02:32.968603 26446 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8279219bcb70418081b545fd56e267aa"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
 from {username='slave'} at 127.0.0.1:39012
I20260709 13:02:32.969316 26446 raft_consensus.cc:493] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:32.968367 26271 catalog_manager.cc:5697] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 reported cstate change: term changed from 1 to 2, leader changed from 07f704db827b4614991c7f6e49ae5207 (127.12.30.66) to 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "1d387e18cd424760b5c49569a29c26a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:32.969736 26446 raft_consensus.cc:3060] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.982425 26446 raft_consensus.cc:515] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.984789 26446 leader_election.cc:290] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), efa1b588222545e18dff850a02f8ec37 (127.12.30.67:44275)
I20260709 13:02:32.986052 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8279219bcb70418081b545fd56e267aa" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:32.986294 26521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8279219bcb70418081b545fd56e267aa" candidate_uuid: "07f704db827b4614991c7f6e49ae5207" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa1b588222545e18dff850a02f8ec37"
I20260709 13:02:32.987828 26372 raft_consensus.cc:3055] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:32.988502 26372 raft_consensus.cc:740] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:32.988517 26521 raft_consensus.cc:3060] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:32.990072 26372 consensus_queue.cc:260] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:32.993631 26372 raft_consensus.cc:3060] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.001942 26372 raft_consensus.cc:2468] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:33.002199 26521 raft_consensus.cc:2468] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f704db827b4614991c7f6e49ae5207 in term 2.
I20260709 13:02:33.003141 26411 leader_election.cc:304] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, 1d387e18cd424760b5c49569a29c26a6; no voters: 
I20260709 13:02:33.004490 26707 raft_consensus.cc:2804] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:33.005012 26707 raft_consensus.cc:697] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Leader. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:33.006152 26707 consensus_queue.cc:237] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:33.014398 26271 catalog_manager.cc:5697] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 reported cstate change: term changed from 1 to 2, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to 07f704db827b4614991c7f6e49ae5207 (127.12.30.66). New cstate: current_term: 2 leader_uuid: "07f704db827b4614991c7f6e49ae5207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:33.132381 26446 raft_consensus.cc:1275] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.134923 26739 consensus_queue.cc:1048] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.144539 26522 raft_consensus.cc:1275] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.146544 26739 consensus_queue.cc:1048] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.239989 26371 raft_consensus.cc:3060] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.247897 26371 raft_consensus.cc:1275] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.248413 26372 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.250010 26640 consensus_queue.cc:1048] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.263764 26372 raft_consensus.cc:1275] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.269649 26521 raft_consensus.cc:1275] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.271497 26640 consensus_queue.cc:1048] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.272051 26715 consensus_queue.cc:1048] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.298843 26446 raft_consensus.cc:1275] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.299638 26522 raft_consensus.cc:1275] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.301676 26770 consensus_queue.cc:1048] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.302377 26707 consensus_queue.cc:1048] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.305538 26522 raft_consensus.cc:3060] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:33.313436 26520 raft_consensus.cc:1275] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.315486 26739 consensus_queue.cc:1048] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.329440 26522 raft_consensus.cc:1275] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.334563 26640 consensus_queue.cc:1048] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.360455 26372 raft_consensus.cc:1275] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.366894 26640 consensus_queue.cc:1048] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.390188 26372 raft_consensus.cc:1275] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.391669 26776 consensus_queue.cc:1048] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.399142 26522 raft_consensus.cc:1275] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 07f704db827b4614991c7f6e49ae5207: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.401190 26640 consensus_queue.cc:1048] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:33.458010 26448 raft_consensus.cc:1275] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.459789 26770 consensus_queue.cc:1048] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:33.469455 26522 raft_consensus.cc:1275] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Refusing update from remote peer 1d387e18cd424760b5c49569a29c26a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:33.472285 26770 consensus_queue.cc:1048] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:35.588797 26269 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35062:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:02:35.613633 26269 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35062:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:02:35.665246 12409 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:02:35.781562 26337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:34883, user_credentials={real_user=slave}} blocked reactor thread for 47443.7us
W20260709 13:02:35.881542 26808 log.cc:927] Time spent T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:02:35.910398 26798 log.cc:927] Time spent T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:02:35.954787 26815 log.cc:927] Time spent T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:35.968854 26446 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
mode: GRACEFUL
new_leader_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:42142
I20260709 13:02:35.990295 26446 raft_consensus.cc:606] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Received request to transfer leadership to TS efa1b588222545e18dff850a02f8ec37
W20260709 13:02:35.998428 26814 log.cc:927] Time spent T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: Append to log took a long time: real 0.169s	user 0.005s	sys 0.000s
I20260709 13:02:35.997704 26830 tablet.cc:1724] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:36.038772 26828 log.cc:927] Time spent T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: Append to log took a long time: real 0.066s	user 0.002s	sys 0.000s
W20260709 13:02:36.048875 26831 log.cc:927] Time spent T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:02:36.049854 26832 tablet.cc:1724] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:36.042330 26399 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:8279219bcb70418081b545fd56e267aa)
W20260709 13:02:36.057585 26399 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:8279219bcb70418081b545fd56e267aa)
W20260709 13:02:36.064289 26826 log.cc:927] Time spent T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: Append to log took a long time: real 0.150s	user 0.000s	sys 0.004s
I20260709 13:02:36.073017 26833 tablet.cc:1724] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:36.064450 26822 log.cc:927] Time spent T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:02:36.115366 26844 tablet.cc:1724] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:36.089781 26338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.67:44275, user_credentials={real_user=slave}} blocked reactor thread for 46241us
I20260709 13:02:36.051403 26829 tablet.cc:1724] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.111398 26837 tablet.cc:1724] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.101899 26835 tablet.cc:1724] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.123812 26842 tablet.cc:1724] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:36.142457 26824 log.cc:927] Time spent T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: Append to log took a long time: real 0.126s	user 0.000s	sys 0.005s
I20260709 13:02:36.168692 26841 tablet.cc:1724] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.174918 26840 tablet.cc:1724] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:36.174983 26337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:34883, user_credentials={real_user=slave}} blocked reactor thread for 56783.2us
I20260709 13:02:36.188159 26833 tablet.cc:1724] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.193606 26269 catalog_manager.cc:5789] TS 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883) does not have the latest schema for tablet 4cf9803ebb9d44b7addf794d5683468c (table soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]). Expected version 1 got 0
W20260709 13:02:36.199843 26548 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:4b22d31181384271a22afd9139db8879)
I20260709 13:02:36.205243 26269 catalog_manager.cc:5789] TS 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883) does not have the latest schema for tablet 8279219bcb70418081b545fd56e267aa (table soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]). Expected version 1 got 0
I20260709 13:02:36.206182 26269 catalog_manager.cc:5789] TS 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883) does not have the latest schema for tablet 35e5c724781f4a038e42c66c91fd0a6b (table soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]). Expected version 1 got 0
I20260709 13:02:36.214677 26846 tablet.cc:1724] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.217993 26267 catalog_manager.cc:5789] TS 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139) does not have the latest schema for tablet 6f1e63296a834ce8b40da0ca2db516af (table soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]). Expected version 1 got 0
I20260709 13:02:36.219156 26267 catalog_manager.cc:5789] TS 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139) does not have the latest schema for tablet b66b819e22154b18ae674dd737f94846 (table soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559]). Expected version 1 got 0
I20260709 13:02:36.220819 26830 tablet.cc:1724] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.243462 26829 tablet.cc:1724] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.243203 26832 tablet.cc:1724] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.265257 26807 raft_consensus.cc:993] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: : Instructing follower efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:36.266673 26640 raft_consensus.cc:1081] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Signalling peer efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:36.269467 26522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:52156
I20260709 13:02:36.270270 26522 raft_consensus.cc:493] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:36.270668 26522 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.290032 26833 tablet.cc:1724] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.290889 26841 tablet.cc:1724] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.299794 26522 raft_consensus.cc:515] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.305671 26522 leader_election.cc:290] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:36.307539 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:36.308306 26837 tablet.cc:1724] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.312630 26447 raft_consensus.cc:3055] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:36.313930 26447 raft_consensus.cc:740] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:36.312923 26840 tablet.cc:1724] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.314937 26447 consensus_queue.cc:260] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.290850 26846 tablet.cc:1724] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.315984 26447 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.306823 26848 tablet.cc:1724] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.287210 26835 tablet.cc:1724] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.290609 26842 tablet.cc:1724] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.344228 26844 tablet.cc:1724] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.346405 26832 tablet.cc:1724] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.394800 26835 tablet.cc:1724] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.400326 26447 raft_consensus.cc:2468] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:36.404893 26486 leader_election.cc:304] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:36.406203 26823 raft_consensus.cc:2804] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:36.407073 26823 raft_consensus.cc:697] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:36.408205 26823 consensus_queue.cc:237] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.478313 26369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000f44e8c7b641bcbf1a129b4ed9d49e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:36.483804 26369 raft_consensus.cc:3060] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.494042 26372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b66b819e22154b18ae674dd737f94846"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:41284
I20260709 13:02:36.496845 26372 raft_consensus.cc:606] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Received request to transfer leadership to TS efa1b588222545e18dff850a02f8ec37
I20260709 13:02:36.501525 26842 tablet.cc:1724] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:36.511307 12409 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:02:36.514798 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:36.539099 26369 raft_consensus.cc:2468] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:36.554101 26269 catalog_manager.cc:5697] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 2 to 3, leader changed from 07f704db827b4614991c7f6e49ae5207 (127.12.30.66) to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:36.572705 26372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e"
dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
mode: GRACEFUL
new_leader_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:41284
I20260709 13:02:36.573504 26372 raft_consensus.cc:606] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Received request to transfer leadership to TS efa1b588222545e18dff850a02f8ec37
I20260709 13:02:36.577569 26447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1"
dest_uuid: "07f704db827b4614991c7f6e49ae5207"
mode: GRACEFUL
new_leader_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:42142
I20260709 13:02:36.578397 26447 raft_consensus.cc:606] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Received request to transfer leadership to TS efa1b588222545e18dff850a02f8ec37
I20260709 13:02:36.580804 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:36.581692 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:36.590835 26266 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62: soft_deleted_table [id=53e6fc11ce7e4dbeb4878fe5ff0a9559] alter complete (version 1)
I20260709 13:02:36.864213 26640 raft_consensus.cc:993] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207: : Instructing follower efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:36.864724 26640 raft_consensus.cc:1081] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Signalling peer efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:36.866714 26522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:52156
I20260709 13:02:36.867271 26522 raft_consensus.cc:493] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:36.867581 26522 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.875057 26522 raft_consensus.cc:515] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.877058 26522 leader_election.cc:290] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:36.877983 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:36.878253 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4a8280554641faab8f7a597754ebf1" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:36.878871 26372 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.878875 26447 raft_consensus.cc:3055] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:36.879541 26447 raft_consensus.cc:740] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07f704db827b4614991c7f6e49ae5207, State: Running, Role: LEADER
I20260709 13:02:36.880685 26447 consensus_queue.cc:260] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.882344 26447 raft_consensus.cc:3060] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.889122 26372 raft_consensus.cc:2468] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:36.890918 26485 leader_election.cc:304] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d387e18cd424760b5c49569a29c26a6, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:36.891978 26823 raft_consensus.cc:2804] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:36.892539 26823 raft_consensus.cc:697] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:36.893337 26447 raft_consensus.cc:2468] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:36.893685 26823 consensus_queue.cc:237] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.895205 26778 raft_consensus.cc:993] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:36.895859 26777 raft_consensus.cc:1081] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Signalling peer efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:36.897882 26522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b66b819e22154b18ae674dd737f94846"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:52148
I20260709 13:02:36.898644 26522 raft_consensus.cc:493] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:36.899129 26522 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.908910 26522 raft_consensus.cc:515] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.911352 26522 leader_election.cc:290] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:36.912760 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:36.913612 26447 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.912853 26269 catalog_manager.cc:5697] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 2 to 3, leader changed from 07f704db827b4614991c7f6e49ae5207 (127.12.30.66) to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:36.914876 26369 raft_consensus.cc:1275] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:36.916678 26755 consensus_queue.cc:1048] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:36.920817 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66b819e22154b18ae674dd737f94846" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:36.921589 26372 raft_consensus.cc:3055] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:36.921955 26372 raft_consensus.cc:740] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:36.922919 26372 consensus_queue.cc:260] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.924540 26372 raft_consensus.cc:3060] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:36.925408 26447 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:36.926851 26486 leader_election.cc:304] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:36.928900 26823 raft_consensus.cc:2804] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:36.932343 26823 raft_consensus.cc:697] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:36.934111 26823 consensus_queue.cc:237] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:36.941037 26372 raft_consensus.cc:2468] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:36.955200 26447 raft_consensus.cc:1275] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:36.956710 26857 consensus_queue.cc:1048] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:02:36.956670 26267 catalog_manager.cc:5697] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 2 to 3, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.108410 26778 raft_consensus.cc:993] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6: : Instructing follower efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:37.108872 26778 raft_consensus.cc:1081] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Signalling peer efa1b588222545e18dff850a02f8ec37 to start an election
I20260709 13:02:37.110651 26522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e"
dest_uuid: "efa1b588222545e18dff850a02f8ec37"
 from {username='slave'} at 127.0.0.1:52148
I20260709 13:02:37.111253 26522 raft_consensus.cc:493] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.111567 26522 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.118007 26522 raft_consensus.cc:515] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:37.119644 26522 leader_election.cc:290] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Requested vote from peers 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139), 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883)
I20260709 13:02:37.120522 26372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1d387e18cd424760b5c49569a29c26a6"
I20260709 13:02:37.121134 26372 raft_consensus.cc:3055] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:37.121120 26447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6201671c09434607ae6a2fc6e005421e" candidate_uuid: "efa1b588222545e18dff850a02f8ec37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07f704db827b4614991c7f6e49ae5207"
I20260709 13:02:37.121712 26372 raft_consensus.cc:740] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1d387e18cd424760b5c49569a29c26a6, State: Running, Role: LEADER
I20260709 13:02:37.122017 26447 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.122782 26372 consensus_queue.cc:260] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:37.124151 26372 raft_consensus.cc:3060] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:37.128192 26447 raft_consensus.cc:2468] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:37.129483 26372 raft_consensus.cc:2468] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa1b588222545e18dff850a02f8ec37 in term 3.
I20260709 13:02:37.130254 26486 leader_election.cc:304] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f704db827b4614991c7f6e49ae5207, efa1b588222545e18dff850a02f8ec37; no voters: 
I20260709 13:02:37.131153 26857 raft_consensus.cc:2804] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:37.131678 26857 raft_consensus.cc:697] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Becoming Leader. State: Replica: efa1b588222545e18dff850a02f8ec37, State: Running, Role: LEADER
I20260709 13:02:37.132673 26857 consensus_queue.cc:237] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } }
I20260709 13:02:37.141290 26267 catalog_manager.cc:5697] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 reported cstate change: term changed from 2 to 3, leader changed from 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65) to efa1b588222545e18dff850a02f8ec37 (127.12.30.67). New cstate: current_term: 3 leader_uuid: "efa1b588222545e18dff850a02f8ec37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa1b588222545e18dff850a02f8ec37" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.348197 26447 raft_consensus.cc:1275] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:37.349627 26857 consensus_queue.cc:1048] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:37.360532 26372 raft_consensus.cc:1275] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:02:37.363018 26857 consensus_queue.cc:1048] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:37.371985 26447 raft_consensus.cc:1275] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:37.373569 26858 consensus_queue.cc:1048] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:37.383651 26369 raft_consensus.cc:1275] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:37.384888 26858 consensus_queue.cc:1048] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:37.612874 26369 raft_consensus.cc:1275] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:37.614552 26823 consensus_queue.cc:1048] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d387e18cd424760b5c49569a29c26a6" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:37.629340 26447 raft_consensus.cc:1275] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Refusing update from remote peer efa1b588222545e18dff850a02f8ec37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:37.639319 26857 consensus_queue.cc:1048] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f704db827b4614991c7f6e49ae5207" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:37.675230 12409 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:02:37.680720 12409 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:02:37.681190 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:37.686157 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:02:37.686595 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:38.728659 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
W20260709 13:02:38.735883 26411 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
W20260709 13:02:38.745816 26411 consensus_peers.cc:597] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:38.751441 26411 consensus_peers.cc:597] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:38.783699 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:38.785023 12409 tablet_replica.cc:333] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.786270 12409 raft_consensus.cc:2243] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.787297 12409 raft_consensus.cc:2272] T 4cf9803ebb9d44b7addf794d5683468c P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.792695 12409 tablet_replica.cc:333] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.793751 12409 raft_consensus.cc:2243] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.794350 12409 raft_consensus.cc:2272] T 000f44e8c7b641bcbf1a129b4ed9d49e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:38.794270 26411 consensus_peers.cc:597] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:38.797487 12409 tablet_replica.cc:333] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.798511 12409 raft_consensus.cc:2243] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.799320 12409 raft_consensus.cc:2272] T 4b22d31181384271a22afd9139db8879 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.803519 12409 tablet_replica.cc:333] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.804328 12409 raft_consensus.cc:2243] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.805157 12409 raft_consensus.cc:2272] T 35e5c724781f4a038e42c66c91fd0a6b P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.808795 12409 tablet_replica.cc:333] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
W20260709 13:02:38.809623 26485 consensus_peers.cc:597] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:38.809937 12409 raft_consensus.cc:2243] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.810752 12409 raft_consensus.cc:2272] T 8279219bcb70418081b545fd56e267aa P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:38.813488 26411 consensus_peers.cc:597] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:38.814385 12409 tablet_replica.cc:333] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.815482 12409 raft_consensus.cc:2243] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:38.816536 12409 raft_consensus.cc:2272] T 6b61a7f8106b49608bcbe252db84ba1b P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.819265 12409 tablet_replica.cc:333] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.820071 12409 raft_consensus.cc:2243] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:38.821314 12409 raft_consensus.cc:2272] T b14217c326114ae4b16d142c842aab60 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.825481 12409 tablet_replica.cc:333] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.826583 12409 raft_consensus.cc:2243] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:38.827963 12409 raft_consensus.cc:2272] T 6f1e63296a834ce8b40da0ca2db516af P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.831908 12409 tablet_replica.cc:333] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.832751 12409 raft_consensus.cc:2243] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.833249 12409 raft_consensus.cc:2272] T 6201671c09434607ae6a2fc6e005421e P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.836668 12409 tablet_replica.cc:333] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.838141 12409 raft_consensus.cc:2243] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.838770 12409 raft_consensus.cc:2272] T bb4a8280554641faab8f7a597754ebf1 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.841811 12409 tablet_replica.cc:333] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.842664 12409 raft_consensus.cc:2243] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.843348 12409 raft_consensus.cc:2272] T 2c2f11d94f7f45f8982152cdfa656466 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.846913 12409 tablet_replica.cc:333] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.847759 12409 raft_consensus.cc:2243] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.848539 12409 raft_consensus.cc:2272] T ed59cec85356493cae6a495fbfb2d404 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.851213 12409 tablet_replica.cc:333] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.851776 12409 raft_consensus.cc:2243] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:38.852800 12409 raft_consensus.cc:2272] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.856139 12409 tablet_replica.cc:333] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.856925 12409 raft_consensus.cc:2243] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.857481 12409 raft_consensus.cc:2272] T b66b819e22154b18ae674dd737f94846 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.860423 12409 tablet_replica.cc:333] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.861136 12409 raft_consensus.cc:2243] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.861888 12409 raft_consensus.cc:2272] T d349544786e74ca88edd37fc4a36bd90 P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.865023 12409 tablet_replica.cc:333] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.865713 12409 raft_consensus.cc:2243] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:38.866742 12409 raft_consensus.cc:2272] T 09955d4a1d6843a6a2c528d4b4263374 P 1d387e18cd424760b5c49569a29c26a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.869577 12409 tablet_replica.cc:333] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.870316 12409 raft_consensus.cc:2243] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:38.871121 12409 raft_consensus.cc:2272] T 86e2049c522743d984904f37d2bff5cb P 1d387e18cd424760b5c49569a29c26a6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:38.874812 12409 tablet_replica.cc:333] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6: stopping tablet replica
I20260709 13:02:38.875469 12409 raft_consensus.cc:2243] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:02:38.875872 26485 consensus_peers.cc:597] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:38.876766 12409 raft_consensus.cc:2272] T 903a21f3964a4311a2109b046548c563 P 1d387e18cd424760b5c49569a29c26a6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:38.926095 26485 consensus_peers.cc:597] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:36139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:38.932586 26485 consensus_peers.cc:597] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:36139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:38.935335 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
W20260709 13:02:38.943877 26485 consensus_peers.cc:597] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:36139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:38.995710 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
W20260709 13:02:39.005863 26411 consensus_peers.cc:597] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:36139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:39.026255 26485 consensus_peers.cc:597] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 -> Peer 1d387e18cd424760b5c49569a29c26a6 (127.12.30.65:36139): Couldn't send request to peer 1d387e18cd424760b5c49569a29c26a6. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:36139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:39.044356 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:39.045315 12409 tablet_replica.cc:333] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.046418 12409 raft_consensus.cc:2243] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.047693 12409 raft_consensus.cc:2272] T 4cf9803ebb9d44b7addf794d5683468c P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.050365 12409 tablet_replica.cc:333] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.051239 12409 raft_consensus.cc:2243] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.052030 12409 raft_consensus.cc:2272] T 4b22d31181384271a22afd9139db8879 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.054986 12409 tablet_replica.cc:333] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.055778 12409 raft_consensus.cc:2243] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:02:39.055610 26486 consensus_peers.cc:597] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 -> Peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Couldn't send request to peer 07f704db827b4614991c7f6e49ae5207. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:39.056560 12409 raft_consensus.cc:2272] T 000f44e8c7b641bcbf1a129b4ed9d49e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.059226 12409 tablet_replica.cc:333] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.059897 12409 raft_consensus.cc:2243] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.060910 12409 raft_consensus.cc:2272] T 35e5c724781f4a038e42c66c91fd0a6b P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.063534 12409 tablet_replica.cc:333] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.064591 12409 raft_consensus.cc:2243] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.065876 12409 raft_consensus.cc:2272] T 8279219bcb70418081b545fd56e267aa P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:39.067839 26486 consensus_peers.cc:597] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 -> Peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Couldn't send request to peer 07f704db827b4614991c7f6e49ae5207. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:39.069908 12409 tablet_replica.cc:333] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.070883 12409 raft_consensus.cc:2243] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.071465 12409 raft_consensus.cc:2272] T 6b61a7f8106b49608bcbe252db84ba1b P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:39.072367 26486 consensus_peers.cc:597] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 -> Peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Couldn't send request to peer 07f704db827b4614991c7f6e49ae5207. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:39.074368 12409 tablet_replica.cc:333] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.075163 12409 raft_consensus.cc:2243] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.075745 12409 raft_consensus.cc:2272] T 6f1e63296a834ce8b40da0ca2db516af P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.078217 12409 tablet_replica.cc:333] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.078903 12409 raft_consensus.cc:2243] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.079491 12409 raft_consensus.cc:2272] T b14217c326114ae4b16d142c842aab60 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.081972 12409 tablet_replica.cc:333] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.082670 12409 raft_consensus.cc:2243] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.083212 12409 raft_consensus.cc:2272] T bb4a8280554641faab8f7a597754ebf1 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.085701 12409 tablet_replica.cc:333] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.086355 12409 raft_consensus.cc:2243] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.086848 12409 raft_consensus.cc:2272] T 6201671c09434607ae6a2fc6e005421e P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.089371 12409 tablet_replica.cc:333] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.090086 12409 raft_consensus.cc:2243] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.091092 12409 raft_consensus.cc:2272] T ed59cec85356493cae6a495fbfb2d404 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.093737 12409 tablet_replica.cc:333] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.094645 12409 raft_consensus.cc:2243] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.095184 12409 raft_consensus.cc:2272] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.097806 12409 tablet_replica.cc:333] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.098423 12409 raft_consensus.cc:2243] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.098889 12409 raft_consensus.cc:2272] T 903a21f3964a4311a2109b046548c563 P 07f704db827b4614991c7f6e49ae5207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.102236 12409 tablet_replica.cc:333] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.102974 12409 raft_consensus.cc:2243] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.103726 12409 raft_consensus.cc:2272] T b66b819e22154b18ae674dd737f94846 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.106885 12409 tablet_replica.cc:333] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.107574 12409 raft_consensus.cc:2243] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.108960 12409 raft_consensus.cc:2272] T d349544786e74ca88edd37fc4a36bd90 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.111899 12409 tablet_replica.cc:333] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.112629 12409 raft_consensus.cc:2243] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.113586 12409 raft_consensus.cc:2272] T 86e2049c522743d984904f37d2bff5cb P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.117080 12409 tablet_replica.cc:333] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.117616 12409 raft_consensus.cc:2243] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.118129 12409 raft_consensus.cc:2272] T 09955d4a1d6843a6a2c528d4b4263374 P 07f704db827b4614991c7f6e49ae5207 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.120523 12409 tablet_replica.cc:333] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207: stopping tablet replica
I20260709 13:02:39.120988 12409 raft_consensus.cc:2243] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.121934 12409 raft_consensus.cc:2272] T 2c2f11d94f7f45f8982152cdfa656466 P 07f704db827b4614991c7f6e49ae5207 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.181478 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
I20260709 13:02:39.246701 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
W20260709 13:02:39.317909 26486 consensus_peers.cc:597] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 -> Peer 07f704db827b4614991c7f6e49ae5207 (127.12.30.66:34883): Couldn't send request to peer 07f704db827b4614991c7f6e49ae5207. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:34883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:39.324633 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:39.325663 12409 tablet_replica.cc:333] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.326926 12409 raft_consensus.cc:2243] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.327757 12409 raft_consensus.cc:2272] T 35e5c724781f4a038e42c66c91fd0a6b P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.330567 12409 tablet_replica.cc:333] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.331431 12409 raft_consensus.cc:2243] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.332180 12409 raft_consensus.cc:2272] T 4cf9803ebb9d44b7addf794d5683468c P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.336761 12409 tablet_replica.cc:333] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.340593 12409 raft_consensus.cc:2243] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.342329 12409 raft_consensus.cc:2272] T 000f44e8c7b641bcbf1a129b4ed9d49e P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.346247 12409 tablet_replica.cc:333] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.347084 12409 raft_consensus.cc:2243] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.348557 12409 raft_consensus.cc:2272] T 4b22d31181384271a22afd9139db8879 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.352789 12409 tablet_replica.cc:333] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.353511 12409 raft_consensus.cc:2243] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.355057 12409 raft_consensus.cc:2272] T 2c2f11d94f7f45f8982152cdfa656466 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.358148 12409 tablet_replica.cc:333] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.358883 12409 raft_consensus.cc:2243] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.359495 12409 raft_consensus.cc:2272] T 8279219bcb70418081b545fd56e267aa P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.363009 12409 tablet_replica.cc:333] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.363708 12409 raft_consensus.cc:2243] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.364358 12409 raft_consensus.cc:2272] T 6b61a7f8106b49608bcbe252db84ba1b P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.367681 12409 tablet_replica.cc:333] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.368861 12409 raft_consensus.cc:2243] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.370754 12409 raft_consensus.cc:2272] T b66b819e22154b18ae674dd737f94846 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.373935 12409 tablet_replica.cc:333] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.374771 12409 raft_consensus.cc:2243] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.375381 12409 raft_consensus.cc:2272] T d349544786e74ca88edd37fc4a36bd90 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.378157 12409 tablet_replica.cc:333] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.378968 12409 raft_consensus.cc:2243] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.379518 12409 raft_consensus.cc:2272] T 86e2049c522743d984904f37d2bff5cb P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.382517 12409 tablet_replica.cc:333] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.383533 12409 raft_consensus.cc:2243] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.384145 12409 raft_consensus.cc:2272] T 09955d4a1d6843a6a2c528d4b4263374 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.386693 12409 tablet_replica.cc:333] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.387467 12409 raft_consensus.cc:2243] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.388091 12409 raft_consensus.cc:2272] T 903a21f3964a4311a2109b046548c563 P efa1b588222545e18dff850a02f8ec37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.390650 12409 tablet_replica.cc:333] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.391546 12409 raft_consensus.cc:2243] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.392264 12409 raft_consensus.cc:2272] T 6f1e63296a834ce8b40da0ca2db516af P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.394917 12409 tablet_replica.cc:333] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.395533 12409 raft_consensus.cc:2243] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.396402 12409 raft_consensus.cc:2272] T b14217c326114ae4b16d142c842aab60 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.399861 12409 tablet_replica.cc:333] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.401396 12409 raft_consensus.cc:2243] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.402868 12409 raft_consensus.cc:2272] T bb4a8280554641faab8f7a597754ebf1 P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.406009 12409 tablet_replica.cc:333] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.406757 12409 raft_consensus.cc:2243] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.408094 12409 raft_consensus.cc:2272] T 6201671c09434607ae6a2fc6e005421e P efa1b588222545e18dff850a02f8ec37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.411114 12409 tablet_replica.cc:333] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.411782 12409 raft_consensus.cc:2243] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.412448 12409 raft_consensus.cc:2272] T ed59cec85356493cae6a495fbfb2d404 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.415371 12409 tablet_replica.cc:333] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37: stopping tablet replica
I20260709 13:02:39.416113 12409 raft_consensus.cc:2243] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:39.416699 12409 raft_consensus.cc:2272] T 2d70bd89a6f7403c9fa79bf2ed68a6c3 P efa1b588222545e18dff850a02f8ec37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.474503 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
I20260709 13:02:39.524243 12409 master.cc:562] Master@127.12.30.126:42727 shutting down...
I20260709 13:02:39.552795 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:39.553675 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:39.554104 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1a749a25ad9c41bda919d20db95b9c62: stopping tablet replica
I20260709 13:02:39.582345 12409 master.cc:584] Master@127.12.30.126:42727 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (14906 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:02:39.664171 12409 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.30.126:37307
I20260709 13:02:39.665541 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:39.672523 26875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:39.673463 26876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:39.675419 26878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:39.675577 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:39.677206 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:39.677457 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:39.677636 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602159677619 us; error 0 us; skew 500 ppm
I20260709 13:02:39.678169 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:39.681137 12409 webserver.cc:533] Webserver started at http://127.12.30.126:45723/ using document root <none> and password file <none>
I20260709 13:02:39.681829 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:39.682060 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:39.682358 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:39.683815 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/master-0-root/instance:
uuid: "ec147c9ae47b47f7a83895db40063882"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-mlt7"
I20260709 13:02:39.688721 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:02:39.692488 26883 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:39.693440 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:02:39.693883 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/master-0-root
uuid: "ec147c9ae47b47f7a83895db40063882"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-mlt7"
I20260709 13:02:39.694204 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:39.710706 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:39.712150 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:39.754104 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.126:37307
I20260709 13:02:39.754215 26934 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.126:37307 every 8 connection(s)
I20260709 13:02:39.758675 26935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:39.771239 26935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882: Bootstrap starting.
I20260709 13:02:39.776446 26935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:39.781286 26935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882: No bootstrap required, opened a new log
I20260709 13:02:39.783712 26935 raft_consensus.cc:359] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec147c9ae47b47f7a83895db40063882" member_type: VOTER }
I20260709 13:02:39.784179 26935 raft_consensus.cc:385] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:39.784397 26935 raft_consensus.cc:740] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec147c9ae47b47f7a83895db40063882, State: Initialized, Role: FOLLOWER
I20260709 13:02:39.784996 26935 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec147c9ae47b47f7a83895db40063882" member_type: VOTER }
I20260709 13:02:39.785468 26935 raft_consensus.cc:399] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:02:39.785681 26935 raft_consensus.cc:493] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:02:39.785985 26935 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:39.791079 26935 raft_consensus.cc:515] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec147c9ae47b47f7a83895db40063882" member_type: VOTER }
I20260709 13:02:39.791626 26935 leader_election.cc:304] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec147c9ae47b47f7a83895db40063882; no voters: 
I20260709 13:02:39.792903 26935 leader_election.cc:290] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:02:39.793292 26938 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:39.795077 26938 raft_consensus.cc:697] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 1 LEADER]: Becoming Leader. State: Replica: ec147c9ae47b47f7a83895db40063882, State: Running, Role: LEADER
I20260709 13:02:39.795774 26938 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec147c9ae47b47f7a83895db40063882" member_type: VOTER }
I20260709 13:02:39.796820 26935 sys_catalog.cc:565] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:02:39.800354 26939 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec147c9ae47b47f7a83895db40063882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec147c9ae47b47f7a83895db40063882" member_type: VOTER } }
I20260709 13:02:39.801455 26940 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec147c9ae47b47f7a83895db40063882. Latest consensus state: current_term: 1 leader_uuid: "ec147c9ae47b47f7a83895db40063882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec147c9ae47b47f7a83895db40063882" member_type: VOTER } }
I20260709 13:02:39.802316 26939 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:39.802415 26940 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:39.811591 26946 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:02:39.820185 26946 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:02:39.821560 12409 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:02:39.830379 26946 catalog_manager.cc:1383] Generated new cluster ID: bd44155246ac4c72a1c296d0d38c9cdb
I20260709 13:02:39.830708 26946 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:02:39.848251 26946 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:02:39.849738 26946 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:02:39.865609 26946 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882: Generated new TSK 0
I20260709 13:02:39.866293 26946 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:02:39.888466 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:39.894976 26956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:39.896584 26957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:39.898191 26959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:39.898701 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:39.899793 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:39.900082 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:39.900260 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602159900244 us; error 0 us; skew 500 ppm
I20260709 13:02:39.900904 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:39.904521 12409 webserver.cc:533] Webserver started at http://127.12.30.65:37549/ using document root <none> and password file <none>
I20260709 13:02:39.905158 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:39.905361 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:39.905654 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:39.906872 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-0-root/instance:
uuid: "183d64694f2647e1b0c710360b97d497"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-mlt7"
I20260709 13:02:39.911978 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:02:39.915892 26964 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:39.916887 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:02:39.917165 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-0-root
uuid: "183d64694f2647e1b0c710360b97d497"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-mlt7"
I20260709 13:02:39.917414 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:39.962278 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:39.963562 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:39.965193 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:39.967696 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:39.967926 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:39.968243 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:39.968432 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.014536 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.65:35259
I20260709 13:02:40.014631 27026 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.65:35259 every 8 connection(s)
I20260709 13:02:40.021744 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:40.031255 27031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:40.032603 27032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:40.041388 12409 server_base.cc:1061] running on GCE node
W20260709 13:02:40.043403 27034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:40.044962 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:40.045326 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:40.045531 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602160045508 us; error 0 us; skew 500 ppm
I20260709 13:02:40.045889 27027 heartbeater.cc:344] Connected to a master server at 127.12.30.126:37307
I20260709 13:02:40.046489 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:40.046619 27027 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:40.048113 27027 heartbeater.cc:507] Master 127.12.30.126:37307 requested a full tablet report, sending...
I20260709 13:02:40.050702 12409 webserver.cc:533] Webserver started at http://127.12.30.66:38213/ using document root <none> and password file <none>
I20260709 13:02:40.051410 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:40.051584 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:40.052242 26900 ts_manager.cc:194] Registered new tserver with Master: 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259)
I20260709 13:02:40.052575 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:40.054739 26900 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47458
I20260709 13:02:40.054804 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-1-root/instance:
uuid: "948af4c6da234b748ca8952d7b81322e"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-mlt7"
I20260709 13:02:40.062014 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.009s	sys 0.000s
I20260709 13:02:40.067714 27039 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.069463 12409 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:02:40.069962 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-1-root
uuid: "948af4c6da234b748ca8952d7b81322e"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-mlt7"
I20260709 13:02:40.070327 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:40.088447 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:40.089886 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:40.091557 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:40.094861 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:40.095124 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.095423 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:40.095641 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.142534 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.66:45595
I20260709 13:02:40.142675 27101 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.66:45595 every 8 connection(s)
I20260709 13:02:40.148619 12409 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:40.158017 27105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:40.161041 27106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:40.161588 27108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:40.163504 12409 server_base.cc:1061] running on GCE node
I20260709 13:02:40.164440 27102 heartbeater.cc:344] Connected to a master server at 127.12.30.126:37307
I20260709 13:02:40.164820 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:02:40.164911 27102 heartbeater.cc:461] Registering TS with master...
W20260709 13:02:40.165175 12409 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:40.165438 12409 hybrid_clock.cc:648] HybridClock initialized: now 1783602160165414 us; error 0 us; skew 500 ppm
I20260709 13:02:40.165939 27102 heartbeater.cc:507] Master 127.12.30.126:37307 requested a full tablet report, sending...
I20260709 13:02:40.166111 12409 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:40.168447 26900 ts_manager.cc:194] Registered new tserver with Master: 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:40.169435 12409 webserver.cc:533] Webserver started at http://127.12.30.67:41605/ using document root <none> and password file <none>
I20260709 13:02:40.170099 12409 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:40.170033 26900 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47474
I20260709 13:02:40.170614 12409 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:40.171051 12409 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:40.172523 12409 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-2-root/instance:
uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-mlt7"
I20260709 13:02:40.177281 12409 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:02:40.181099 27113 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.182133 12409 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:02:40.182415 12409 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-2-root
uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-mlt7"
I20260709 13:02:40.182943 12409 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPCA3qi/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601726996059-12409-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:40.201452 12409 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:40.202670 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:40.204039 12409 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:40.206380 12409 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:40.206563 12409 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.206844 12409 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:40.207044 12409 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.252322 12409 rpc_server.cc:307] RPC server started. Bound to: 127.12.30.67:37099
I20260709 13:02:40.252454 27175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.30.67:37099 every 8 connection(s)
I20260709 13:02:40.269822 27176 heartbeater.cc:344] Connected to a master server at 127.12.30.126:37307
I20260709 13:02:40.270341 27176 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:40.271080 27176 heartbeater.cc:507] Master 127.12.30.126:37307 requested a full tablet report, sending...
I20260709 13:02:40.273665 26900 ts_manager.cc:194] Registered new tserver with Master: 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:40.274832 12409 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019025986s
I20260709 13:02:40.275490 26900 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47484
I20260709 13:02:40.304764 26900 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47500:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:40.310176 26900 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:40.382256 27141 tablet_service.cc:1511] Processing CreateTablet for tablet f4cc5719ad354c1f897af009ef1e528f (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:40.383797 27141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4cc5719ad354c1f897af009ef1e528f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.385239 27067 tablet_service.cc:1511] Processing CreateTablet for tablet f4cc5719ad354c1f897af009ef1e528f (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:40.386755 27067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4cc5719ad354c1f897af009ef1e528f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.388033 27140 tablet_service.cc:1511] Processing CreateTablet for tablet 486278766c1e42e6908c196fff181e29 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:40.389022 27066 tablet_service.cc:1511] Processing CreateTablet for tablet 486278766c1e42e6908c196fff181e29 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:40.389649 27140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 486278766c1e42e6908c196fff181e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.390425 27066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 486278766c1e42e6908c196fff181e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.393417 27065 tablet_service.cc:1511] Processing CreateTablet for tablet a180fcba65e243f2ace7ab30c1128475 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:40.397006 27065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a180fcba65e243f2ace7ab30c1128475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.401842 26992 tablet_service.cc:1511] Processing CreateTablet for tablet f4cc5719ad354c1f897af009ef1e528f (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:40.403366 26992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4cc5719ad354c1f897af009ef1e528f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.409078 26991 tablet_service.cc:1511] Processing CreateTablet for tablet 486278766c1e42e6908c196fff181e29 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:40.410542 26991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 486278766c1e42e6908c196fff181e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.413096 26990 tablet_service.cc:1511] Processing CreateTablet for tablet a180fcba65e243f2ace7ab30c1128475 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:40.414552 26990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a180fcba65e243f2ace7ab30c1128475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.423849 27139 tablet_service.cc:1511] Processing CreateTablet for tablet a180fcba65e243f2ace7ab30c1128475 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:40.425472 27139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a180fcba65e243f2ace7ab30c1128475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.428092 26989 tablet_service.cc:1511] Processing CreateTablet for tablet ebc2ac862d0a4a2184e9a3c18a7264a2 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:40.429702 26989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc2ac862d0a4a2184e9a3c18a7264a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.435539 27064 tablet_service.cc:1511] Processing CreateTablet for tablet ebc2ac862d0a4a2184e9a3c18a7264a2 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:40.437163 27064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc2ac862d0a4a2184e9a3c18a7264a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.440618 26988 tablet_service.cc:1511] Processing CreateTablet for tablet d2dfd29191814d62bb243bd4be7fdba2 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:40.442101 26988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2dfd29191814d62bb243bd4be7fdba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.446911 27138 tablet_service.cc:1511] Processing CreateTablet for tablet ebc2ac862d0a4a2184e9a3c18a7264a2 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:40.448478 27138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc2ac862d0a4a2184e9a3c18a7264a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.449669 26987 tablet_service.cc:1511] Processing CreateTablet for tablet 194f838ca919474c8162e2def0e3da14 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:40.451100 26987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194f838ca919474c8162e2def0e3da14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.454283 27137 tablet_service.cc:1511] Processing CreateTablet for tablet d2dfd29191814d62bb243bd4be7fdba2 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:40.457417 26986 tablet_service.cc:1511] Processing CreateTablet for tablet 43741078ec4b4b40bffbb89360aea0c3 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:40.459000 26986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43741078ec4b4b40bffbb89360aea0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.462982 27067 tablet_service.cc:1511] Processing CreateTablet for tablet 194f838ca919474c8162e2def0e3da14 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:40.464504 27067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194f838ca919474c8162e2def0e3da14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.469247 26985 tablet_service.cc:1511] Processing CreateTablet for tablet c4efc5a1ab5e4577a3ca32495d6e4b04 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:40.471354 26985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4efc5a1ab5e4577a3ca32495d6e4b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.475265 26984 tablet_service.cc:1511] Processing CreateTablet for tablet 7e4e0210defe44639398856436eb821c (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:40.477164 26984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e4e0210defe44639398856436eb821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.469141 27063 tablet_service.cc:1511] Processing CreateTablet for tablet 43741078ec4b4b40bffbb89360aea0c3 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:40.479038 27136 tablet_service.cc:1511] Processing CreateTablet for tablet 194f838ca919474c8162e2def0e3da14 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:40.480340 27063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43741078ec4b4b40bffbb89360aea0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.482152 27133 tablet_service.cc:1511] Processing CreateTablet for tablet 7e4e0210defe44639398856436eb821c (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:40.479874 27134 tablet_service.cc:1511] Processing CreateTablet for tablet c4efc5a1ab5e4577a3ca32495d6e4b04 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:40.485069 27065 tablet_service.cc:1511] Processing CreateTablet for tablet d2dfd29191814d62bb243bd4be7fdba2 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:40.485427 27135 tablet_service.cc:1511] Processing CreateTablet for tablet 43741078ec4b4b40bffbb89360aea0c3 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:40.486415 27065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2dfd29191814d62bb243bd4be7fdba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.455794 27137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2dfd29191814d62bb243bd4be7fdba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.489351 27136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194f838ca919474c8162e2def0e3da14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.490458 27135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43741078ec4b4b40bffbb89360aea0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.492327 27134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4efc5a1ab5e4577a3ca32495d6e4b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.492452 27062 tablet_service.cc:1511] Processing CreateTablet for tablet c4efc5a1ab5e4577a3ca32495d6e4b04 (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:40.493621 27133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e4e0210defe44639398856436eb821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.494035 27062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4efc5a1ab5e4577a3ca32495d6e4b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.497308 27061 tablet_service.cc:1511] Processing CreateTablet for tablet 7e4e0210defe44639398856436eb821c (DEFAULT_TABLE table=test-workload [id=a650d702217444038bf5f84c4e732628]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:40.498826 27061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e4e0210defe44639398856436eb821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.553946 27192 tablet_bootstrap.cc:492] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:40.645220 27192 tablet_bootstrap.cc:654] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.649668 27191 tablet_bootstrap.cc:492] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:40.656635 27193 tablet_bootstrap.cc:492] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:40.665408 27193 tablet_bootstrap.cc:654] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.676259 27192 tablet_bootstrap.cc:492] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:40.676908 27192 ts_tablet_manager.cc:1403] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.123s	user 0.023s	sys 0.005s
I20260709 13:02:40.677774 27191 tablet_bootstrap.cc:654] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.679610 27192 raft_consensus.cc:359] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.680363 27192 raft_consensus.cc:385] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.680717 27192 raft_consensus.cc:740] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.681515 27192 consensus_queue.cc:260] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.682710 27193 tablet_bootstrap.cc:492] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:40.683228 27193 ts_tablet_manager.cc:1403] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 13:02:40.684381 27191 tablet_bootstrap.cc:492] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:40.684576 27192 ts_tablet_manager.cc:1434] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:02:40.685058 27191 ts_tablet_manager.cc:1403] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.036s	user 0.004s	sys 0.008s
I20260709 13:02:40.685633 27192 tablet_bootstrap.cc:492] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:40.685102 27176 heartbeater.cc:499] Master 127.12.30.126:37307 was elected leader, sending a full tablet report...
I20260709 13:02:40.686555 27193 raft_consensus.cc:359] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.687368 27193 raft_consensus.cc:385] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.687731 27193 raft_consensus.cc:740] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.688542 27193 consensus_queue.cc:260] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.688956 27191 raft_consensus.cc:359] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.689983 27191 raft_consensus.cc:385] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.690354 27191 raft_consensus.cc:740] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.691419 27191 consensus_queue.cc:260] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.693567 27192 tablet_bootstrap.cc:654] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.713943 27191 ts_tablet_manager.cc:1434] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.028s	user 0.025s	sys 0.001s
I20260709 13:02:40.715751 27027 heartbeater.cc:499] Master 127.12.30.126:37307 was elected leader, sending a full tablet report...
I20260709 13:02:40.716180 27192 tablet_bootstrap.cc:492] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:40.716308 27193 ts_tablet_manager.cc:1434] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.033s	user 0.017s	sys 0.017s
I20260709 13:02:40.716892 27192 ts_tablet_manager.cc:1403] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.012s
I20260709 13:02:40.717607 27193 tablet_bootstrap.cc:492] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:40.719705 27102 heartbeater.cc:499] Master 127.12.30.126:37307 was elected leader, sending a full tablet report...
I20260709 13:02:40.720600 27192 raft_consensus.cc:359] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.721236 27192 raft_consensus.cc:385] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.721535 27192 raft_consensus.cc:740] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.722472 27192 consensus_queue.cc:260] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.725023 27192 ts_tablet_manager.cc:1434] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:40.725584 27191 tablet_bootstrap.cc:492] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:40.726013 27192 tablet_bootstrap.cc:492] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:40.727557 27193 tablet_bootstrap.cc:654] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.732556 27197 raft_consensus.cc:493] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.733172 27197 raft_consensus.cc:515] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.736763 27191 tablet_bootstrap.cc:654] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.742193 27197 leader_election.cc:290] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:40.749246 27192 tablet_bootstrap.cc:654] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.755044 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
W20260709 13:02:40.756944 27177 tablet.cc:2406] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260709 13:02:40.758260 27117 leader_election.cc:343] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:40.764190 27192 tablet_bootstrap.cc:492] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:40.764513 27193 tablet_bootstrap.cc:492] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:40.764884 27192 ts_tablet_manager.cc:1403] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.004s
I20260709 13:02:40.765059 27193 ts_tablet_manager.cc:1403] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.007s
I20260709 13:02:40.765375 27191 tablet_bootstrap.cc:492] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:40.766048 27191 ts_tablet_manager.cc:1403] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.041s	user 0.026s	sys 0.010s
I20260709 13:02:40.768205 27192 raft_consensus.cc:359] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.768388 27193 raft_consensus.cc:359] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.769299 27193 raft_consensus.cc:385] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.769569 27192 raft_consensus.cc:385] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.769680 27193 raft_consensus.cc:740] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.769574 27191 raft_consensus.cc:359] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.771795 27191 raft_consensus.cc:385] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.771492 27193 consensus_queue.cc:260] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.769945 27192 raft_consensus.cc:740] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.772181 27191 raft_consensus.cc:740] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.773078 27192 consensus_queue.cc:260] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.773147 27191 consensus_queue.cc:260] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.775439 27193 ts_tablet_manager.cc:1434] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:02:40.775439 27191 ts_tablet_manager.cc:1434] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.005s
I20260709 13:02:40.778434 27193 tablet_bootstrap.cc:492] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:40.777338 27192 ts_tablet_manager.cc:1434] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.012s	user 0.000s	sys 0.006s
I20260709 13:02:40.779554 27192 tablet_bootstrap.cc:492] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:40.784204 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:40.785133 27077 raft_consensus.cc:2468] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:40.778647 27191 tablet_bootstrap.cc:492] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:40.785971 27193 tablet_bootstrap.cc:654] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.785969 27192 tablet_bootstrap.cc:654] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.786859 27115 leader_election.cc:304] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:40.788329 27197 raft_consensus.cc:2804] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:40.788782 27197 raft_consensus.cc:493] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.789158 27197 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.799369 27197 raft_consensus.cc:515] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.802802 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:40.803032 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:40.803661 27077 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:40.804591 27117 leader_election.cc:343] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:40.808861 26948 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:40.812029 27197 leader_election.cc:290] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:40.813120 27077 raft_consensus.cc:2468] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:40.814548 27115 leader_election.cc:304] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:40.815867 27197 raft_consensus.cc:2804] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:40.820104 27197 raft_consensus.cc:697] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:40.821846 27191 tablet_bootstrap.cc:654] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.821450 27197 consensus_queue.cc:237] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.822366 27193 tablet_bootstrap.cc:492] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:40.823076 27193 ts_tablet_manager.cc:1403] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.012s
I20260709 13:02:40.828253 27192 tablet_bootstrap.cc:492] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:40.828897 27192 ts_tablet_manager.cc:1403] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.029s
I20260709 13:02:40.828697 27193 raft_consensus.cc:359] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.829677 27193 raft_consensus.cc:385] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.830080 27193 raft_consensus.cc:740] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.830976 27193 consensus_queue.cc:260] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.831746 27191 tablet_bootstrap.cc:492] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:40.832310 27191 ts_tablet_manager.cc:1403] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.004s
I20260709 13:02:40.832526 27192 raft_consensus.cc:359] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.833271 27192 raft_consensus.cc:385] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.833287 27193 ts_tablet_manager.cc:1434] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 13:02:40.833683 27192 raft_consensus.cc:740] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.834612 27193 tablet_bootstrap.cc:492] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:40.834582 27192 consensus_queue.cc:260] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.837455 27192 ts_tablet_manager.cc:1434] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:40.838551 27192 tablet_bootstrap.cc:492] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:40.843044 27193 tablet_bootstrap.cc:654] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.835366 27191 raft_consensus.cc:359] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.843947 27191 raft_consensus.cc:385] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.844321 27191 raft_consensus.cc:740] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.845486 27192 tablet_bootstrap.cc:654] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.845163 27191 consensus_queue.cc:260] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.847534 27191 ts_tablet_manager.cc:1434] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:02:40.851382 26899 catalog_manager.cc:5697] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 1 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:40.857717 27193 tablet_bootstrap.cc:492] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:40.858512 27193 ts_tablet_manager.cc:1403] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.000s
I20260709 13:02:40.862453 27198 raft_consensus.cc:493] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.862692 27193 raft_consensus.cc:359] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.862987 27198 raft_consensus.cc:515] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.865280 27191 tablet_bootstrap.cc:492] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:40.866816 27197 raft_consensus.cc:493] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.867494 27197 raft_consensus.cc:515] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.869716 27197 leader_election.cc:290] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:40.871793 27197 raft_consensus.cc:493] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.871977 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194f838ca919474c8162e2def0e3da14" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:40.872380 27197 raft_consensus.cc:515] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
W20260709 13:02:40.873694 27115 leader_election.cc:343] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:40.874305 27191 tablet_bootstrap.cc:654] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.876682 27197 leader_election.cc:290] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:40.877183 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194f838ca919474c8162e2def0e3da14" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:40.877692 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:40.878342 27001 raft_consensus.cc:2468] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:40.880048 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
W20260709 13:02:40.880450 27117 leader_election.cc:343] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:40.881006 27117 leader_election.cc:304] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:40.881294 27192 tablet_bootstrap.cc:492] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:40.882496 27117 leader_election.cc:304] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:40.884888 27207 raft_consensus.cc:2749] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:40.885272 27193 raft_consensus.cc:385] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.885717 27207 raft_consensus.cc:2804] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:40.886235 27207 raft_consensus.cc:493] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.885741 27193 raft_consensus.cc:740] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.886714 27207 raft_consensus.cc:3060] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:40.888450 27115 leader_election.cc:343] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:40.888813 27193 consensus_queue.cc:260] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.883667 27192 ts_tablet_manager.cc:1403] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.012s
I20260709 13:02:40.881979 27198 leader_election.cc:290] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:40.904385 27193 ts_tablet_manager.cc:1434] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.045s	user 0.010s	sys 0.000s
I20260709 13:02:40.906366 27193 tablet_bootstrap.cc:492] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:40.911182 27192 raft_consensus.cc:359] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.912392 27192 raft_consensus.cc:385] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.912798 27192 raft_consensus.cc:740] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.913839 27197 raft_consensus.cc:493] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.914563 27197 raft_consensus.cc:515] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.918407 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:40.918819 27197 leader_election.cc:290] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259)
I20260709 13:02:40.919406 27077 raft_consensus.cc:2468] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:40.921782 27198 raft_consensus.cc:493] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.936722 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:40.937148 27198 raft_consensus.cc:515] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.937683 27115 leader_election.cc:304] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:40.922780 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
W20260709 13:02:40.938413 26966 leader_election.cc:343] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:40.932595 27192 consensus_queue.cc:260] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.934793 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:40.903132 27207 raft_consensus.cc:515] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.940387 27198 leader_election.cc:290] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:40.938443 27197 raft_consensus.cc:2804] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:40.928783 27191 tablet_bootstrap.cc:492] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:40.945153 27197 raft_consensus.cc:493] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.945454 27191 ts_tablet_manager.cc:1403] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.080s	user 0.032s	sys 0.021s
I20260709 13:02:40.945788 27197 raft_consensus.cc:3060] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.938910 27001 raft_consensus.cc:2468] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:40.942637 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:40.941468 27207 leader_election.cc:290] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:40.942870 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:40.949563 27191 raft_consensus.cc:359] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.950738 27002 raft_consensus.cc:3060] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.950830 27191 raft_consensus.cc:385] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.951368 27191 raft_consensus.cc:740] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.952106 27191 consensus_queue.cc:260] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.954241 27191 ts_tablet_manager.cc:1434] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:40.955302 27191 tablet_bootstrap.cc:492] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:40.943667 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc2ac862d0a4a2184e9a3c18a7264a2" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:40.944628 27151 raft_consensus.cc:2393] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 183d64694f2647e1b0c710360b97d497 in current term 1: Already voted for candidate 5f2ef05158654ce59432eeb8cb5b04bb in this term.
I20260709 13:02:40.956646 27150 raft_consensus.cc:2468] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
W20260709 13:02:40.957875 27115 leader_election.cc:343] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:40.943413 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc2ac862d0a4a2184e9a3c18a7264a2" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:40.958220 26965 leader_election.cc:304] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:40.958699 27077 raft_consensus.cc:2468] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:40.959239 27198 raft_consensus.cc:2804] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:40.959559 26965 leader_election.cc:304] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 183d64694f2647e1b0c710360b97d497; no voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:40.936498 27193 tablet_bootstrap.cc:654] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.959617 27198 raft_consensus.cc:493] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.961467 27198 raft_consensus.cc:3060] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.962909 27191 tablet_bootstrap.cc:654] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:40.966012 27197 raft_consensus.cc:515] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.969097 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:40.969691 27077 raft_consensus.cc:3060] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.970834 27198 raft_consensus.cc:515] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.973908 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc2ac862d0a4a2184e9a3c18a7264a2" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:40.974529 27076 raft_consensus.cc:3060] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.976781 27191 tablet_bootstrap.cc:492] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:40.977432 27191 ts_tablet_manager.cc:1403] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.022s	user 0.002s	sys 0.013s
I20260709 13:02:40.978657 27193 tablet_bootstrap.cc:492] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:40.979290 27193 ts_tablet_manager.cc:1403] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.073s	user 0.007s	sys 0.008s
I20260709 13:02:40.979421 27077 raft_consensus.cc:2468] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:40.980792 27115 leader_election.cc:304] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:40.980948 27191 raft_consensus.cc:359] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.981664 27207 raft_consensus.cc:2804] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:40.981829 27191 raft_consensus.cc:385] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.982013 27002 raft_consensus.cc:2468] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:40.982187 27207 raft_consensus.cc:697] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:40.982333 27191 raft_consensus.cc:740] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.982710 27218 raft_consensus.cc:2749] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:40.983544 27117 leader_election.cc:304] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:40.983491 27191 consensus_queue.cc:260] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.985673 27192 ts_tablet_manager.cc:1434] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.081s	user 0.014s	sys 0.003s
I20260709 13:02:40.985308 27207 consensus_queue.cc:237] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:40.982677 27193 raft_consensus.cc:359] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.990936 27193 raft_consensus.cc:385] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:40.986812 27192 tablet_bootstrap.cc:492] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:40.991151 27076 raft_consensus.cc:2468] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:40.988724 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:40.993428 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc2ac862d0a4a2184e9a3c18a7264a2" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:40.993826 27002 raft_consensus.cc:3060] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.994112 27151 raft_consensus.cc:3060] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:40.988325 27191 ts_tablet_manager.cc:1434] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:02:40.999748 27192 tablet_bootstrap.cc:654] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.001276 27191 tablet_bootstrap.cc:492] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:40.998044 26898 catalog_manager.cc:5697] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 1 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:40.991357 27193 raft_consensus.cc:740] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:40.992182 27207 raft_consensus.cc:2804] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.005290 27207 raft_consensus.cc:697] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:41.006194 27207 consensus_queue.cc:237] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:40.992643 27198 leader_election.cc:290] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:40.994015 26966 leader_election.cc:304] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:40.989223 27197 leader_election.cc:290] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259)
I20260709 13:02:41.007946 27191 tablet_bootstrap.cc:654] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.007987 27198 raft_consensus.cc:2804] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.008380 27199 raft_consensus.cc:493] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.008709 27198 raft_consensus.cc:697] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:41.008903 27199 raft_consensus.cc:515] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.009225 27002 raft_consensus.cc:2468] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:41.009647 27198 consensus_queue.cc:237] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.013567 27192 tablet_bootstrap.cc:492] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:41.014375 27192 ts_tablet_manager.cc:1403] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.007s
I20260709 13:02:41.018785 27192 raft_consensus.cc:359] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.020164 27192 raft_consensus.cc:385] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.020596 27192 raft_consensus.cc:740] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.021623 27192 consensus_queue.cc:260] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.024868 27192 ts_tablet_manager.cc:1434] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:02:41.026146 27191 tablet_bootstrap.cc:492] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:41.026419 27192 tablet_bootstrap.cc:492] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:41.026775 27191 ts_tablet_manager.cc:1403] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.005s
I20260709 13:02:41.030063 27191 raft_consensus.cc:359] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.033653 27199 leader_election.cc:290] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.033823 27191 raft_consensus.cc:385] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.042672 27191 raft_consensus.cc:740] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.043548 27191 consensus_queue.cc:260] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.043658 27193 consensus_queue.cc:260] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.045998 27191 ts_tablet_manager.cc:1434] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.003s
I20260709 13:02:41.034780 27192 tablet_bootstrap.cc:654] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.039165 26898 catalog_manager.cc:5697] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.048228 27151 raft_consensus.cc:2468] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:41.052430 27218 raft_consensus.cc:493] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.053043 27218 raft_consensus.cc:515] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.053515 27193 ts_tablet_manager.cc:1434] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.074s	user 0.003s	sys 0.004s
I20260709 13:02:41.054724 27193 tablet_bootstrap.cc:492] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:41.056905 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:41.060297 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194f838ca919474c8162e2def0e3da14" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:41.061659 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:41.062462 27191 tablet_bootstrap.cc:492] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
W20260709 13:02:41.067229 27043 leader_election.cc:343] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.062554 27076 raft_consensus.cc:2393] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 183d64694f2647e1b0c710360b97d497 in current term 1: Already voted for candidate 5f2ef05158654ce59432eeb8cb5b04bb in this term.
I20260709 13:02:41.073421 27191 tablet_bootstrap.cc:654] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.073298 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194f838ca919474c8162e2def0e3da14" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:41.073957 27218 leader_election.cc:290] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.074433 27151 raft_consensus.cc:2468] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:41.074890 27218 raft_consensus.cc:493] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.075387 27218 raft_consensus.cc:515] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.075958 27040 leader_election.cc:304] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:41.077327 27199 raft_consensus.cc:2804] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.077697 27199 raft_consensus.cc:493] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.078025 27199 raft_consensus.cc:3060] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.080346 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a180fcba65e243f2ace7ab30c1128475" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
W20260709 13:02:41.084616 26965 leader_election.cc:343] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.086637 26900 catalog_manager.cc:5697] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 1 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.087984 27191 tablet_bootstrap.cc:492] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:41.088766 27191 ts_tablet_manager.cc:1403] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.009s
I20260709 13:02:41.091775 27199 raft_consensus.cc:515] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.092630 27218 leader_election.cc:290] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.098937 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194f838ca919474c8162e2def0e3da14" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:41.099591 27151 raft_consensus.cc:3060] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.099659 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a180fcba65e243f2ace7ab30c1128475" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:41.100394 27076 raft_consensus.cc:2468] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:41.102396 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "194f838ca919474c8162e2def0e3da14" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:41.103826 26966 leader_election.cc:304] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 183d64694f2647e1b0c710360b97d497; no voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:41.105516 26966 leader_election.cc:304] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:41.106613 27218 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.107033 27199 leader_election.cc:290] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.094219 27193 tablet_bootstrap.cc:654] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.109774 27151 raft_consensus.cc:2468] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:41.093122 27191 raft_consensus.cc:359] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.114344 27191 raft_consensus.cc:385] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.114734 27040 leader_election.cc:304] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:41.114830 27191 raft_consensus.cc:740] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.115972 27199 raft_consensus.cc:2804] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.117654 27191 consensus_queue.cc:260] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.119510 27230 raft_consensus.cc:2804] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.120152 27230 raft_consensus.cc:493] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.120558 27230 raft_consensus.cc:3060] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:41.125070 27043 leader_election.cc:343] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.125034 27199 raft_consensus.cc:697] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:41.127099 27199 consensus_queue.cc:237] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.129755 27218 raft_consensus.cc:2749] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.133857 27192 tablet_bootstrap.cc:492] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:41.134725 27192 ts_tablet_manager.cc:1403] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.109s	user 0.055s	sys 0.032s
I20260709 13:02:41.139185 27192 raft_consensus.cc:359] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.140424 27192 raft_consensus.cc:385] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.140798 27192 raft_consensus.cc:740] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.138981 27230 raft_consensus.cc:515] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.133883 27193 tablet_bootstrap.cc:492] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:41.142120 27193 ts_tablet_manager.cc:1403] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.003s
I20260709 13:02:41.142298 27191 ts_tablet_manager.cc:1434] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.053s	user 0.011s	sys 0.021s
I20260709 13:02:41.142791 27234 raft_consensus.cc:493] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.143383 27234 raft_consensus.cc:515] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.143783 27191 tablet_bootstrap.cc:492] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:41.145457 27193 raft_consensus.cc:359] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.146281 27193 raft_consensus.cc:385] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.146574 27193 raft_consensus.cc:740] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.146656 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4efc5a1ab5e4577a3ca32495d6e4b04" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:41.147554 27193 consensus_queue.cc:260] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.149178 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4efc5a1ab5e4577a3ca32495d6e4b04" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:41.148756 26898 catalog_manager.cc:5697] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 0 to 1, leader changed from <none> to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 1 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.149823 27151 raft_consensus.cc:2468] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:41.150550 27192 consensus_queue.cc:260] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.151367 27191 tablet_bootstrap.cc:654] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.153213 27192 ts_tablet_manager.cc:1434] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:02:41.154371 27192 tablet_bootstrap.cc:492] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:41.156308 27207 raft_consensus.cc:493] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.156894 27207 raft_consensus.cc:515] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.159680 27207 leader_election.cc:290] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:41.150053 27193 ts_tablet_manager.cc:1434] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:02:41.160871 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:41.161856 27193 tablet_bootstrap.cc:492] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:41.163424 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:41.164672 27040 leader_election.cc:304] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
W20260709 13:02:41.166075 27115 leader_election.cc:343] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.167148 27002 raft_consensus.cc:2468] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:41.167578 27233 raft_consensus.cc:2804] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.168064 27233 raft_consensus.cc:493] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.168413 27233 raft_consensus.cc:3060] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:41.169034 27117 leader_election.cc:343] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.169531 27117 leader_election.cc:304] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:41.174192 27192 tablet_bootstrap.cc:654] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.178261 27193 tablet_bootstrap.cc:654] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.179942 27207 raft_consensus.cc:2749] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.180680 27234 leader_election.cc:290] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.184695 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a180fcba65e243f2ace7ab30c1128475" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:41.185395 27076 raft_consensus.cc:3060] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.187641 27192 tablet_bootstrap.cc:492] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:41.188391 27192 ts_tablet_manager.cc:1403] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.008s
I20260709 13:02:41.190721 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a180fcba65e243f2ace7ab30c1128475" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:41.191455 27230 leader_election.cc:290] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
W20260709 13:02:41.197374 26965 leader_election.cc:343] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.198167 27076 raft_consensus.cc:2468] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:41.199672 26966 leader_election.cc:304] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:41.200560 27192 raft_consensus.cc:359] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.200882 27233 raft_consensus.cc:515] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.201972 27230 raft_consensus.cc:2804] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.203229 27233 leader_election.cc:290] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.201332 27192 raft_consensus.cc:385] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.203855 27192 raft_consensus.cc:740] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.204372 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4efc5a1ab5e4577a3ca32495d6e4b04" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:41.204528 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4efc5a1ab5e4577a3ca32495d6e4b04" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:41.205181 27002 raft_consensus.cc:3060] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.205265 27151 raft_consensus.cc:3060] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.205749 27192 consensus_queue.cc:260] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.207604 27191 tablet_bootstrap.cc:492] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:41.208073 27192 ts_tablet_manager.cc:1434] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:02:41.209007 27191 ts_tablet_manager.cc:1403] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.065s	user 0.031s	sys 0.032s
I20260709 13:02:41.209468 27192 tablet_bootstrap.cc:492] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:41.212401 27230 raft_consensus.cc:697] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:41.213008 27191 raft_consensus.cc:359] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.213807 27191 raft_consensus.cc:385] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.213485 27230 consensus_queue.cc:237] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.214131 27191 raft_consensus.cc:740] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.215250 27002 raft_consensus.cc:2468] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:41.215639 27192 tablet_bootstrap.cc:654] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.222150 27191 consensus_queue.cc:260] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.220621 27151 raft_consensus.cc:2468] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:41.223991 27193 tablet_bootstrap.cc:492] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:41.222150 27043 leader_election.cc:304] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:41.233958 27193 ts_tablet_manager.cc:1403] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.009s
I20260709 13:02:41.235944 27233 raft_consensus.cc:2804] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.236660 27233 raft_consensus.cc:697] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:41.237658 27233 consensus_queue.cc:237] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.240447 27193 raft_consensus.cc:359] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.241549 27193 raft_consensus.cc:385] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.227699 27191 ts_tablet_manager.cc:1434] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 13:02:41.241968 27193 raft_consensus.cc:740] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.243230 27191 tablet_bootstrap.cc:492] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:41.243112 27193 consensus_queue.cc:260] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.245885 27193 ts_tablet_manager.cc:1434] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:02:41.247000 27193 tablet_bootstrap.cc:492] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:41.241969 26900 catalog_manager.cc:5697] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.250209 26898 catalog_manager.cc:5697] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 0 to 1, leader changed from <none> to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 1 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.253140 27192 tablet_bootstrap.cc:492] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:41.253726 27192 ts_tablet_manager.cc:1403] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.000s
I20260709 13:02:41.254691 27193 tablet_bootstrap.cc:654] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.260007 27191 tablet_bootstrap.cc:654] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.265246 27193 tablet_bootstrap.cc:492] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:41.265818 27193 ts_tablet_manager.cc:1403] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.000s
I20260709 13:02:41.268919 27193 raft_consensus.cc:359] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.269701 27193 raft_consensus.cc:385] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.269999 27193 raft_consensus.cc:740] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.270848 27193 consensus_queue.cc:260] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.273465 27193 ts_tablet_manager.cc:1434] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:41.274471 27193 tablet_bootstrap.cc:492] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:41.278177 27192 raft_consensus.cc:359] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.280323 27193 tablet_bootstrap.cc:654] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.295225 27192 raft_consensus.cc:385] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.295794 27192 raft_consensus.cc:740] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.316284 27192 consensus_queue.cc:260] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.317559 27207 raft_consensus.cc:493] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.318224 27207 raft_consensus.cc:515] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.350291 27192 ts_tablet_manager.cc:1434] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.086s	user 0.011s	sys 0.000s
I20260709 13:02:41.408341 27191 tablet_bootstrap.cc:492] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:41.408969 27191 ts_tablet_manager.cc:1403] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.166s	user 0.056s	sys 0.065s
I20260709 13:02:41.409025 27193 tablet_bootstrap.cc:492] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:41.409588 27193 ts_tablet_manager.cc:1403] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.135s	user 0.043s	sys 0.057s
I20260709 13:02:41.412072 27191 raft_consensus.cc:359] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.412397 27193 raft_consensus.cc:359] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.412858 27191 raft_consensus.cc:385] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.413166 27193 raft_consensus.cc:385] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.413558 27193 raft_consensus.cc:740] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.413218 27191 raft_consensus.cc:740] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.414345 27193 consensus_queue.cc:260] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.414958 27191 consensus_queue.cc:260] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.416560 27193 ts_tablet_manager.cc:1434] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:41.417248 27191 ts_tablet_manager.cc:1434] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:41.446887 27230 raft_consensus.cc:493] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.447579 27230 raft_consensus.cc:515] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.449846 27230 leader_election.cc:290] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.469957 27207 leader_election.cc:290] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:41.473503 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:41.474834 27075 raft_consensus.cc:2468] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:41.476526 27207 consensus_queue.cc:1048] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:41.477151 27197 consensus_queue.cc:1048] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:41.477929 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:41.478534 27151 raft_consensus.cc:2468] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:41.477164 27115 leader_election.cc:304] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:41.479971 26965 leader_election.cc:304] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:41.480788 27197 raft_consensus.cc:2804] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.482092 27197 raft_consensus.cc:493] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.482518 27197 raft_consensus.cc:3060] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.482600 27233 raft_consensus.cc:493] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.485424 27233 raft_consensus.cc:515] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.483820 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:41.487709 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:41.488341 27076 raft_consensus.cc:2468] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:41.489427 27233 leader_election.cc:290] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.484506 27230 raft_consensus.cc:2804] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.490202 27230 raft_consensus.cc:493] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.490859 27230 raft_consensus.cc:3060] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.490939 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:41.492396 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e4e0210defe44639398856436eb821c" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:41.491501 27219 consensus_queue.cc:1048] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:41.493175 27001 raft_consensus.cc:2393] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 948af4c6da234b748ca8952d7b81322e in current term 1: Already voted for candidate 5f2ef05158654ce59432eeb8cb5b04bb in this term.
I20260709 13:02:41.494206 27219 consensus_queue.cc:1048] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:41.497146 27040 leader_election.cc:304] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:41.499306 27233 raft_consensus.cc:3060] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.511147 27233 raft_consensus.cc:2749] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.520489 27198 consensus_queue.cc:1048] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:02:41.565878 27230 raft_consensus.cc:515] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.579046 27218 consensus_queue.cc:1048] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:41.581543 27000 raft_consensus.cc:2393] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f2ef05158654ce59432eeb8cb5b04bb in current term 1: Already voted for candidate 183d64694f2647e1b0c710360b97d497 in this term.
I20260709 13:02:41.600853 27252 raft_consensus.cc:493] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.601599 27252 raft_consensus.cc:515] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.604453 27197 raft_consensus.cc:515] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.607589 27197 leader_election.cc:290] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:41.607707 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a180fcba65e243f2ace7ab30c1128475" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:41.609011 27077 raft_consensus.cc:2393] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f2ef05158654ce59432eeb8cb5b04bb in current term 1: Already voted for candidate 183d64694f2647e1b0c710360b97d497 in this term.
I20260709 13:02:41.609632 27230 leader_election.cc:290] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.610754 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:41.611599 27151 raft_consensus.cc:2393] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 183d64694f2647e1b0c710360b97d497 in current term 1: Already voted for candidate 5f2ef05158654ce59432eeb8cb5b04bb in this term.
I20260709 13:02:41.612614 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:41.613345 27077 raft_consensus.cc:3060] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.623306 27252 leader_election.cc:290] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:41.623088 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:41.624985 27001 raft_consensus.cc:2393] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f2ef05158654ce59432eeb8cb5b04bb in current term 1: Already voted for candidate 183d64694f2647e1b0c710360b97d497 in this term.
I20260709 13:02:41.623648 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:41.623147 27002 raft_consensus.cc:3060] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.627490 27252 consensus_queue.cc:1048] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:41.632301 27233 raft_consensus.cc:493] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.633916 27233 raft_consensus.cc:515] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.636394 27233 leader_election.cc:290] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.632396 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a180fcba65e243f2ace7ab30c1128475" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:41.633105 26966 leader_election.cc:304] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 183d64694f2647e1b0c710360b97d497; no voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:41.637526 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:41.638026 27230 raft_consensus.cc:2749] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.638098 27151 raft_consensus.cc:2468] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:41.639163 27040 leader_election.cc:304] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:41.639485 27233 consensus_queue.cc:1048] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:41.639511 27117 leader_election.cc:304] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:41.640118 27234 raft_consensus.cc:2804] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.640512 27234 raft_consensus.cc:493] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.640853 27234 raft_consensus.cc:3060] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.644544 27233 consensus_queue.cc:1048] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:41.647784 27197 raft_consensus.cc:3060] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.652984 27233 consensus_queue.cc:1048] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:41.658957 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:41.659956 27000 raft_consensus.cc:2468] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:41.660401 27252 consensus_queue.cc:1048] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:41.660722 27197 raft_consensus.cc:2749] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.672801 27234 raft_consensus.cc:515] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.690408 27149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:41.691099 27149 raft_consensus.cc:3060] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.707284 27149 raft_consensus.cc:2468] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:41.682890 27077 raft_consensus.cc:2468] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:41.708333 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:41.709132 27000 raft_consensus.cc:3060] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.709805 27115 leader_election.cc:304] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:41.711025 27252 raft_consensus.cc:2804] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.711534 27252 raft_consensus.cc:697] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:41.712458 27252 consensus_queue.cc:237] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.708398 27234 leader_election.cc:290] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:41.715356 27040 leader_election.cc:304] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:41.716271 27234 raft_consensus.cc:2804] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.716775 27234 raft_consensus.cc:697] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:41.717603 27234 consensus_queue.cc:237] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:41.725019 27000 raft_consensus.cc:2468] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:41.728948 27199 consensus_queue.cc:1048] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:41.740974 26897 catalog_manager.cc:5697] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 1 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.741995 26899 catalog_manager.cc:5697] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 0 to 1, leader changed from <none> to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 1 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.808141 27230 consensus_queue.cc:1048] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:41.824440 27230 consensus_queue.cc:1048] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.014424 27150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
mode: GRACEFUL
new_leader_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:33110
I20260709 13:02:42.015046 27150 raft_consensus.cc:606] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Received request to transfer leadership to TS 183d64694f2647e1b0c710360b97d497
I20260709 13:02:42.020994 27150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "486278766c1e42e6908c196fff181e29"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
mode: GRACEFUL
new_leader_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:33110
I20260709 13:02:42.021667 27150 raft_consensus.cc:606] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Received request to transfer leadership to TS 183d64694f2647e1b0c710360b97d497
I20260709 13:02:42.038589 27150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
mode: GRACEFUL
new_leader_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:33110
I20260709 13:02:42.039142 27150 raft_consensus.cc:606] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Received request to transfer leadership to TS 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:42.062003 27197 raft_consensus.cc:993] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb: : Instructing follower 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:42.062541 27252 raft_consensus.cc:1081] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Signalling peer 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:42.064370 27002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "486278766c1e42e6908c196fff181e29"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:56596
I20260709 13:02:42.064973 27002 raft_consensus.cc:493] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.065274 27002 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.071614 27002 raft_consensus.cc:515] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:42.073699 27002 leader_election.cc:290] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:42.074848 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:42.074846 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:42.075554 27150 raft_consensus.cc:3055] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.075626 27077 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.075912 27150 raft_consensus.cc:740] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:42.076807 27150 consensus_queue.cc:260] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:42.077906 27150 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.081929 27077 raft_consensus.cc:2468] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:42.083292 26966 leader_election.cc:304] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:42.084115 27230 raft_consensus.cc:2804] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.084483 27150 raft_consensus.cc:2468] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:42.084748 27230 raft_consensus.cc:697] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:42.085757 27230 consensus_queue.cc:237] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:42.092945 27197 raft_consensus.cc:993] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb: : Instructing follower 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:42.093454 27252 raft_consensus.cc:1081] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Signalling peer 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:42.093885 26897 catalog_manager.cc:5697] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 1 to 2, leader changed from 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67) to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.095073 27002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:56596
I20260709 13:02:42.095634 27002 raft_consensus.cc:493] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.096125 27002 raft_consensus.cc:3060] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.103461 27002 raft_consensus.cc:515] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:42.105434 27002 leader_election.cc:290] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Requested vote from peers 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:42.106240 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:42.106444 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4cc5719ad354c1f897af009ef1e528f" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:42.107017 27150 raft_consensus.cc:3055] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.107066 27077 raft_consensus.cc:3060] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.107379 27150 raft_consensus.cc:740] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:42.108242 27150 consensus_queue.cc:260] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:42.109261 27150 raft_consensus.cc:3060] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.112957 27077 raft_consensus.cc:2468] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:42.114084 26966 leader_election.cc:304] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:42.114786 27150 raft_consensus.cc:2468] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:42.114919 27218 raft_consensus.cc:2804] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.115490 27218 raft_consensus.cc:697] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:42.116252 27218 consensus_queue.cc:237] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } }
I20260709 13:02:42.123147 26897 catalog_manager.cc:5697] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 1 to 2, leader changed from 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67) to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.192359 27252 consensus_queue.cc:1048] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.204380 27197 consensus_queue.cc:1048] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.219102 27252 raft_consensus.cc:993] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb: : Instructing follower 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:42.219647 27275 raft_consensus.cc:1081] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Signalling peer 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:42.221341 27077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3"
dest_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:54258
I20260709 13:02:42.221870 27077 raft_consensus.cc:493] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.222193 27077 raft_consensus.cc:3060] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.227612 27077 raft_consensus.cc:515] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:42.229336 27077 leader_election.cc:290] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 2 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:42.230334 27002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:42.230490 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43741078ec4b4b40bffbb89360aea0c3" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:42.231148 27002 raft_consensus.cc:3060] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.231244 27150 raft_consensus.cc:3055] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.231678 27150 raft_consensus.cc:740] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:42.232527 27150 consensus_queue.cc:260] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:42.233657 27150 raft_consensus.cc:3060] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.237697 27002 raft_consensus.cc:2468] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 2.
I20260709 13:02:42.239208 27043 leader_election.cc:304] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:42.240751 27266 raft_consensus.cc:2804] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.241387 27266 raft_consensus.cc:697] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 2 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:42.242462 27266 consensus_queue.cc:237] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:42.245970 27150 raft_consensus.cc:2468] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 2.
I20260709 13:02:42.248204 27266 consensus_queue.cc:1048] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.250273 26897 catalog_manager.cc:5697] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 1 to 2, leader changed from 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67) to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 2 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.265589 27279 consensus_queue.cc:1048] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:02:42.519665 27172 debug-util.cc:398] Leaking SignalData structure 0x7b080012b6e0 after lost signal to thread 12412
W20260709 13:02:42.520712 27172 debug-util.cc:398] Leaking SignalData structure 0x7b0800188920 after lost signal to thread 26934
W20260709 13:02:42.521756 27172 debug-util.cc:398] Leaking SignalData structure 0x7b0800289280 after lost signal to thread 27026
W20260709 13:02:42.522650 27172 debug-util.cc:398] Leaking SignalData structure 0x7b0800089280 after lost signal to thread 27101
I20260709 13:02:42.530452 27077 raft_consensus.cc:1275] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:42.531975 27281 consensus_queue.cc:1048] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:42.544358 27150 raft_consensus.cc:1275] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:42.546227 27218 consensus_queue.cc:1048] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:42.602476 27149 raft_consensus.cc:1275] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:42.604027 27218 consensus_queue.cc:1048] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:42.614209 27077 raft_consensus.cc:1275] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:42.615671 27282 consensus_queue.cc:1048] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:42.770290 27149 raft_consensus.cc:1275] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:42.771723 27286 consensus_queue.cc:1048] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:42.780743 27002 raft_consensus.cc:1275] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:42.782665 27279 consensus_queue.cc:1048] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:45.084690 26897 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55530:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:45.087218 26897 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:45.157961 26987 tablet_service.cc:1511] Processing CreateTablet for tablet 06958f3c898c48469fde073474821db4 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:45.159328 26987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06958f3c898c48469fde073474821db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.159401 27065 tablet_service.cc:1511] Processing CreateTablet for tablet 06958f3c898c48469fde073474821db4 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:45.160847 27065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06958f3c898c48469fde073474821db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.165839 27135 tablet_service.cc:1511] Processing CreateTablet for tablet 06958f3c898c48469fde073474821db4 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:45.167330 27135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06958f3c898c48469fde073474821db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.169029 27138 tablet_service.cc:1511] Processing CreateTablet for tablet 8c9a97ce1cbc4fc7afa5f7e6f9667e35 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:45.170601 27138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c9a97ce1cbc4fc7afa5f7e6f9667e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.174988 26984 tablet_service.cc:1511] Processing CreateTablet for tablet 8c9a97ce1cbc4fc7afa5f7e6f9667e35 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:45.176225 26986 tablet_service.cc:1511] Processing CreateTablet for tablet df50bd530bcf4f3897564361f46c94c9 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:45.178172 26986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df50bd530bcf4f3897564361f46c94c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.182644 27137 tablet_service.cc:1511] Processing CreateTablet for tablet 26e2417c48ff4a298b480607a8d56306 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:45.185127 27139 tablet_service.cc:1511] Processing CreateTablet for tablet df50bd530bcf4f3897564361f46c94c9 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:45.185949 26988 tablet_service.cc:1511] Processing CreateTablet for tablet 17cd77ddaa4c49c69401db747c839a2e (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:45.187282 27134 tablet_service.cc:1511] Processing CreateTablet for tablet 17cd77ddaa4c49c69401db747c839a2e (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:45.189682 26991 tablet_service.cc:1511] Processing CreateTablet for tablet c9f6e15b98c84ff9984133d1818ec329 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:45.191872 27063 tablet_service.cc:1511] Processing CreateTablet for tablet 8c9a97ce1cbc4fc7afa5f7e6f9667e35 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:45.202390 27063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c9a97ce1cbc4fc7afa5f7e6f9667e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.192986 27067 tablet_service.cc:1511] Processing CreateTablet for tablet 26e2417c48ff4a298b480607a8d56306 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:45.205531 27067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e2417c48ff4a298b480607a8d56306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.194061 27061 tablet_service.cc:1511] Processing CreateTablet for tablet df50bd530bcf4f3897564361f46c94c9 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:45.210302 27061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df50bd530bcf4f3897564361f46c94c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.196024 26990 tablet_service.cc:1511] Processing CreateTablet for tablet bff299a80e7444d09bdb0fe966ace0c4 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:45.198681 27062 tablet_service.cc:1511] Processing CreateTablet for tablet 17cd77ddaa4c49c69401db747c839a2e (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:45.195173 26992 tablet_service.cc:1511] Processing CreateTablet for tablet 56f7a265065f4c47b22e0159d88fc262 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:45.216904 27062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17cd77ddaa4c49c69401db747c839a2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.187327 26988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17cd77ddaa4c49c69401db747c839a2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.177044 26985 tablet_service.cc:1511] Processing CreateTablet for tablet 26e2417c48ff4a298b480607a8d56306 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:45.219696 26991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9f6e15b98c84ff9984133d1818ec329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.220993 26985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e2417c48ff4a298b480607a8d56306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.224462 27133 tablet_service.cc:1511] Processing CreateTablet for tablet c9f6e15b98c84ff9984133d1818ec329 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:45.229511 26989 tablet_service.cc:1511] Processing CreateTablet for tablet 3511bbeafe1f4e65ba4b3148f4dfac2c (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:45.236159 27134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17cd77ddaa4c49c69401db747c839a2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.245141 27133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9f6e15b98c84ff9984133d1818ec329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.246937 27139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df50bd530bcf4f3897564361f46c94c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.248593 27137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e2417c48ff4a298b480607a8d56306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.262295 26990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff299a80e7444d09bdb0fe966ace0c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.264961 26992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f7a265065f4c47b22e0159d88fc262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.267750 26989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3511bbeafe1f4e65ba4b3148f4dfac2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.270289 26984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c9a97ce1cbc4fc7afa5f7e6f9667e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.282809 27140 tablet_service.cc:1511] Processing CreateTablet for tablet 56f7a265065f4c47b22e0159d88fc262 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:45.285312 27140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f7a265065f4c47b22e0159d88fc262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.285792 27141 tablet_service.cc:1511] Processing CreateTablet for tablet bff299a80e7444d09bdb0fe966ace0c4 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:45.290684 27141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff299a80e7444d09bdb0fe966ace0c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.305585 27066 tablet_service.cc:1511] Processing CreateTablet for tablet c9f6e15b98c84ff9984133d1818ec329 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:45.307216 27066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9f6e15b98c84ff9984133d1818ec329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.309661 27064 tablet_service.cc:1511] Processing CreateTablet for tablet 56f7a265065f4c47b22e0159d88fc262 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:45.311203 27064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f7a265065f4c47b22e0159d88fc262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.313591 27060 tablet_service.cc:1511] Processing CreateTablet for tablet bff299a80e7444d09bdb0fe966ace0c4 (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:45.315063 27060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff299a80e7444d09bdb0fe966ace0c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.317394 27059 tablet_service.cc:1511] Processing CreateTablet for tablet 3511bbeafe1f4e65ba4b3148f4dfac2c (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:45.318749 27059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3511bbeafe1f4e65ba4b3148f4dfac2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.324970 27303 tablet_bootstrap.cc:492] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:45.353226 27303 tablet_bootstrap.cc:654] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.362689 27304 tablet_bootstrap.cc:492] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:45.369349 27136 tablet_service.cc:1511] Processing CreateTablet for tablet 3511bbeafe1f4e65ba4b3148f4dfac2c (DEFAULT_TABLE table=soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:45.371446 27136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3511bbeafe1f4e65ba4b3148f4dfac2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:45.393182 27303 tablet_bootstrap.cc:492] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:45.395525 27303 ts_tablet_manager.cc:1403] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.071s	user 0.027s	sys 0.000s
I20260709 13:02:45.401569 27303 raft_consensus.cc:359] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.404628 27303 raft_consensus.cc:385] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.405498 27303 raft_consensus.cc:740] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.406757 27304 tablet_bootstrap.cc:654] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.406803 27303 consensus_queue.cc:260] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.431339 27303 ts_tablet_manager.cc:1434] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.004s
I20260709 13:02:45.435755 27303 tablet_bootstrap.cc:492] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:45.453104 27303 tablet_bootstrap.cc:654] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.491210 27303 tablet_bootstrap.cc:492] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:45.492789 27303 ts_tablet_manager.cc:1403] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.009s
I20260709 13:02:45.501538 27303 raft_consensus.cc:359] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.505307 27303 raft_consensus.cc:385] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.506469 27303 raft_consensus.cc:740] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.508761 27303 consensus_queue.cc:260] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.526062 27303 ts_tablet_manager.cc:1434] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 13:02:45.562453 27303 tablet_bootstrap.cc:492] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:45.566517 27304 tablet_bootstrap.cc:492] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:45.568660 27304 ts_tablet_manager.cc:1403] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.206s	user 0.046s	sys 0.101s
I20260709 13:02:45.573808 27303 tablet_bootstrap.cc:654] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.573905 27304 raft_consensus.cc:359] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.574844 27304 raft_consensus.cc:385] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.575238 27304 raft_consensus.cc:740] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.617852 27304 consensus_queue.cc:260] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.634416 27304 ts_tablet_manager.cc:1434] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.065s	user 0.002s	sys 0.005s
I20260709 13:02:45.649240 27301 tablet_bootstrap.cc:492] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:45.661247 27304 tablet_bootstrap.cc:492] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:45.666106 27301 tablet_bootstrap.cc:654] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.677248 27304 tablet_bootstrap.cc:654] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:45.693917 26966 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:45595, user_credentials={real_user=slave}} blocked reactor thread for 393908us
I20260709 13:02:45.719800 27301 tablet_bootstrap.cc:492] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:45.720522 27301 ts_tablet_manager.cc:1403] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.029s
I20260709 13:02:45.728317 27303 tablet_bootstrap.cc:492] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:45.729000 27303 ts_tablet_manager.cc:1403] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.167s	user 0.009s	sys 0.005s
I20260709 13:02:45.733104 27304 tablet_bootstrap.cc:492] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:45.740358 27304 ts_tablet_manager.cc:1403] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.019s
I20260709 13:02:45.743104 27304 raft_consensus.cc:359] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.723685 27301 raft_consensus.cc:359] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.742762 27303 raft_consensus.cc:359] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.743882 27304 raft_consensus.cc:385] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.744196 27303 raft_consensus.cc:385] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.744429 27304 raft_consensus.cc:740] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.744800 27303 raft_consensus.cc:740] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.744050 27301 raft_consensus.cc:385] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.745640 27301 raft_consensus.cc:740] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.746742 27301 consensus_queue.cc:260] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.750367 27304 consensus_queue.cc:260] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.753434 27304 ts_tablet_manager.cc:1434] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:02:45.749606 27301 ts_tablet_manager.cc:1434] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:02:45.755210 27288 raft_consensus.cc:493] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.756088 27301 tablet_bootstrap.cc:492] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:45.755833 27288 raft_consensus.cc:515] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.759166 27288 leader_election.cc:290] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:45.764020 27301 tablet_bootstrap.cc:654] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.754817 27304 tablet_bootstrap.cc:492] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:45.769380 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff299a80e7444d09bdb0fe966ace0c4" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
W20260709 13:02:45.774315 27117 leader_election.cc:343] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:45.778519 27304 tablet_bootstrap.cc:654] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.781056 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff299a80e7444d09bdb0fe966ace0c4" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
W20260709 13:02:45.785156 27115 leader_election.cc:343] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:45.785712 27115 leader_election.cc:304] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:45.789868 27288 raft_consensus.cc:2749] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:45.795415 27303 consensus_queue.cc:260] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.800108 27301 tablet_bootstrap.cc:492] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:45.801298 27301 ts_tablet_manager.cc:1403] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.007s
I20260709 13:02:45.805321 27301 raft_consensus.cc:359] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.812220 27303 ts_tablet_manager.cc:1434] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.001s
I20260709 13:02:45.816974 27304 tablet_bootstrap.cc:492] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:45.818272 27304 ts_tablet_manager.cc:1403] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.007s
I20260709 13:02:45.818612 27303 tablet_bootstrap.cc:492] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:45.821696 27304 raft_consensus.cc:359] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.822749 27304 raft_consensus.cc:385] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.823151 27304 raft_consensus.cc:740] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.824087 27304 consensus_queue.cc:260] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.817485 27301 raft_consensus.cc:385] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.825323 27301 raft_consensus.cc:740] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.826351 27301 consensus_queue.cc:260] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.829365 27301 ts_tablet_manager.cc:1434] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.005s
I20260709 13:02:45.831712 27308 raft_consensus.cc:493] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.836292 27308 raft_consensus.cc:515] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.838908 27308 leader_election.cc:290] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:45.840963 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:45.841835 27301 tablet_bootstrap.cc:492] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:45.835657 27304 ts_tablet_manager.cc:1434] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:02:45.845232 27304 tablet_bootstrap.cc:492] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:45.847555 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
W20260709 13:02:45.849355 26966 leader_election.cc:343] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:45.850302 26965 leader_election.cc:343] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:45.850806 26965 leader_election.cc:304] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 183d64694f2647e1b0c710360b97d497; no voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:45.851730 27308 raft_consensus.cc:2749] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:45.859516 27304 tablet_bootstrap.cc:654] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.832343 27303 tablet_bootstrap.cc:654] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.866729 27301 tablet_bootstrap.cc:654] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:45.882099 26948 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:45.884908 27303 tablet_bootstrap.cc:492] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:45.885780 27303 ts_tablet_manager.cc:1403] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.003s
I20260709 13:02:45.889528 27303 raft_consensus.cc:359] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.890501 27303 raft_consensus.cc:385] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.890923 27303 raft_consensus.cc:740] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.892972 27304 tablet_bootstrap.cc:492] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:45.892632 27303 consensus_queue.cc:260] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.893620 27304 ts_tablet_manager.cc:1403] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.005s
I20260709 13:02:45.896123 27303 ts_tablet_manager.cc:1434] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:02:45.897505 27303 tablet_bootstrap.cc:492] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:45.901962 27301 tablet_bootstrap.cc:492] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:45.903281 27301 ts_tablet_manager.cc:1403] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.012s
I20260709 13:02:45.907073 27303 tablet_bootstrap.cc:654] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.907274 27301 raft_consensus.cc:359] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.897511 27304 raft_consensus.cc:359] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.908417 27304 raft_consensus.cc:385] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.908648 27300 raft_consensus.cc:493] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.908859 27304 raft_consensus.cc:740] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.909409 27300 raft_consensus.cc:515] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.910332 27304 consensus_queue.cc:260] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.912496 27300 leader_election.cc:290] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:45.913518 27304 ts_tablet_manager.cc:1434] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.001s
I20260709 13:02:45.914758 27304 tablet_bootstrap.cc:492] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:45.915444 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:45.915727 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:45.916504 27150 raft_consensus.cc:2468] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
W20260709 13:02:45.917407 27043 leader_election.cc:343] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:45.918879 27040 leader_election.cc:304] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:45.921016 27300 raft_consensus.cc:2804] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:45.921543 27300 raft_consensus.cc:493] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.921981 27300 raft_consensus.cc:3060] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:45.923630 27304 tablet_bootstrap.cc:654] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.915108 27301 raft_consensus.cc:385] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.924167 27301 raft_consensus.cc:740] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.925017 27301 consensus_queue.cc:260] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.931021 27300 raft_consensus.cc:515] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.927747 27301 ts_tablet_manager.cc:1434] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:02:45.938633 27303 tablet_bootstrap.cc:492] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:45.939639 27303 ts_tablet_manager.cc:1403] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.009s
I20260709 13:02:45.940201 27301 tablet_bootstrap.cc:492] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:45.946358 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
W20260709 13:02:45.948273 27043 leader_election.cc:343] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:45.949721 27300 leader_election.cc:290] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:45.943307 27303 raft_consensus.cc:359] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.950750 27303 raft_consensus.cc:385] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:45.950913 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:45.951215 27303 raft_consensus.cc:740] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:45.951668 27150 raft_consensus.cc:3060] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:45.952226 27303 consensus_queue.cc:260] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.955135 27303 ts_tablet_manager.cc:1434] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:02:45.956328 27303 tablet_bootstrap.cc:492] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:45.960121 27301 tablet_bootstrap.cc:654] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.960772 27150 raft_consensus.cc:2468] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:45.964232 27040 leader_election.cc:304] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:45.964334 27303 tablet_bootstrap.cc:654] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:45.965289 27300 raft_consensus.cc:2804] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:45.965936 27300 raft_consensus.cc:697] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:45.967058 27300 consensus_queue.cc:237] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.970028 27317 raft_consensus.cc:493] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.971194 27317 raft_consensus.cc:515] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:45.975982 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff299a80e7444d09bdb0fe966ace0c4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
W20260709 13:02:45.978485 27043 leader_election.cc:343] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:45.983119 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff299a80e7444d09bdb0fe966ace0c4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:45.984117 27150 raft_consensus.cc:2468] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:45.986171 27040 leader_election.cc:304] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:45.986737 27301 tablet_bootstrap.cc:492] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:45.987450 27317 leader_election.cc:290] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:45.987717 27301 ts_tablet_manager.cc:1403] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.008s
I20260709 13:02:46.006911 27317 raft_consensus.cc:2804] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.007506 27317 raft_consensus.cc:493] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.007983 27317 raft_consensus.cc:3060] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.014412 27301 raft_consensus.cc:359] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.020517 27304 tablet_bootstrap.cc:492] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:46.021636 27304 ts_tablet_manager.cc:1403] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.107s	user 0.028s	sys 0.028s
I20260709 13:02:46.016225 27301 raft_consensus.cc:385] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.024796 27301 raft_consensus.cc:740] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.031534 27288 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.032490 27288 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.033965 27304 raft_consensus.cc:359] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.034786 27304 raft_consensus.cc:385] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.035089 27304 raft_consensus.cc:740] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.036031 27304 consensus_queue.cc:260] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.036927 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:46.037242 27301 consensus_queue.cc:260] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.038479 27304 ts_tablet_manager.cc:1434] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:02:46.028654 27322 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.040329 27322 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
W20260709 13:02:46.041633 27117 leader_election.cc:343] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.042927 27322 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.047021 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.048012 27150 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:46.049690 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.050388 27075 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:46.050428 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:46.049096 27288 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.051997 27115 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
W20260709 13:02:46.052353 27043 leader_election.cc:343] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.053402 27040 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:46.054497 27303 tablet_bootstrap.cc:492] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:46.055064 27303 ts_tablet_manager.cc:1403] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.099s	user 0.016s	sys 0.001s
I20260709 13:02:46.055770 27288 raft_consensus.cc:2804] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.056316 27288 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.058463 27303 raft_consensus.cc:359] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.060454 27288 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.060555 27300 raft_consensus.cc:2804] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.060809 27303 raft_consensus.cc:385] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.061152 27300 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.061245 27301 ts_tablet_manager.cc:1434] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 13:02:46.061625 27300 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.061412 27303 raft_consensus.cc:740] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.062572 27304 tablet_bootstrap.cc:492] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:46.063017 27303 consensus_queue.cc:260] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.059078 26897 catalog_manager.cc:5697] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 0 to 1, leader changed from <none> to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 1 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.067160 27301 tablet_bootstrap.cc:492] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:46.079653 27301 tablet_bootstrap.cc:654] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.065352 27317 raft_consensus.cc:515] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.087244 27300 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.088037 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff299a80e7444d09bdb0fe966ace0c4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:46.088325 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff299a80e7444d09bdb0fe966ace0c4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:46.088966 27150 raft_consensus.cc:3060] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.089600 27300 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.090646 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:46.105572 27149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:46.092905 27317 leader_election.cc:290] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
W20260709 13:02:46.107151 27043 leader_election.cc:343] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.094909 27288 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.114171 27327 raft_consensus.cc:493] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.119974 27303 ts_tablet_manager.cc:1434] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.065s	user 0.020s	sys 0.005s
I20260709 13:02:46.121848 27303 tablet_bootstrap.cc:492] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
W20260709 13:02:46.120000 27043 leader_election.cc:343] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.129285 27303 tablet_bootstrap.cc:654] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.145361 27149 raft_consensus.cc:2393] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 948af4c6da234b748ca8952d7b81322e in current term 1: Already voted for candidate 5f2ef05158654ce59432eeb8cb5b04bb in this term.
I20260709 13:02:46.150962 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
W20260709 13:02:46.152889 27117 leader_election.cc:343] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.157856 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:46.158874 27075 raft_consensus.cc:2393] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f2ef05158654ce59432eeb8cb5b04bb in current term 1: Already voted for candidate 948af4c6da234b748ca8952d7b81322e in this term.
I20260709 13:02:46.157927 27040 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:46.156731 27288 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.132854 27327 raft_consensus.cc:515] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.161645 27300 raft_consensus.cc:2749] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:46.164249 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
W20260709 13:02:46.165908 27117 leader_election.cc:343] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.167279 27115 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:46.170156 27301 tablet_bootstrap.cc:492] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:46.170748 27301 ts_tablet_manager.cc:1403] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.104s	user 0.028s	sys 0.010s
I20260709 13:02:46.171499 27304 tablet_bootstrap.cc:654] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.174304 27301 raft_consensus.cc:359] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.175314 27301 raft_consensus.cc:385] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.176092 27288 raft_consensus.cc:2749] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:02:46.175631 27301 raft_consensus.cc:740] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.176292 27303 tablet_bootstrap.cc:492] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:46.177825 27301 consensus_queue.cc:260] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.167382 27150 raft_consensus.cc:2468] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:46.182650 27303 ts_tablet_manager.cc:1403] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.014s
I20260709 13:02:46.183140 27040 leader_election.cc:304] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:46.184947 27300 raft_consensus.cc:2804] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:46.185494 27300 raft_consensus.cc:697] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:46.186669 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.186378 27300 consensus_queue.cc:237] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.187340 27327 leader_election.cc:290] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.187515 27075 raft_consensus.cc:2468] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:46.188894 27115 leader_election.cc:304] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:46.191354 27304 tablet_bootstrap.cc:492] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:46.192018 27304 ts_tablet_manager.cc:1403] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.130s	user 0.016s	sys 0.000s
I20260709 13:02:46.196367 27327 raft_consensus.cc:2804] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.196933 27327 raft_consensus.cc:493] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.197319 27327 raft_consensus.cc:3060] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.205533 27303 raft_consensus.cc:359] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.206457 27303 raft_consensus.cc:385] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.206792 27303 raft_consensus.cc:740] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.206220 26899 catalog_manager.cc:5697] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 0 to 1, leader changed from <none> to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 1 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.207739 27303 consensus_queue.cc:260] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.209249 27301 ts_tablet_manager.cc:1434] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.002s
I20260709 13:02:46.195850 27304 raft_consensus.cc:359] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.211025 27304 raft_consensus.cc:385] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.211455 27304 raft_consensus.cc:740] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.211756 27303 ts_tablet_manager.cc:1434] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:02:46.212414 27304 consensus_queue.cc:260] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.213935 27301 tablet_bootstrap.cc:492] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:46.221439 27304 ts_tablet_manager.cc:1434] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:02:46.222760 27304 tablet_bootstrap.cc:492] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:46.223289 27303 tablet_bootstrap.cc:492] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:46.227463 27327 raft_consensus.cc:515] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.228386 27301 tablet_bootstrap.cc:654] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.229382 27288 raft_consensus.cc:493] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.229910 27288 raft_consensus.cc:515] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.232142 27288 leader_election.cc:290] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.234946 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:46.236539 27303 tablet_bootstrap.cc:654] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.237177 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:46.238127 27075 raft_consensus.cc:3060] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.240046 27327 leader_election.cc:290] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.240358 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:46.241298 27301 tablet_bootstrap.cc:492] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:46.241257 27000 raft_consensus.cc:2468] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:46.242092 27301 ts_tablet_manager.cc:1403] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.007s
I20260709 13:02:46.242547 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
W20260709 13:02:46.246748 27115 leader_election.cc:343] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.248085 27327 raft_consensus.cc:493] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.248358 27117 leader_election.cc:343] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.248693 27327 raft_consensus.cc:515] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.249148 27117 leader_election.cc:304] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:46.248863 27308 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.249938 27308 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.250713 27288 raft_consensus.cc:2804] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.251204 27288 raft_consensus.cc:493] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.251504 27288 raft_consensus.cc:3060] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.256349 27327 leader_election.cc:290] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.258884 27301 raft_consensus.cc:359] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.259702 27301 raft_consensus.cc:385] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.260041 27301 raft_consensus.cc:740] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.263391 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.264145 27076 raft_consensus.cc:2468] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:46.265914 27115 leader_election.cc:304] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:46.266114 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:46.260850 27301 consensus_queue.cc:260] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.266835 27000 raft_consensus.cc:2468] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 0.
I20260709 13:02:46.270116 27308 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.270921 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.232748 27304 tablet_bootstrap.cc:654] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.272118 27076 raft_consensus.cc:2393] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 183d64694f2647e1b0c710360b97d497 in current term 1: Already voted for candidate 948af4c6da234b748ca8952d7b81322e in this term.
I20260709 13:02:46.266855 27327 raft_consensus.cc:2804] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.273976 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.274813 27150 raft_consensus.cc:2393] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 183d64694f2647e1b0c710360b97d497 in current term 1: Already voted for candidate 5f2ef05158654ce59432eeb8cb5b04bb in this term.
I20260709 13:02:46.274998 27301 ts_tablet_manager.cc:1434] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.004s
I20260709 13:02:46.275501 27288 raft_consensus.cc:515] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.276238 27301 tablet_bootstrap.cc:492] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:46.285873 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:46.286247 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:46.286504 27000 raft_consensus.cc:3060] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:46.288077 27115 leader_election.cc:343] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:46.288782 27288 leader_election.cc:290] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.291543 27301 tablet_bootstrap.cc:654] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.276957 27075 raft_consensus.cc:2468] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:46.277343 26965 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 183d64694f2647e1b0c710360b97d497; no voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:46.279423 27304 tablet_bootstrap.cc:492] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:46.293493 27304 ts_tablet_manager.cc:1403] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.000s
I20260709 13:02:46.295195 27115 leader_election.cc:304] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497
I20260709 13:02:46.296103 27308 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.296476 27288 raft_consensus.cc:2804] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:46.297727 27288 raft_consensus.cc:697] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:46.298791 27288 consensus_queue.cc:237] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.274035 27327 raft_consensus.cc:493] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.282680 27303 tablet_bootstrap.cc:492] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:46.300125 27327 raft_consensus.cc:3060] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.300412 27303 ts_tablet_manager.cc:1403] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.004s
I20260709 13:02:46.311247 27303 raft_consensus.cc:359] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.312068 27303 raft_consensus.cc:385] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.311789 27304 raft_consensus.cc:359] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.312379 27303 raft_consensus.cc:740] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.312639 27304 raft_consensus.cc:385] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.312940 27304 raft_consensus.cc:740] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.313218 27303 consensus_queue.cc:260] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.313755 27304 consensus_queue.cc:260] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.315322 27300 raft_consensus.cc:493] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.315763 27300 raft_consensus.cc:515] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.317917 27300 leader_election.cc:290] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.318681 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:46.319746 27001 raft_consensus.cc:2468] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:46.324323 27308 raft_consensus.cc:2749] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:46.324941 27327 raft_consensus.cc:515] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.326251 27000 raft_consensus.cc:2468] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:46.326366 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.327348 27303 ts_tablet_manager.cc:1434] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:02:46.327661 27150 raft_consensus.cc:2468] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 0.
I20260709 13:02:46.326754 27043 leader_election.cc:304] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:46.328500 27303 tablet_bootstrap.cc:492] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb: Bootstrap starting.
I20260709 13:02:46.328600 27117 leader_election.cc:304] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:46.329962 27330 raft_consensus.cc:2804] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:46.330472 27330 raft_consensus.cc:697] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:46.331367 27327 leader_election.cc:290] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:46.331316 27330 consensus_queue.cc:237] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.332281 27301 tablet_bootstrap.cc:492] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:46.332616 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:46.333250 27075 raft_consensus.cc:3060] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.330492 27304 ts_tablet_manager.cc:1434] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:02:46.336238 27304 tablet_bootstrap.cc:492] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:46.328938 27300 raft_consensus.cc:2804] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.336796 27300 raft_consensus.cc:493] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.339219 27300 raft_consensus.cc:3060] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.342942 27301 ts_tablet_manager.cc:1403] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.006s
I20260709 13:02:46.342882 27075 raft_consensus.cc:2468] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:46.344430 27115 leader_election.cc:304] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:46.345461 27288 raft_consensus.cc:2804] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:46.346431 27288 raft_consensus.cc:697] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:46.346683 27301 raft_consensus.cc:359] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.349750 27304 tablet_bootstrap.cc:654] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.348129 27301 raft_consensus.cc:385] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.356734 27301 raft_consensus.cc:740] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.357604 27301 consensus_queue.cc:260] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.349303 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:46.357601 27303 tablet_bootstrap.cc:654] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.365137 26897 catalog_manager.cc:5697] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 1 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.360316 27301 ts_tablet_manager.cc:1434] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 13:02:46.368225 27301 tablet_bootstrap.cc:492] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:46.361353 27288 consensus_queue.cc:237] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.360723 27000 raft_consensus.cc:3060] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.383031 27300 raft_consensus.cc:515] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.376335 27301 tablet_bootstrap.cc:654] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.389401 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:46.390070 27001 raft_consensus.cc:3060] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.391054 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:46.391817 27150 raft_consensus.cc:3060] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.399544 27001 raft_consensus.cc:2468] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:46.401350 27043 leader_election.cc:304] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:46.402371 27317 raft_consensus.cc:2804] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:46.403128 27317 raft_consensus.cc:697] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:46.404109 27317 consensus_queue.cc:237] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.405870 27308 raft_consensus.cc:493] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.406486 27308 raft_consensus.cc:515] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.408731 27308 leader_election.cc:290] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.410810 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.411840 27300 leader_election.cc:290] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.411306 26897 catalog_manager.cc:5697] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 1 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.413302 27150 raft_consensus.cc:2468] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 1.
I20260709 13:02:46.413429 27149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.414295 27149 raft_consensus.cc:2393] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 183d64694f2647e1b0c710360b97d497 in current term 1: Already voted for candidate 948af4c6da234b748ca8952d7b81322e in this term.
I20260709 13:02:46.414337 26897 catalog_manager.cc:5697] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 1 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.415839 26965 leader_election.cc:304] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 183d64694f2647e1b0c710360b97d497; no voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:46.417043 27308 raft_consensus.cc:3060] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.422271 27000 raft_consensus.cc:2468] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 1.
I20260709 13:02:46.426395 27308 raft_consensus.cc:2749] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:46.427314 27303 tablet_bootstrap.cc:492] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb: No bootstrap required, opened a new log
I20260709 13:02:46.428095 27303 ts_tablet_manager.cc:1403] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent bootstrapping tablet: real 0.100s	user 0.015s	sys 0.001s
I20260709 13:02:46.432622 27303 raft_consensus.cc:359] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.433642 27303 raft_consensus.cc:385] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.433971 27303 raft_consensus.cc:740] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.435705 27303 consensus_queue.cc:260] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.438158 27304 tablet_bootstrap.cc:492] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:46.438875 27304 ts_tablet_manager.cc:1403] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.026s
I20260709 13:02:46.445221 27317 consensus_queue.cc:1048] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:46.444926 26897 catalog_manager.cc:5697] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 0 to 1, leader changed from <none> to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 1 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.446211 27325 consensus_queue.cc:1048] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:46.446455 27304 raft_consensus.cc:359] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.452276 27289 raft_consensus.cc:493] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.452786 27289 raft_consensus.cc:515] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.455006 27289 leader_election.cc:290] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.456234 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.457202 27150 raft_consensus.cc:2468] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:46.457434 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.458252 27075 raft_consensus.cc:2468] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:46.459131 26965 leader_election.cc:304] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:46.459506 27304 raft_consensus.cc:385] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.460167 27289 raft_consensus.cc:2804] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.460644 27289 raft_consensus.cc:493] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.461041 27289 raft_consensus.cc:3060] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.459807 27304 raft_consensus.cc:740] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.469900 27304 consensus_queue.cc:260] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.485437 27304 ts_tablet_manager.cc:1434] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.003s
I20260709 13:02:46.491978 27304 tablet_bootstrap.cc:492] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e: Bootstrap starting.
I20260709 13:02:46.489902 27289 raft_consensus.cc:515] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.515877 27303 ts_tablet_manager.cc:1434] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 13:02:46.522694 27317 raft_consensus.cc:493] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.523254 27317 raft_consensus.cc:515] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.525141 27317 leader_election.cc:290] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.526315 27149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.530372 27301 tablet_bootstrap.cc:492] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:46.530709 27304 tablet_bootstrap.cc:654] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.530927 27301 ts_tablet_manager.cc:1403] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.163s	user 0.034s	sys 0.040s
I20260709 13:02:46.534096 27301 raft_consensus.cc:359] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.534816 27301 raft_consensus.cc:385] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.534751 27289 leader_election.cc:290] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.535403 27301 raft_consensus.cc:740] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.536197 27301 consensus_queue.cc:260] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.538664 27301 ts_tablet_manager.cc:1434] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:46.540017 27301 tablet_bootstrap.cc:492] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497: Bootstrap starting.
I20260709 13:02:46.592685 27149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:46.594583 27149 raft_consensus.cc:3060] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.596344 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183d64694f2647e1b0c710360b97d497" is_pre_election: true
I20260709 13:02:46.597940 27001 raft_consensus.cc:2393] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 948af4c6da234b748ca8952d7b81322e in current term 1: Already voted for candidate 5f2ef05158654ce59432eeb8cb5b04bb in this term.
I20260709 13:02:46.603160 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:46.604303 27075 raft_consensus.cc:3060] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.613487 27308 raft_consensus.cc:493] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.614884 27308 raft_consensus.cc:515] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.618624 27301 tablet_bootstrap.cc:654] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:46.621493 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e2417c48ff4a298b480607a8d56306" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.622758 27150 raft_consensus.cc:2468] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:46.624549 27325 consensus_queue.cc:1048] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:46.619481 27308 leader_election.cc:290] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.635669 26965 leader_election.cc:304] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:46.638166 27075 raft_consensus.cc:2468] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:46.640146 27289 raft_consensus.cc:2804] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:46.640929 27289 raft_consensus.cc:493] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.641518 27289 raft_consensus.cc:3060] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.644950 27304 tablet_bootstrap.cc:492] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e: No bootstrap required, opened a new log
I20260709 13:02:46.646171 27304 ts_tablet_manager.cc:1403] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e: Time spent bootstrapping tablet: real 0.154s	user 0.007s	sys 0.010s
I20260709 13:02:46.650569 27043 leader_election.cc:304] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 948af4c6da234b748ca8952d7b81322e; no voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:46.651634 26966 leader_election.cc:304] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:46.653208 27354 raft_consensus.cc:2804] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:46.653898 27354 raft_consensus.cc:697] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:46.653971 27325 raft_consensus.cc:3060] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.655501 27354 consensus_queue.cc:237] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.650862 27304 raft_consensus.cc:359] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.658887 27304 raft_consensus.cc:385] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.659257 27304 raft_consensus.cc:740] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.661295 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e2417c48ff4a298b480607a8d56306" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.662523 27149 raft_consensus.cc:2468] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
W20260709 13:02:46.662763 27043 consensus_peers.cc:597] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e -> Peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Couldn't send request to peer 183d64694f2647e1b0c710360b97d497. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:46.664204 27304 consensus_queue.cc:260] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.665511 27076 raft_consensus.cc:2468] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 0.
I20260709 13:02:46.673499 27304 ts_tablet_manager.cc:1434] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.005s
I20260709 13:02:46.681725 27301 tablet_bootstrap.cc:492] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497: No bootstrap required, opened a new log
I20260709 13:02:46.682910 27301 ts_tablet_manager.cc:1403] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497: Time spent bootstrapping tablet: real 0.143s	user 0.019s	sys 0.000s
I20260709 13:02:46.713807 27289 raft_consensus.cc:515] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.715710 27301 raft_consensus.cc:359] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.722669 27301 raft_consensus.cc:385] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:46.723178 27301 raft_consensus.cc:740] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Initialized, Role: FOLLOWER
I20260709 13:02:46.724647 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e2417c48ff4a298b480607a8d56306" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:46.727293 27076 raft_consensus.cc:3060] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.728384 27301 consensus_queue.cc:260] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.729684 27289 leader_election.cc:290] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.732196 27301 ts_tablet_manager.cc:1434] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497: Time spent starting tablet: real 0.049s	user 0.011s	sys 0.001s
I20260709 13:02:46.725833 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e2417c48ff4a298b480607a8d56306" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:46.736297 27151 raft_consensus.cc:3060] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.755215 27325 raft_consensus.cc:2749] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:46.755548 26897 catalog_manager.cc:5697] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.756611 27151 raft_consensus.cc:2468] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:46.758064 26965 leader_election.cc:304] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:46.758677 27076 raft_consensus.cc:2468] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:46.758914 27354 raft_consensus.cc:2804] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:46.760929 27354 raft_consensus.cc:697] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:46.762490 27354 consensus_queue.cc:237] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.800341 27288 consensus_queue.cc:1048] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:46.806310 27330 consensus_queue.cc:1048] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:46.800675 26898 catalog_manager.cc:5697] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 1 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.840416 27327 consensus_queue.cc:1048] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:46.852167 27327 consensus_queue.cc:1048] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:46.866166 27354 raft_consensus.cc:493] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.867561 27329 consensus_queue.cc:1048] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:46.866756 27354 raft_consensus.cc:515] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:46.874501 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:46.877529 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:46.880362 26966 leader_election.cc:304] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 183d64694f2647e1b0c710360b97d497; no voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:46.880823 27354 leader_election.cc:290] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:46.882817 27289 raft_consensus.cc:2749] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:02:46.890134 26948 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:46.896224 27001 raft_consensus.cc:3060] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:46.911283 27329 consensus_queue.cc:1048] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:46.925863 27325 consensus_queue.cc:1048] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:46.941577 27321 consensus_queue.cc:1048] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:47.063483 27354 consensus_queue.cc:1048] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:47.079783 27308 consensus_queue.cc:1048] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:47.095888 27001 raft_consensus.cc:3060] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:47.156788 27308 consensus_queue.cc:1048] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:47.169118 27358 consensus_queue.cc:1048] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:02:47.896679 26948 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:47.960440 27358 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.961148 27358 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:47.963114 27358 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:47.964059 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e" is_pre_election: true
I20260709 13:02:47.964298 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" is_pre_election: true
I20260709 13:02:47.964918 27077 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:47.965129 27151 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 1.
I20260709 13:02:47.966195 26966 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:47.967036 27358 raft_consensus.cc:2804] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:02:47.967415 27358 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.967728 27358 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.977941 27358 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:47.980098 27358 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:47.980978 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:47.981281 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:47.981761 27077 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.981873 27151 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:47.993005 27151 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:47.993022 27077 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:47.994933 26966 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:47.995975 27358 raft_consensus.cc:2804] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:47.996474 27358 raft_consensus.cc:697] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:47.997526 27358 consensus_queue.cc:237] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.006085 26900 catalog_manager.cc:5697] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 0 to 2, leader changed from <none> to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.416335 27077 raft_consensus.cc:1275] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:02:48.417568 27358 consensus_queue.cc:1048] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:48.426378 27075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06958f3c898c48469fde073474821db4"
dest_uuid: "948af4c6da234b748ca8952d7b81322e"
mode: GRACEFUL
new_leader_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:48890
I20260709 13:02:48.427138 27075 raft_consensus.cc:606] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Received request to transfer leadership to TS 183d64694f2647e1b0c710360b97d497
I20260709 13:02:48.434629 27151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
mode: GRACEFUL
new_leader_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:33110
I20260709 13:02:48.435459 27151 raft_consensus.cc:606] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Received request to transfer leadership to TS 183d64694f2647e1b0c710360b97d497
I20260709 13:02:48.447116 27150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
mode: GRACEFUL
new_leader_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:33110
I20260709 13:02:48.447432 27151 raft_consensus.cc:1275] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:02:48.447809 27150 raft_consensus.cc:606] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Received request to transfer leadership to TS 183d64694f2647e1b0c710360b97d497
I20260709 13:02:48.449251 27358 consensus_queue.cc:1048] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:48.483428 27001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
mode: GRACEFUL
new_leader_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:54954
I20260709 13:02:48.484431 27001 raft_consensus.cc:606] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Received request to transfer leadership to TS 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:48.491708 27150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
mode: GRACEFUL
new_leader_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:33110
I20260709 13:02:48.492390 27150 raft_consensus.cc:606] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Received request to transfer leadership to TS 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:48.499413 27001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
mode: GRACEFUL
new_leader_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:54954
I20260709 13:02:48.500212 27001 raft_consensus.cc:606] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Received request to transfer leadership to TS 948af4c6da234b748ca8952d7b81322e
I20260709 13:02:48.513794 27077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c"
dest_uuid: "948af4c6da234b748ca8952d7b81322e"
mode: GRACEFUL
new_leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
 from {username='slave'} at 127.0.0.1:48890
I20260709 13:02:48.514593 27077 raft_consensus.cc:606] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Received request to transfer leadership to TS 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:48.581418 27380 raft_consensus.cc:993] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: : Instructing follower 5f2ef05158654ce59432eeb8cb5b04bb to start an election
I20260709 13:02:48.582008 27321 raft_consensus.cc:1081] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Signalling peer 5f2ef05158654ce59432eeb8cb5b04bb to start an election
I20260709 13:02:48.584007 27150 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
 from {username='slave'} at 127.0.0.1:54434
I20260709 13:02:48.584599 27150 raft_consensus.cc:493] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:48.584959 27150 raft_consensus.cc:3060] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.590283 27374 raft_consensus.cc:993] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: : Instructing follower 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:48.592272 27390 raft_consensus.cc:1081] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Signalling peer 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:48.594425 27001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:56596
I20260709 13:02:48.594473 27150 raft_consensus.cc:515] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.595109 27001 raft_consensus.cc:493] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:48.595590 27001 raft_consensus.cc:3060] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.597465 27150 leader_election.cc:290] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 2 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:48.597978 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:48.598137 27077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511bbeafe1f4e65ba4b3148f4dfac2c" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:48.598811 27077 raft_consensus.cc:3055] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:48.598757 27000 raft_consensus.cc:3060] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.599403 27077 raft_consensus.cc:740] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:48.600543 27077 consensus_queue.cc:260] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.601804 27077 raft_consensus.cc:3060] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.607954 27001 raft_consensus.cc:515] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.610457 27001 leader_election.cc:290] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:48.611195 27000 raft_consensus.cc:2468] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 2.
I20260709 13:02:48.611757 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:48.612033 27077 raft_consensus.cc:2468] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 2.
I20260709 13:02:48.612624 27075 raft_consensus.cc:3060] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.613601 27115 leader_election.cc:304] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:48.614714 27390 raft_consensus.cc:2804] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:48.615231 27390 raft_consensus.cc:697] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:48.615996 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df50bd530bcf4f3897564361f46c94c9" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:48.616694 27150 raft_consensus.cc:3055] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:48.616322 27390 consensus_queue.cc:237] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.617040 27150 raft_consensus.cc:740] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:48.617936 27150 consensus_queue.cc:260] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.619072 27150 raft_consensus.cc:3060] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.623039 27075 raft_consensus.cc:2468] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:48.624747 26966 leader_election.cc:304] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:48.625725 27358 raft_consensus.cc:2804] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:48.626256 27358 raft_consensus.cc:697] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:48.627467 27358 consensus_queue.cc:237] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.627775 27150 raft_consensus.cc:2468] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:48.627768 26900 catalog_manager.cc:5697] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 1 to 2, leader changed from 948af4c6da234b748ca8952d7b81322e (127.12.30.66) to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 2 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:48.637418 26899 catalog_manager.cc:5697] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 1 to 2, leader changed from 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67) to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.738408 27374 raft_consensus.cc:993] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: : Instructing follower 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:48.738991 27390 raft_consensus.cc:1081] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Signalling peer 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:48.740525 27000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:56596
I20260709 13:02:48.741122 27000 raft_consensus.cc:493] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:48.741466 27000 raft_consensus.cc:3060] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.750363 27000 raft_consensus.cc:515] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.752403 27000 leader_election.cc:290] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:48.753043 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:48.753422 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c9a97ce1cbc4fc7afa5f7e6f9667e35" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:48.753875 27075 raft_consensus.cc:3060] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.753973 27150 raft_consensus.cc:3055] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:48.754400 27150 raft_consensus.cc:740] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:48.755393 27150 consensus_queue.cc:260] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.756363 27150 raft_consensus.cc:3060] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:02:48.756482 27117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.762894 27075 raft_consensus.cc:2468] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:48.764367 26966 leader_election.cc:304] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:48.764892 27150 raft_consensus.cc:2468] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:48.765111 27358 raft_consensus.cc:2804] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:48.765736 27358 raft_consensus.cc:697] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:48.766963 27358 consensus_queue.cc:237] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.777197 26899 catalog_manager.cc:5697] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 1 to 2, leader changed from 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67) to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.951890 27380 raft_consensus.cc:993] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: : Instructing follower 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:48.952515 27321 raft_consensus.cc:1081] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Signalling peer 183d64694f2647e1b0c710360b97d497 to start an election
I20260709 13:02:48.954164 27000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06958f3c898c48469fde073474821db4"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
 from {username='slave'} at 127.0.0.1:56602
I20260709 13:02:48.954691 27000 raft_consensus.cc:493] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:48.954993 27000 raft_consensus.cc:3060] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.961102 27000 raft_consensus.cc:515] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.962978 27000 leader_election.cc:290] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Requested vote from peers 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:48.963938 27075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:48.964165 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06958f3c898c48469fde073474821db4" candidate_uuid: "183d64694f2647e1b0c710360b97d497" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:48.964684 27075 raft_consensus.cc:3055] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:48.964689 27151 raft_consensus.cc:3060] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.966609 27075 raft_consensus.cc:740] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:48.967435 27075 consensus_queue.cc:260] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.968454 27075 raft_consensus.cc:3060] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:48.970865 27151 raft_consensus.cc:2468] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:48.972213 26965 leader_election.cc:304] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:48.973227 27358 raft_consensus.cc:2804] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:48.973671 27358 raft_consensus.cc:697] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Leader. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:48.974612 27358 consensus_queue.cc:237] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:48.974932 27075 raft_consensus.cc:2468] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 183d64694f2647e1b0c710360b97d497 in term 2.
I20260709 13:02:48.981936 26899 catalog_manager.cc:5697] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 reported cstate change: term changed from 1 to 2, leader changed from 948af4c6da234b748ca8952d7b81322e (127.12.30.66) to 183d64694f2647e1b0c710360b97d497 (127.12.30.65). New cstate: current_term: 2 leader_uuid: "183d64694f2647e1b0c710360b97d497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:48.991973 27289 raft_consensus.cc:993] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: : Instructing follower 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:48.992436 27358 raft_consensus.cc:1081] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Signalling peer 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:48.993790 27075 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e"
dest_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:54268
I20260709 13:02:48.994345 27075 raft_consensus.cc:493] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:48.994648 27075 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:48.997154 27289 raft_consensus.cc:993] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: : Instructing follower 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:48.997716 27358 raft_consensus.cc:1081] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Signalling peer 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:48.999130 27077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262"
dest_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:54268
I20260709 13:02:48.999564 27077 raft_consensus.cc:493] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:48.999823 27077 raft_consensus.cc:3060] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.000794 27075 raft_consensus.cc:515] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.002576 27075 leader_election.cc:290] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 3 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:49.003656 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:49.004305 27151 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:49.004349 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17cd77ddaa4c49c69401db747c839a2e" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:49.005010 27000 raft_consensus.cc:3055] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:49.005225 27000 raft_consensus.cc:740] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:49.005920 27000 consensus_queue.cc:260] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.005841 27077 raft_consensus.cc:515] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.006965 27000 raft_consensus.cc:3060] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:49.008361 27077 leader_election.cc:290] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 2 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:49.008734 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:49.008909 27150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f7a265065f4c47b22e0159d88fc262" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:49.009348 27001 raft_consensus.cc:3055] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:49.009385 27150 raft_consensus.cc:3060] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.009821 27001 raft_consensus.cc:740] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:49.010699 27001 consensus_queue.cc:260] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.011512 27151 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 3.
I20260709 13:02:49.011859 27001 raft_consensus.cc:3060] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.012991 27040 leader_election.cc:304] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:49.013823 27321 raft_consensus.cc:2804] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:49.014312 27321 raft_consensus.cc:697] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 3 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:49.015245 27321 consensus_queue.cc:237] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.016422 27150 raft_consensus.cc:2468] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 2.
I20260709 13:02:49.017472 27040 leader_election.cc:304] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:49.018214 27380 raft_consensus.cc:2804] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:49.018705 27380 raft_consensus.cc:697] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 2 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:49.019559 27380 consensus_queue.cc:237] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.019781 27001 raft_consensus.cc:2468] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 2.
I20260709 13:02:49.022789 27000 raft_consensus.cc:2468] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 3.
I20260709 13:02:49.024849 26899 catalog_manager.cc:5697] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 2 to 3, leader changed from 183d64694f2647e1b0c710360b97d497 (127.12.30.65) to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 3 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:49.037858 26899 catalog_manager.cc:5697] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 1 to 2, leader changed from 183d64694f2647e1b0c710360b97d497 (127.12.30.65) to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 2 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:49.042596 27374 raft_consensus.cc:993] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: : Instructing follower 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:49.043305 27390 raft_consensus.cc:1081] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Signalling peer 948af4c6da234b748ca8952d7b81322e to start an election
I20260709 13:02:49.046001 27077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329"
dest_uuid: "948af4c6da234b748ca8952d7b81322e"
 from {username='slave'} at 127.0.0.1:54258
I20260709 13:02:49.046622 27077 raft_consensus.cc:493] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:49.046779 27075 raft_consensus.cc:1275] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 5f2ef05158654ce59432eeb8cb5b04bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.046981 27077 raft_consensus.cc:3060] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.048228 27390 consensus_queue.cc:1048] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.057548 27150 raft_consensus.cc:1275] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.057559 27077 raft_consensus.cc:515] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.059533 27358 consensus_queue.cc:1048] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.062229 27151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
I20260709 13:02:49.062543 27077 leader_election.cc:290] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 2 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67:37099)
I20260709 13:02:49.062918 27151 raft_consensus.cc:3055] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:49.063258 27151 raft_consensus.cc:740] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:49.064263 27151 consensus_queue.cc:260] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.065919 27151 raft_consensus.cc:3060] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.065968 27001 raft_consensus.cc:1275] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Refusing update from remote peer 5f2ef05158654ce59432eeb8cb5b04bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.068140 27374 consensus_queue.cc:1048] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.074374 27000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9f6e15b98c84ff9984133d1818ec329" candidate_uuid: "948af4c6da234b748ca8952d7b81322e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:49.075062 27000 raft_consensus.cc:3060] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.085465 27000 raft_consensus.cc:2468] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 2.
I20260709 13:02:49.086752 27043 leader_election.cc:304] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:49.087579 27380 raft_consensus.cc:2804] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:49.088110 27380 raft_consensus.cc:697] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 2 LEADER]: Becoming Leader. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:49.089231 27151 raft_consensus.cc:2468] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948af4c6da234b748ca8952d7b81322e in term 2.
I20260709 13:02:49.089092 27380 consensus_queue.cc:237] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:49.098343 27075 raft_consensus.cc:1275] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.100031 27404 consensus_queue.cc:1048] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.113826 26899 catalog_manager.cc:5697] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e reported cstate change: term changed from 1 to 2, leader changed from 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67) to 948af4c6da234b748ca8952d7b81322e (127.12.30.66). New cstate: current_term: 2 leader_uuid: "948af4c6da234b748ca8952d7b81322e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:49.218715 27150 raft_consensus.cc:1275] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.220121 27404 consensus_queue.cc:1048] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.231974 27075 raft_consensus.cc:1275] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.234747 27404 consensus_queue.cc:1048] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.462826 27150 raft_consensus.cc:1275] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.464502 27289 consensus_queue.cc:1048] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:49.476854 27075 raft_consensus.cc:1275] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 183d64694f2647e1b0c710360b97d497: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.478756 27404 consensus_queue.cc:1048] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.512616 27150 raft_consensus.cc:1275] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.514247 27380 consensus_queue.cc:1048] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.526072 27000 raft_consensus.cc:1275] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.528247 27001 raft_consensus.cc:1275] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.528306 27380 consensus_queue.cc:1048] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:49.530689 27406 consensus_queue.cc:1048] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.546176 27150 raft_consensus.cc:1275] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:49.547675 27406 consensus_queue.cc:1048] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.586642 27000 raft_consensus.cc:1275] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:49.587970 27406 consensus_queue.cc:1048] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:49.597483 27150 raft_consensus.cc:1275] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 FOLLOWER]: Refusing update from remote peer 948af4c6da234b748ca8952d7b81322e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:02:49.598899 27321 consensus_queue.cc:1048] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:51.518844 26899 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55530:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:02:51.541222 26899 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55530:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:02:51.590332 12409 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:51.768504 27458 tablet.cc:1724] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:51.808231 27446 log.cc:927] Time spent T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: Append to log took a long time: real 0.083s	user 0.002s	sys 0.000s
W20260709 13:02:51.851902 27459 log.cc:927] Time spent T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260709 13:02:51.851094 27469 tablet.cc:1724] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:51.878754 27464 log.cc:927] Time spent T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: Append to log took a long time: real 0.079s	user 0.004s	sys 0.000s
I20260709 13:02:51.956979 27483 tablet.cc:1724] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:51.856925 27472 tablet.cc:1724] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:51.989487 27480 tablet.cc:1724] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:51.979656 27477 tablet.cc:1724] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:51.993669 27478 tablet.cc:1724] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.010251 27458 tablet.cc:1724] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:52.019994 27476 log.cc:927] Time spent T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:02:52.032111 27489 tablet.cc:1724] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:51.998627 27483 tablet.cc:1724] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:52.058107 27043 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.65:35259, user_credentials={real_user=slave}} blocked reactor thread for 105595us
I20260709 13:02:52.019793 27485 tablet.cc:1724] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.092592 27478 tablet.cc:1724] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.106591 27469 tablet.cc:1724] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.110939 27489 tablet.cc:1724] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.029636 27488 tablet.cc:1724] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.120781 27472 tablet.cc:1724] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.119153 27477 tablet.cc:1724] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.128530 27485 tablet.cc:1724] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:02:52.136832 26966 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.30.66:45595, user_credentials={real_user=slave}} blocked reactor thread for 67248.7us
I20260709 13:02:52.140262 27480 tablet.cc:1724] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.143162 27494 tablet.cc:1724] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.152426 27472 tablet.cc:1724] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.159907 27492 tablet.cc:1724] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.162472 27483 tablet.cc:1724] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.168678 27488 tablet.cc:1724] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.170095 27458 tablet.cc:1724] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.186060 26999 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "486278766c1e42e6908c196fff181e29"
dest_uuid: "183d64694f2647e1b0c710360b97d497"
mode: GRACEFUL
new_leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
 from {username='slave'} at 127.0.0.1:51426
I20260709 13:02:52.186851 26999 raft_consensus.cc:606] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Received request to transfer leadership to TS 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:52.210515 27485 tablet.cc:1724] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.246598 27477 tablet.cc:1724] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:02:52.249670 26899 catalog_manager.cc:6242] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882: soft_deleted_table [id=e99513f2cf374bef904992cb5f7dafbe] alter complete (version 1)
I20260709 13:02:52.250491 27076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2"
dest_uuid: "948af4c6da234b748ca8952d7b81322e"
mode: GRACEFUL
new_leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
 from {username='slave'} at 127.0.0.1:59880
I20260709 13:02:52.251204 27076 raft_consensus.cc:606] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Received request to transfer leadership to TS 5f2ef05158654ce59432eeb8cb5b04bb
I20260709 13:02:52.258311 12409 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:52.259035 12409 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:52.442889 27491 raft_consensus.cc:993] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497: : Instructing follower 5f2ef05158654ce59432eeb8cb5b04bb to start an election
I20260709 13:02:52.443562 27443 raft_consensus.cc:1081] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Signalling peer 5f2ef05158654ce59432eeb8cb5b04bb to start an election
I20260709 13:02:52.445343 27151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "486278766c1e42e6908c196fff181e29"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:02:52.445786 27151 raft_consensus.cc:493] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:52.446058 27151 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:52.451063 27151 raft_consensus.cc:515] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:52.452713 27151 leader_election.cc:290] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 3 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:52.453426 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:52.453820 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "486278766c1e42e6908c196fff181e29" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:52.454042 27001 raft_consensus.cc:3055] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:52.454427 27001 raft_consensus.cc:740] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 183d64694f2647e1b0c710360b97d497, State: Running, Role: LEADER
I20260709 13:02:52.454507 27076 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:52.455170 27001 consensus_queue.cc:260] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:52.456287 27001 raft_consensus.cc:3060] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:52.461118 27076 raft_consensus.cc:2468] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 3.
I20260709 13:02:52.461977 27001 raft_consensus.cc:2468] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 3.
I20260709 13:02:52.462535 27115 leader_election.cc:304] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f2ef05158654ce59432eeb8cb5b04bb, 948af4c6da234b748ca8952d7b81322e; no voters: 
I20260709 13:02:52.463677 27487 raft_consensus.cc:2804] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:52.464143 27487 raft_consensus.cc:697] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:52.464880 27487 consensus_queue.cc:237] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:52.472604 26900 catalog_manager.cc:5697] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 2 to 3, leader changed from 183d64694f2647e1b0c710360b97d497 (127.12.30.65) to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 3 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.689689 27467 raft_consensus.cc:993] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e: : Instructing follower 5f2ef05158654ce59432eeb8cb5b04bb to start an election
I20260709 13:02:52.690320 27465 raft_consensus.cc:1081] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Signalling peer 5f2ef05158654ce59432eeb8cb5b04bb to start an election
I20260709 13:02:52.691745 27151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2"
dest_uuid: "5f2ef05158654ce59432eeb8cb5b04bb"
 from {username='slave'} at 127.0.0.1:54434
I20260709 13:02:52.692240 27151 raft_consensus.cc:493] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:52.692507 27151 raft_consensus.cc:3060] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:52.697281 27151 raft_consensus.cc:515] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:52.699043 27151 leader_election.cc:290] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 2 election: Requested vote from peers 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259), 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595)
I20260709 13:02:52.699625 27001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "183d64694f2647e1b0c710360b97d497"
I20260709 13:02:52.700156 27076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2dfd29191814d62bb243bd4be7fdba2" candidate_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "948af4c6da234b748ca8952d7b81322e"
I20260709 13:02:52.700512 27001 raft_consensus.cc:3060] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:52.700789 27076 raft_consensus.cc:3055] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:52.701229 27076 raft_consensus.cc:740] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 948af4c6da234b748ca8952d7b81322e, State: Running, Role: LEADER
I20260709 13:02:52.702085 27076 consensus_queue.cc:260] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:52.703377 27076 raft_consensus.cc:3060] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:52.706676 27001 raft_consensus.cc:2468] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 2.
I20260709 13:02:52.707715 27117 leader_election.cc:304] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183d64694f2647e1b0c710360b97d497, 5f2ef05158654ce59432eeb8cb5b04bb; no voters: 
I20260709 13:02:52.708542 27487 raft_consensus.cc:2804] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:52.708680 27076 raft_consensus.cc:2468] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f2ef05158654ce59432eeb8cb5b04bb in term 2.
I20260709 13:02:52.709024 27487 raft_consensus.cc:697] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 LEADER]: Becoming Leader. State: Replica: 5f2ef05158654ce59432eeb8cb5b04bb, State: Running, Role: LEADER
I20260709 13:02:52.710094 27487 consensus_queue.cc:237] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } }
I20260709 13:02:52.718407 26900 catalog_manager.cc:5697] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb reported cstate change: term changed from 1 to 2, leader changed from 948af4c6da234b748ca8952d7b81322e (127.12.30.66) to 5f2ef05158654ce59432eeb8cb5b04bb (127.12.30.67). New cstate: current_term: 2 leader_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f2ef05158654ce59432eeb8cb5b04bb" member_type: VOTER last_known_addr { host: "127.12.30.67" port: 37099 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.853924 27076 raft_consensus.cc:1275] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 3 FOLLOWER]: Refusing update from remote peer 5f2ef05158654ce59432eeb8cb5b04bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:52.855333 27487 consensus_queue.cc:1048] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:52.870039 27001 raft_consensus.cc:1275] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Refusing update from remote peer 5f2ef05158654ce59432eeb8cb5b04bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:52.872341 27487 consensus_queue.cc:1048] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:53.122922 27073 raft_consensus.cc:1275] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Refusing update from remote peer 5f2ef05158654ce59432eeb8cb5b04bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.124992 27487 consensus_queue.cc:1048] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "948af4c6da234b748ca8952d7b81322e" member_type: VOTER last_known_addr { host: "127.12.30.66" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:53.142534 27001 raft_consensus.cc:1275] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Refusing update from remote peer 5f2ef05158654ce59432eeb8cb5b04bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.144601 27487 consensus_queue.cc:1048] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "183d64694f2647e1b0c710360b97d497" member_type: VOTER last_known_addr { host: "127.12.30.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:53.336710 12409 tablet_server.cc:179] TabletServer@127.12.30.65:0 shutting down...
I20260709 13:02:53.382232 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:53.383080 12409 tablet_replica.cc:333] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.383872 12409 raft_consensus.cc:2243] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.385178 12409 raft_consensus.cc:2272] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.387768 12409 tablet_replica.cc:333] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.388744 12409 raft_consensus.cc:2243] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.389259 12409 raft_consensus.cc:2272] T 17cd77ddaa4c49c69401db747c839a2e P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.391785 12409 tablet_replica.cc:333] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.392830 12409 raft_consensus.cc:2243] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.393335 12409 raft_consensus.cc:2272] T 56f7a265065f4c47b22e0159d88fc262 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.395601 12409 tablet_replica.cc:333] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.396443 12409 raft_consensus.cc:2243] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.397446 12409 raft_consensus.cc:2272] T 06958f3c898c48469fde073474821db4 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.400146 12409 tablet_replica.cc:333] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.401197 12409 raft_consensus.cc:2243] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.401830 12409 raft_consensus.cc:2272] T 194f838ca919474c8162e2def0e3da14 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.404973 12409 tablet_replica.cc:333] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.405905 12409 raft_consensus.cc:2243] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.407081 12409 raft_consensus.cc:2272] T 26e2417c48ff4a298b480607a8d56306 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.410182 12409 tablet_replica.cc:333] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.410791 12409 raft_consensus.cc:2243] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.411301 12409 raft_consensus.cc:2272] T 43741078ec4b4b40bffbb89360aea0c3 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.413800 12409 tablet_replica.cc:333] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.414393 12409 raft_consensus.cc:2243] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.414800 12409 raft_consensus.cc:2272] T bff299a80e7444d09bdb0fe966ace0c4 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.416982 12409 tablet_replica.cc:333] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.417639 12409 raft_consensus.cc:2243] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.418673 12409 raft_consensus.cc:2272] T f4cc5719ad354c1f897af009ef1e528f P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.421299 12409 tablet_replica.cc:333] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.422336 12409 raft_consensus.cc:2243] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.422884 12409 raft_consensus.cc:2272] T d2dfd29191814d62bb243bd4be7fdba2 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.426614 12409 tablet_replica.cc:333] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.427772 12409 raft_consensus.cc:2243] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.428421 12409 raft_consensus.cc:2272] T c9f6e15b98c84ff9984133d1818ec329 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.431083 12409 tablet_replica.cc:333] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.431732 12409 raft_consensus.cc:2243] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.432768 12409 raft_consensus.cc:2272] T a180fcba65e243f2ace7ab30c1128475 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.435266 12409 tablet_replica.cc:333] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.435950 12409 raft_consensus.cc:2243] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.436841 12409 raft_consensus.cc:2272] T df50bd530bcf4f3897564361f46c94c9 P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.439136 12409 tablet_replica.cc:333] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.439739 12409 raft_consensus.cc:2243] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.440207 12409 raft_consensus.cc:2272] T 486278766c1e42e6908c196fff181e29 P 183d64694f2647e1b0c710360b97d497 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.444005 12409 tablet_replica.cc:333] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.444883 12409 raft_consensus.cc:2243] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.445444 12409 raft_consensus.cc:2272] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.447856 12409 tablet_replica.cc:333] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.448477 12409 raft_consensus.cc:2243] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.449295 12409 raft_consensus.cc:2272] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.451479 12409 tablet_replica.cc:333] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.452054 12409 raft_consensus.cc:2243] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.452497 12409 raft_consensus.cc:2272] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 183d64694f2647e1b0c710360b97d497 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.454603 12409 tablet_replica.cc:333] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497: stopping tablet replica
I20260709 13:02:53.455076 12409 raft_consensus.cc:2243] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.455535 12409 raft_consensus.cc:2272] T 7e4e0210defe44639398856436eb821c P 183d64694f2647e1b0c710360b97d497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.500144 12409 tablet_server.cc:196] TabletServer@127.12.30.65:0 shutdown complete.
W20260709 13:02:53.540901 27043 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35259: connect: Connection refused (error 111) [suppressed 16 similar messages]
W20260709 13:02:53.547557 27043 consensus_peers.cc:597] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e -> Peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Couldn't send request to peer 183d64694f2647e1b0c710360b97d497. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.550742 12409 tablet_server.cc:179] TabletServer@127.12.30.66:0 shutting down...
I20260709 13:02:53.590603 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:53.591459 12409 tablet_replica.cc:333] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.592495 12409 raft_consensus.cc:2243] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.593163 12409 raft_consensus.cc:2272] T df50bd530bcf4f3897564361f46c94c9 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.595676 12409 tablet_replica.cc:333] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.596318 12409 raft_consensus.cc:2243] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.596827 12409 raft_consensus.cc:2272] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.599594 12409 tablet_replica.cc:333] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.600343 12409 raft_consensus.cc:2243] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.600894 12409 raft_consensus.cc:2272] T 7e4e0210defe44639398856436eb821c P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:53.602283 27043 consensus_peers.cc:597] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e -> Peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Couldn't send request to peer 183d64694f2647e1b0c710360b97d497. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.603638 12409 tablet_replica.cc:333] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.604352 12409 raft_consensus.cc:2243] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.604773 12409 raft_consensus.cc:2272] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.606912 12409 tablet_replica.cc:333] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.607766 12409 raft_consensus.cc:2243] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.608321 12409 raft_consensus.cc:2272] T 26e2417c48ff4a298b480607a8d56306 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.610611 12409 tablet_replica.cc:333] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.611225 12409 raft_consensus.cc:2243] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.612242 12409 raft_consensus.cc:2272] T 43741078ec4b4b40bffbb89360aea0c3 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:53.612594 27115 consensus_peers.cc:597] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb -> Peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Couldn't send request to peer 948af4c6da234b748ca8952d7b81322e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.614501 12409 tablet_replica.cc:333] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.615231 12409 raft_consensus.cc:2243] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.616061 12409 raft_consensus.cc:2272] T 17cd77ddaa4c49c69401db747c839a2e P 948af4c6da234b748ca8952d7b81322e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.618116 12409 tablet_replica.cc:333] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.618736 12409 raft_consensus.cc:2243] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.619501 12409 raft_consensus.cc:2272] T 56f7a265065f4c47b22e0159d88fc262 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.621521 12409 tablet_replica.cc:333] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.622100 12409 raft_consensus.cc:2243] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.622546 12409 raft_consensus.cc:2272] T 06958f3c898c48469fde073474821db4 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.624788 12409 tablet_replica.cc:333] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.625535 12409 raft_consensus.cc:2243] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.626406 12409 raft_consensus.cc:2272] T 194f838ca919474c8162e2def0e3da14 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:53.628706 27115 consensus_peers.cc:597] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb -> Peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Couldn't send request to peer 948af4c6da234b748ca8952d7b81322e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.629012 12409 tablet_replica.cc:333] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.629746 12409 raft_consensus.cc:2243] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.630617 12409 raft_consensus.cc:2272] T bff299a80e7444d09bdb0fe966ace0c4 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.632817 12409 tablet_replica.cc:333] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.633371 12409 raft_consensus.cc:2243] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.633857 12409 raft_consensus.cc:2272] T f4cc5719ad354c1f897af009ef1e528f P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.636492 12409 tablet_replica.cc:333] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.637454 12409 raft_consensus.cc:2243] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.637867 12409 raft_consensus.cc:2272] T d2dfd29191814d62bb243bd4be7fdba2 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.641381 12409 tablet_replica.cc:333] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.641986 12409 raft_consensus.cc:2243] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.642818 12409 raft_consensus.cc:2272] T c9f6e15b98c84ff9984133d1818ec329 P 948af4c6da234b748ca8952d7b81322e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.645049 12409 tablet_replica.cc:333] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.645593 12409 raft_consensus.cc:2243] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.646055 12409 raft_consensus.cc:2272] T a180fcba65e243f2ace7ab30c1128475 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.648187 12409 tablet_replica.cc:333] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.648692 12409 raft_consensus.cc:2243] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.649571 12409 raft_consensus.cc:2272] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.651824 12409 tablet_replica.cc:333] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.652411 12409 raft_consensus.cc:2243] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.652851 12409 raft_consensus.cc:2272] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 948af4c6da234b748ca8952d7b81322e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.655022 12409 tablet_replica.cc:333] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e: stopping tablet replica
I20260709 13:02:53.655520 12409 raft_consensus.cc:2243] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.656023 12409 raft_consensus.cc:2272] T 486278766c1e42e6908c196fff181e29 P 948af4c6da234b748ca8952d7b81322e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:53.686913 27117 consensus_peers.cc:597] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb -> Peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Couldn't send request to peer 183d64694f2647e1b0c710360b97d497. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.702775 12409 tablet_server.cc:196] TabletServer@127.12.30.66:0 shutdown complete.
W20260709 13:02:53.709084 27117 consensus_peers.cc:597] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb -> Peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Couldn't send request to peer 183d64694f2647e1b0c710360b97d497. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:53.733708 27117 consensus_peers.cc:597] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb -> Peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Couldn't send request to peer 183d64694f2647e1b0c710360b97d497. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.745012 12409 tablet_server.cc:179] TabletServer@127.12.30.67:0 shutting down...
W20260709 13:02:53.745103 27117 consensus_peers.cc:597] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb -> Peer 183d64694f2647e1b0c710360b97d497 (127.12.30.65:35259): Couldn't send request to peer 183d64694f2647e1b0c710360b97d497. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.65:35259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.790271 12409 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:53.791180 12409 tablet_replica.cc:333] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.792186 12409 raft_consensus.cc:2243] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.792958 12409 raft_consensus.cc:2272] T df50bd530bcf4f3897564361f46c94c9 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.795871 12409 tablet_replica.cc:333] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.796717 12409 raft_consensus.cc:2243] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.797271 12409 raft_consensus.cc:2272] T 8c9a97ce1cbc4fc7afa5f7e6f9667e35 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.799703 12409 tablet_replica.cc:333] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.800467 12409 raft_consensus.cc:2243] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.800987 12409 raft_consensus.cc:2272] T 17cd77ddaa4c49c69401db747c839a2e P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.803309 12409 tablet_replica.cc:333] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.804044 12409 raft_consensus.cc:2243] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.805027 12409 raft_consensus.cc:2272] T 7e4e0210defe44639398856436eb821c P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.807350 12409 tablet_replica.cc:333] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.808176 12409 raft_consensus.cc:2243] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.809094 12409 raft_consensus.cc:2272] T 3511bbeafe1f4e65ba4b3148f4dfac2c P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.811326 12409 tablet_replica.cc:333] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.812072 12409 raft_consensus.cc:2243] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.812644 12409 raft_consensus.cc:2272] T f4cc5719ad354c1f897af009ef1e528f P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.814927 12409 tablet_replica.cc:333] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.815564 12409 raft_consensus.cc:2243] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.816053 12409 raft_consensus.cc:2272] T bff299a80e7444d09bdb0fe966ace0c4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.818248 12409 tablet_replica.cc:333] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.818890 12409 raft_consensus.cc:2243] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.819346 12409 raft_consensus.cc:2272] T 26e2417c48ff4a298b480607a8d56306 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.821473 12409 tablet_replica.cc:333] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.822140 12409 raft_consensus.cc:2243] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.822660 12409 raft_consensus.cc:2272] T 43741078ec4b4b40bffbb89360aea0c3 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.824842 12409 tablet_replica.cc:333] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.825450 12409 raft_consensus.cc:2243] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.826264 12409 raft_consensus.cc:2272] T c9f6e15b98c84ff9984133d1818ec329 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.829314 12409 tablet_replica.cc:333] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.830175 12409 raft_consensus.cc:2243] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.830837 12409 raft_consensus.cc:2272] T a180fcba65e243f2ace7ab30c1128475 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:53.831996 27115 consensus_peers.cc:597] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb -> Peer 948af4c6da234b748ca8952d7b81322e (127.12.30.66:45595): Couldn't send request to peer 948af4c6da234b748ca8952d7b81322e. Status: Network error: Client connection negotiation failed: client connection to 127.12.30.66:45595: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:53.833549 12409 tablet_replica.cc:333] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.834239 12409 raft_consensus.cc:2243] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.835150 12409 raft_consensus.cc:2272] T d2dfd29191814d62bb243bd4be7fdba2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.837982 12409 tablet_replica.cc:333] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.838624 12409 raft_consensus.cc:2243] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.839100 12409 raft_consensus.cc:2272] T 56f7a265065f4c47b22e0159d88fc262 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.841331 12409 tablet_replica.cc:333] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.841897 12409 raft_consensus.cc:2243] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.842353 12409 raft_consensus.cc:2272] T 06958f3c898c48469fde073474821db4 P 5f2ef05158654ce59432eeb8cb5b04bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.844481 12409 tablet_replica.cc:333] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.845036 12409 raft_consensus.cc:2243] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.845505 12409 raft_consensus.cc:2272] T 194f838ca919474c8162e2def0e3da14 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.847781 12409 tablet_replica.cc:333] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.848372 12409 raft_consensus.cc:2243] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.848800 12409 raft_consensus.cc:2272] T c4efc5a1ab5e4577a3ca32495d6e4b04 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.850996 12409 tablet_replica.cc:333] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.851508 12409 raft_consensus.cc:2243] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:53.852029 12409 raft_consensus.cc:2272] T ebc2ac862d0a4a2184e9a3c18a7264a2 P 5f2ef05158654ce59432eeb8cb5b04bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.854173 12409 tablet_replica.cc:333] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb: stopping tablet replica
I20260709 13:02:53.854700 12409 raft_consensus.cc:2243] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:53.855600 12409 raft_consensus.cc:2272] T 486278766c1e42e6908c196fff181e29 P 5f2ef05158654ce59432eeb8cb5b04bb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:53.929461 12409 tablet_server.cc:196] TabletServer@127.12.30.67:0 shutdown complete.
I20260709 13:02:53.977599 12409 master.cc:562] Master@127.12.30.126:37307 shutting down...
I20260709 13:02:54.013576 12409 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:54.014206 12409 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:54.014559 12409 tablet_replica.cc:333] T 00000000000000000000000000000000 P ec147c9ae47b47f7a83895db40063882: stopping tablet replica
I20260709 13:02:54.038857 12409 master.cc:584] Master@127.12.30.126:37307 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (14439 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (29346 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (446936 ms total)
[  PASSED  ] 8 tests.
I20260709 13:02:54.106189 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 442 messages since previous log ~17 seconds ago
I20260709 13:02:54.106462 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 450 messages since previous log ~18 seconds ago
I20260709 13:02:54.106637 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 450 messages since previous log ~18 seconds ago
I20260709 13:02:54.106851 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:02:54.107074 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:02:54.107290 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 2 messages since previous log ~71 seconds ago
I20260709 13:02:54.107569 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~0 seconds ago
I20260709 13:02:54.107878 12409 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~13 seconds ago
